Consider the state-space system:
\begin{align*} \left[\begin{array}{c} 1\\ x_{t+1}\\ y_{t+1} \end{array}\right] & =F\left[\begin{array}{c} 1\\ x_{t}\\ y_{t} \end{array}\right]+w_{t+1},\quad w_{t+1}\sim\left(0,W\right)\\ \left[\begin{array}{c} 1\\ x_{t}\\ \widehat{y}_{t} \end{array}\right] & =H\left[\begin{array}{c} 1\\ x_{t}\\ y_{t} \end{array}\right]+v_{t},\quad v_{t}\sim(0,V) \end{align*}
with $H=\left(\begin{array}{ccc} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{array}\right)$, $W=\left(\begin{array}{ccc} 0 & 0 & 0\\ 0 & 0 & 0\\ 0 & 0 & \sigma_{y}^{2} \end{array}\right)$, and $V=\left(\begin{array}{ccc} 0 & 0 & 0\\ 0 & 0 & 0\\ 0 & 0 & \sigma_{v}^{2} \end{array}\right).$
So the state space includes the constant 1 and $x_t$ is observed while $y_t$ is not.
The steady-state Kalman filter gives the mean-squared error $\mathbf{P}$ for the state estimate as:
$\mathbf{P}=\mathbf{F}\left[\mathbf{P}-\mathbf{P H}\left(\mathbf{H}^{\prime} \mathbf{P H}+\mathbf{V}\right)^{-1} \mathbf{H}^{\prime} \mathbf{P}\right] \mathbf{F}^{\prime}+\mathbf{W}$
The matrix $\left(\mathbf{H}^{\prime} \mathbf{P H}+\mathbf{V}\right)^{-1}$ is not invertible. How does one solve for $\mathbf{P}$ in this case?
First, if 1 is a constant, you should exclude from your model.
The model you presented include observations $x_t$, and $y_t$, so you observation model is incorrect. I suggest writing out your observation model as a function for each observation, then convert to matrix representation.
Let the state vector be denoted by $\mathbf{x}_t = [x_t, y_t]^T$. If your system is observing $y_t$ directly, the observation model is the following: $$ h(\mathbf{x}_t, \mathbf{v}_t)=h(x_y,y_t,v_t) = y_t + v_t $$
Now, you compute the Jacobian. Even though the system is linear, you can still compute the Jacobian of your observation model to obtain $\mathbf{H}$ as it is an easy procedure to obtain $\mathbf{H}$ correctly. Note, the Jacobian is simply equivalent to the observation matrix $\mathbf{H}$ for a linear system.
Now, $$ \mathbf{H} = \frac{\partial h}{\partial \mathbf{x}} = \left(\begin{array}{cc} \frac{\partial h}{\partial x} & \frac{\partial h}{\partial y} \end{array}\right) = \left(\begin{array}{cc} 0 & 1 \end{array}\right) $$
The expression you are concerned with should always be invertible if the system is modeled correctly. $\mathbf{H}$ is a $m \times n$ matrix where $m$ is the number of observations and $n$ is the number of states. In general, $\mathbf{V}$ must be positive definite, and $\mathbf{H} \mathbf{P} \mathbf{H}'$ is a positive semidefinite matrix as the rank should be equal to the number of rows. Since a positive definite matrix plus a positive semidefinite matrix is a positive definite matrix, your expression is invertible because a positive definite matrix is invertible.
Edit: Corrected a few typos.
Edit 2: You also have your transposes backwards for $\mathbf{H}$, you should have $\mathbf{H} \mathbf{P} \mathbf{H}'$.