The estimation error in Kalman filter

Question

I aim to understand whether the estimation error in a standard Kalman filter with "active noise" can be zero (at some time).

More precisely, a state-space model is given by \begin{align} x_{t+1}&= Fx_t + G w_t\\ y_t&= Hx_t + v_t \end{align} where $w_t,v_t$ are pairwise independent and are i.i.d. according to $w_t\sim N(0,W)$ and $v_t\sim N(0,V)$. Denote the estimation error covariance as: $$\Sigma_{t|t-1} = \mathbf{cov}(x_t-\mathbb{E}[x_t|y^{t-1}]).$$

It is clear that $\Sigma_{t|t-1}$ is positive semidefinite.

1. If we assume $W,V\succ0$, can we show that $\Sigma_{t|t-1}$ is positive definite at all times? in other words, there will always be an estimation error. (I even think that $W\succ0$ is sufficient)
2. If the correlation $\mathbb{E}[w_tv_t^T]\neq0$, does it change the answer?

Answer 1

In general, $w$ and $v$ are assumed to be uncorrelated, so that doesn't change the answer. The covariance of the Kalman filter (KF) can be written concisely as follows:

$$ \Sigma_{k}^{-1} = \bigr( F_k \Sigma_{k-1} F_k^T + W \bigr)^{-1} + H^T_k V^{-1} H_k $$

where this form is can be computed using the Woodbury Matrix Identity with the most popular form of the KF equations. This form isn't necessary but is the most simple for answering your question.

So, if we assume that $W \succ 0$ and $V \succ 0$ are positive definite, then the covariance matrix as computed by the KF will also be positive definite. This is because of the following properties:

1. The inverse of a positive definite matrix is positive definite, so if $A$ is positive definite, then $A^{-1}$ is also positive definite.
2. If $A$ is $n \times n$ positive definite matrix, and $B$ is an $m \times n$ matrix with rank $m$, then $C = BAB^T$ is a positive definite matrix.
3. If $A$ is an $n \times n$ positive definite matrix and $B$ is an $n \times n$ positive semi-definite matrix, then $A + B$ is positive definite.

This means that both terms in the previous equation are positive definite; thus, the sum of the terms is also positive definite. Therefore, as long as $W$ and $V$ are positive definite, then the covariance matrix will also be positive definite.