Kalman filter) Observation matrix of measurement equation and what is a good signal?

Question

I am trying to use a Kalman filter, but my data are somewhat deviating from the assumptions.

1. The noises in my measurement equation are not normally distributed. First of all, they are not zero-mean. Secondly, although the second-moment seems okay but there are some outliers. How do I address this? I understand that data often fails to meet assumptions in practice, but I would like to make the noises at least zero-mean. What can I do to remedy this? Can I just subtract non-zero mean from noises? Non-zero mean implies there is a bias in noise dataset so can I just substract the mean value (let's say, 0.8), and compute variance?
2. What is the optimal measurement equation, and how can I determine it?
3. Additionally, I am curious about observation matrix. Suppose a 1-D measurement equation.

$$ S_{t}=hx_{t}+\varepsilon_{t} $$

$S_{t}$ is a signal and I say that $ \varepsilon_{t} \sim N(0, R)$, and $h<0$. In this case, the value of Kalman gain $P_{t|t-1}H^{\top}(HP_{t|t-1}H^{\top}+R)^{-1}<0$. However, I haven't seen a negative Kalman gain. I am wondering if I can't use a signal having a negative relationship with a state variable.

4. The values of Kalman gains are pretty low. They range from 0 to 0.1. It implies that the information in the signal plays little role in estimating a state variable. Does it mean that the signal is useless (not good enough)? Because the signal doesn't have enough information to estimate the state variable?

Answer 1

This question needs a bit more information to provide a complete answer, but I have a few comments if you are still working on this:

1. If your noise is not zero-mean, you can include a bias in your states, then subtract the estimated bias. If you know the bias, then you can simply substract it without issue.
2. There is no "optimal" measurement equation. The measurement equation is part of your model and describes how the measurements relate to your states. If you're learning your model from a different procedure where you have uncertainty in your model parameters, the kalman filter doesn't handle this case.
3. This process model doesn't make sense. Please, write the states (e.g.,$x$) you have along with the process model (e.g.,$f(x)$) as well as the measurements (e.g.,$z$) along with the measurement model (e.g., $z=h(x)$).
4. I wouldn't worry about interpreting the exact values of the Kalman gain. Instead, check 1) if state estimate converges to a reasonable value and 2) if the covariance is decreasing.

I suggest first addressing (3) (which is related to (1) and (2)) because you can't look at the performance if you can't write the process and observation models. Then, look at the performance (4).