I have designed a Kalman filter which has 2 models.
For switching between models in the Kalman filter, I want to use a Chi-squared $ χ^2 $ test based on the innovation $ ν(k) $ of the Kalman filter, which k is step time.
Apparently, I should use normalized innovation squared (NIS)
$ NIS = ν(k)^T * [H_kP^-P^T+R] * ν(k) $
in a hypothesis test. It should be compared with a threshold $χ^2_U$ that has a standard table ( Based on Degree of freedom and significance level ). Also, I use the $Chi2inv()$ command in MATLAB for the calculation of the threshold.
If $NIS>χ^2_U$ , the first model is rejected and I should switch to another model.
But my $NIS$ value is always less than $χ^2_U$ because $NIS<10^{-20}$, which makes it impossible to use the Chi-squared test.
How should I overcome this problem?
I'm a little confused by your notation, it could be correct but is of a form I haven't seen before. You could try using the NIS formulation given by Bar-Shalom in Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software.
There, the NIS parameter should equal $e^T_{z,k}P^{-1}_{k|k}e_{z,k}$. Where the measurement innovation $e_{z,k}$ is defined as $e_{z,k} = z_k - H\hat{x}_{k|k-1}$. $P_{k|k}$ is the estimate covariance, $z_k$ is the measurement vector, $H$ is the observation matrix, and $\hat{x}_{k|k}$ is the a priori state estimate.
You should consider using both the upper and lower $\chi^2$ bounds by the way, since $NIS < \chi^2_{lower}$ could indicate that your filter is behaving overly conservatively and is perhaps overly-tuned or that your process or measurement noises are inappropriately sized.