What is Kalman filter?

Question

I am not an economics grad. I am an engineer. I was working on beta value in market model regression.This is how I proceed.

Firstly, I calculated daily returns from adjusted closing price of all stock. Then, I calculated daily market returns. This calculation was from period 2000 to 2018.

Based on this calculation, I computed beta value by performing regression analysis for two variables. Simple OLS method.

It is well known fact that beta is not constant. Thus I performed rolling regression for estimating beta. So, I did regression analysis for one year. I got beta value for each company for year 2000,2001,2002,2003.

This exercise proved well known fact that beta is not constant. Beta is time-varying. Thus, I researched and came to know about time varying parameter model.

$$y_t = \alpha_t + \beta_t X_t + \epsilon_t$$

And there are three different equation for beta

$\beta_t=\beta_t-1 + \epsilon_t$ <- Random walk model

$\beta_t=β̄ + ξt$ <- Mean reverting model

I thought this are part of Kalman filter. I read about Kalman filter. There are those matrices and all. How does Kalman filter help in beta value? Please explain me what Kalman filter does in estimating beta ?

Answer 1

The Kalman Filter is a BLUE estimator, a Best Linear Uncorrelated Estimator. It is based on state space estimation, and observations which corrects (or updates) the states. There are various "add-ons" which have been added to take into account noise, the observability and controllability Gramians (based on Lyapunov equations), and so on and so forth.

But what it all boils down to is to best estimate the state of a system.

$$ y_t = Beta_{0,t} + Beta_{1,t} * x_t + v_t (1) $$

Is your base equation, right?

Now, let us see what the Kalman Filter's equations actually are:

$$ x_k = F_k * x_{k-1} + B_k * u_k (2) $$

$$ F_k $$ is your state transition matrix. How do you go from one state, at time K, to another state at time K+1.

I will rewrite the maths of (1) a bit:

$$ x_{k+1} = Beta_{0,k} + Beta_{1,k} * x_k + v_k $$

Or:

$$ x_{k} = Beta_{0,k-1} + Beta_{1,k-1} * x_{k-1} + w_{k-1} (3) $$

With a simple change of variable (k => k-1, v = w)

Assuming that the process noise does not change, we can assert that:

$$ w_k = w_{k-1} $$

Therefore, it yields the following equation:

$$ x_{k} = Beta_{0,k-1} + Beta_{1,k-1} * x_{k-1} + w_{k} (4) $$

Kalman Filter:

$$ x_k = F_k * x_{k-1} + B_k * u_k $$

Looks slightly closer, right? Let's go one step further:

We can say that there is no input control, right? You don't give actual commands to the system you are estimating (slow down, speed up, emmergency stop, etc...)

So we can say that the following equation applies:

$$ B_k = 0 $$

This leads to us eliminating two terms:

$$ x_k = F_k * x_{k-1} $$

It's getting real close right about now, however it's still not exactly similar. Remember how we said there process noise is invariant? Well, guess what, state transition matrix is also time-invariant.

That means that:

$$ F_k = F_{k-1} $$

Such that:

$$ x_k = F_{k-1} * x_{k-1} $$

Now, what if instead of calling the state transition matrix F_k, we name it Beta?

No input control Kalman Filter with variable name replacement:

$$ x_k = Beta_{k-1} * x_{k-1} $$

And your rewritten equation:

$$ x_k = Beta_{0,k-1} + Beta_{1,k-1} * x_{k-1} + w_{k} $$

---

We are on a good way to solving things, but there is still something weird with:

$$ x_k = Beta_{0,k-1} + Beta_{1,k-1} [ommitted] $$

Well if you look at the most used TVP regression, the following equation applies:

$$ y_k = Beta_{0,k} + Beta_{1,k} * x_{1,k} + ... + Beta_{k,k} * x_{k,k} + w_k $$

Now going back to equation (5), which is basically our Kalman Filter equation, right? We can develop all the terms, and call the initial state of the system:

$$ Initial state: Beta_{0,k} $$

Such that our final equation (Kalman Filter):

$$ x_k = Beta_{0,k} + Beta_{1,k} * x_{1,k} + ... + Beta_{k,k} * x_{k,k} $$

In which case, the final answer to your question would be:

$$ F_k (KF) = Beta_{k,k} $$

If you are purely talking about the estimating step (which is not time-dependent,) and:

$$ K_k (KF) = Beta_{k,k} $$

If your system is time-dependent.

This update part is also based on observations, and what we call observability gramian.

I will edit this post if I see a more relevant correlation, but this is the simplest way I could explain.

Answer 2

You wrote that you have a time varying parameter model with the following equation: $$y_t = \alpha_t + \beta_t X_t + \epsilon_t$$

and that you have three different equations for $\beta$

First one: $\beta_t=\beta_t-1 + \epsilon_t$ <- Random walk model

Second one: $\beta_t=β̄ + ξ_t$ <- Mean reverting model

what's the third one?

Nevertheless I will continue ...

To my understanding The Kalman filter is an algorithm that uses three things:

1. a system's dynamic model
2. known control inputs to that system
3. multiple sequential measurements

to calculate an estimation of the system's varying quantities that is better than the estimation which is obtained by using just one measurement.

In other words it is an estimator that filters out the noise and that infers parameters from indirect, inaccurate and uncertain observations. Kalman filters are recursive. That's why I would guess that $y_t = X_t$ $_+$ $_1$ in your equation and that it can also be written as follows:

$X_t$ $_+$ $_1$ = $\alpha_t + \beta_t X_t + \epsilon_t$

or

$X_t$ = $\alpha_t$ $_-$ $_1$ + $\beta_t$ $_-$ $_1$ $X_t$ $_-$ $_1$ + $\epsilon_t$ $_-$ $_1$

Assume now you have dicrete points $k$ in time

You would get $X_k$ = $\alpha_k$ $_-$ $_1$ + $\beta_k$ $_-$ $_1$ $X_k$ $_-$ $_1$ + $\epsilon_k$ $_-$ $_1$

Now assume also that that you have one of your three models for $\beta$ (you provided only two of them in your question but it's fine.) which can be written as the $n$ x $n$ matrix A that relates the state at the previous time step $k-1$ to the state at the current step $k$, in abscence of noise. Note that A might change in reality with each time step. However let's assume it is constant and use a Kalman filter.

Now let's insert A instead of $\beta_t$ which is now constant and equals also $\beta_t$ $_-$ $_1$ in

$X_k$ = $\alpha_k$ $_-$ $_1$ + $\beta_k$ $_-$ $_1$ $X_k$ $_-$ $_1$ + $\epsilon_k$ $_-$ $_1$

we would get:

$X_k$ = $\alpha_k$ $_-$ $_1$ + $A$$X_k$ $_-$ $_1$ + $\epsilon_k$ $_-$ $_1$

Now we define $w_k = \alpha_k - \epsilon_k$

This means that we have: $X_k$ = $A$$X_k$ $_-$ $_1$ + $w_k$ $_-$ $_1$

Now we are done with formulating the problem for the Kalman algoithm as:

$X_k$ = $A$$X_k$ $_-$ $_1$ $+ Bu_k$ $_-$ $_1 +$ $w_k$ $_-$ $_1$ with the special case that the optional $n$x$1$ matrix $B$ that relates the optional control input to the state of the measurement is Zero.

Now you can solve your problem using the following time update and mesurement update equations like described in chapter 4.1. (pages 20 to 24) in the link below: https://www.cs.unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_08.pdf

I hope this helps!

Feel free to contact me if you still have questions or if my explanation and formulation of a Kalman problem out of your problem needs any rectifications...