Consider the controlled system $x_{t+1} = x_t + u_t + 3\epsilon_{t+1}$, where the $\epsilon_t$ are independent $N(0,1)$ variables. The instantaneous cost at time t is $x_t^2 + 2u_t^2$. Assuming that $x_t$ is observable at time $t$, calculate the optimal control under steady-state (stationary) conditions and find the expected cost per unit time incurred when this control is used.
Suppose now that at time $t$ one observes, not $x_t$, but $y_t = x_{t-1} + 2\eta_t$, where the $\eta_t$ are again independent $N(0,1)$ variables independent of the $\epsilon_t$. Show that the law of $\hat{x_t}$ conditional on $(y_1,...,y_t)$ has steady-state variance 12.
Find the optimal control and a recursion for the optimal state estimate under stationary conditions.