LQR problem with interaction term between state and control

Question

Consider the usual linear process $x_{t+1} =Ax_t+Bu_t+Cw_{t+1}$ where $w_t$ is an independent and identically distributed $N(0,I)$ process. The objective is $$ V=E\sum_{t=0}^\infty \beta^t(x_t'Qx_t + u_t'Ru_t + x_t' S u_t) $$ The problem is identical to a stochastic LQR, but with the additional state-control interaction term $x_t' S u_t$. Is there a way to reduce this problem to the usual LQR formulation? Alternatively, is there a reference treating this general case?

Answer 1

Remark: you want to put a 2 your $x_t^\top S u_t$ term to save some headaches later, and throughout my hints I assume there is 2 on the cross term. I will also neglect the discounting factor $\beta$.

This is known as LQR with a cross term (or cross coupled LQR). There are a few ways to approach solving the problem: working out the solution through Hamilton-Jacobi-Bellman, forming the Hamiltonian, or "guessing" at the solution. For the sake of simplicity, I will go with the last one.

Start by defining $$ u_1=u_t+R^{-1} S^\top x_t, $$ then define your dynamics and cost in terms of $u_1$. This will yield a new $A$ matrix: $$\bar{A}=A-BR^{-1}S^\top,$$ and a new $Q$ matrix: $$\bar{Q}=Q-SR^{-1}S^\top.$$ This provides us with a new condition that needs to be satisfied: $(A-BR^{-1}S^\top,Q-SR^{-1}S^\top)$ is detectable.

Now with $\bar{A}$, $\bar{Q}$, and the resulting modified cost for $u_1$, we get back to a known LQR without cross terms and $$ u_t=-\underbrace{R^{-1}(S^\top+B^\top P)}_{K}x_t. $$

As for a reference, I have to imagine it is hidden away in one of Bertsakas' books or Kumar and Varaiya.