I am following a textbook derivation for the solution to the LQR problem in optimal control and am having trouble getting the same expression as the author. I think my issue is just in taking the directional derivatives of the quadratic terms. The problem is as follows.
We consider a nonlinear system $$\mathbf{\dot{x}} = \mathbf{f}(\mathbf{x,u}) \qquad \mathbf{x}(t) \in \mathbb{R}^n, \mathbf{u}(t)\in \mathbb{R}^m$$ We seek to minimize the cost function
$$ J = \int_0^{\tau} \frac{1}{2}(\mathbf{x}^T\mathbf{Qx} + \mathbf{u}^T\mathbf{Ru})dt + \frac{1}{2}(\mathbf{x}^T\mathbf{Q_fx}) $$
Where $\mathbf{Q}, \mathbf{Q}_f, \mathbf{R}$ are symmetric and positive-definite. We use a Lagrange multiplier $\mathbf{\lambda}(t) \in \mathbb{R}^n$ and consider the augmented cost function $$H = J +\int_0^\tau \mathbf{\lambda}^T (\mathbf{f}(\mathbf{x,u})-\mathbf{\dot{x}})dt $$ By the Lagrange multiplier theroem, we require the conditions $$ \frac{\partial H}{\partial \mathbf{\lambda}} = 0, \quad\frac{\partial H}{\partial \mathbf{x}} = 0,\quad \frac{\partial H}{\partial \mathbf{u}}=0$$
Where I get lost is in the next step, where the author takes directional derivatives of H with respect to $\mathbf{\lambda}, \mathbf{x}$ and $\mathbf{u}$. The book has
$$ \frac{\partial H}{\partial \mathbf{\lambda}} \cdot \delta \mathbf{\lambda} = \int_0^\tau \delta \mathbf{\lambda}^T(\mathbf{f}(\mathbf{x,u})-\mathbf{\dot{x}}) dt = 0 \quad \forall \delta \mathbf{\lambda}$$ Which of course requires the integrand to be $0$. I (think I) see how this one works out, but where I don't follow is in the calculation of
$$ \frac{\partial H}{\partial \mathbf{u}} \cdot \delta \mathbf{u} = \int_0^{\tau} \left (\frac{1}{2}(\delta \mathbf{u}^T\mathbf{Ru} + \mathbf{u}^T\mathbf{R}\delta \mathbf{u}) + \lambda^T \frac{\partial \mathbf{f}}{\partial \mathbf{u}} \delta \lambda\right)dt$$ And a similar form for $ \frac{\partial H}{\partial \mathbf{x}} \cdot \delta \mathbf{x}$
If anyone could help show how the derivative of the quadratic term $\mathbf{u}^T\mathbf{Ru}$ works out, it would be greatly appreciated. My matrix calculus is very rusty and I never did too much of it in the first place. I imagine my mistake lies in appllying the product rule for these terms.
A more succinct question might be for me to ask how to calculate $$ \frac{\partial}{\partial \mathbf{u}} \left(\mathbf{u}^T\mathbf{Ru} \right) \cdot \delta \mathbf{u} $$
Note: this is my first post on here, so if I am missing any details/context or if there are any formatting errors, etc. please let me know.
Thank you in advance!!