Partial derivative of $x_1^TAB^Tx_2$

Question

Let $A$ and $B$ be orthogonal matrices, and $x_1$, $x_2$ be vectors. What are:

$$ \dfrac{\partial}{\partial x_1}\left( x_1^TAB^Tx_2 \right)$$

$$ \dfrac{\partial}{\partial x_2}\left( x_1^TAB^Tx_2 \right)$$

Are they

$$\dfrac{\partial}{\partial x_1}\left( x_1^TAB^Tx_2 \right) = AB^Tx_2 $$

$$\dfrac{\partial}{\partial x_2}\left( x_1^TAB^Tx_2 \right) = B A^Tx_1 $$

Answer 1

Yes, with the convention that all results are column vectors, then what you have written is true. In fact, for any constant matrix $A$ and constant column vector $b$,

$$ \frac{\partial}{\partial x} x^T A b = Ab $$

Thus, we have

$$ \frac{\partial}{\partial x_1} x_1^T AB^T x_2 = \frac{\partial}{\partial x_1} x_1^T (AB^T) x_2 = AB^T x_2 $$

Since $ x_1^T AB^T x_2 $ is a scalar, $$ x_1^T AB^T x_2 = (x_1^T AB^T x_2)^T = x_2^T BA^T x_1 $$

and so

$$ \frac{\partial}{\partial x_2} x_1^T AB^T x_2 = \frac{\partial}{\partial x_2} x_2^T BA^T x_1 = BA^T x_1 $$

Whether $A, B$ are orthogonal is irrelevant.

Answer 2

Let

$$f ({\rm x}, {\rm y}) := {\rm x}^\top {\rm A} \, {\rm B}^\top {\rm y} = \langle {\rm x} , {\rm A} \,{\rm B}^\top {\rm y} \rangle = \langle {\rm B} \,{\rm A}^\top {\rm x}, {\rm y} \rangle$$

Hence,

$$\begin{aligned} \nabla_{{\rm x}} f ({\rm x}, {\rm y}) &= \color{blue}{{\rm A} \,{\rm B}^\top {\rm y}} \\ \nabla_{{\rm y}} f ({\rm x}, {\rm y}) &= \color{blue}{{\rm B} \,{\rm A}^\top {\rm x}} \end{aligned}$$