What's the derivative of an inner product with respect to the inner product matrix?

Question

When you have an expression like this:

$$g \left ( \mathbf{X} \right ) = \mathbf{a}^T \mathbf{X} \mathbf{b},$$

where $\mathbf{a} \in \mathbb{R}^d$, $\mathbf{b} \in \mathbb{R}^e$ and $\mathbf{X} \in \mathbb{R}^{d \times e}$,

What's the matrix derivative (Jacobian?) of the expression with respect to $\mathbf{X}$, i.e. $\frac{\partial g}{\partial \mathbf{X}} = \frac{\partial \left ( \mathbf{a}^T \mathbf{X} \mathbf{b} \right )}{\partial \mathbf{X}}$?

My guess is that it is $\mathbf{a} \mathbf{b}^T$.

Answer 1

This map is linear in $X$, so it's derivative is the map itself: $$Dg(X)Y = g(Y) = a^T Y b$$ As a proof just write $$g(X+Y) - g(X) = a^T(X+Y)b - a^T X b = a^TYb$$ and apply the definition of the derivative of a multivariate function.

Answer 2

You can find a ton of matrix derivative identities here: https://en.wikipedia.org/wiki/Matrix_calculus

For your question, look in the scalar-by-matrix derivative formulas section.