Product rule for matrix-valued and vector-valued functions

Question

Given $g: \mathbb{R}^n \to \mathbb{R}^{n \times m}$ and $f: \mathbb{R}^n \to \mathbb{R}^{m}$ how can we compute $\nabla (g(x)f(x))$?

Answer 1

The $(i,j)$-term of $\nabla(fg)$ is given by \begin{align} \partial_i((g(x)f(x))_j) &= \partial_i\left(\sum_{k=1}^n g_{kj}(x)f_k(x)\right)\\ &= \sum_{k=1}^n (\partial_ig_{kj})(x)f_k(x) + \sum_{k=1}^n g_{kj}(x)(\partial_i f_k)(x)\\ &= ((\partial_ig)(x)f(x))_j + (g(x)(\partial_i f)(x))_j \end{align} so \begin{align*} \nabla(g(x)f(x)) &= \begin{bmatrix} \partial_i((g(x)f(x))_j)\end{bmatrix}_{i,j=1}^n \\ &= \begin{bmatrix} (\partial_1 g)(x)f(x)\\ \vdots \\ (\partial_n g)(x)f(x)\end{bmatrix} + \begin{bmatrix} g(x)(\partial_1f)(x)\\ \vdots \\ g(x)(\partial_nf)(x)\end{bmatrix}\\ &= \begin{bmatrix} (\partial_1 g)(x)f(x)\\ \vdots \\ (\partial_n g)(x)f(x)\end{bmatrix} + g(x)(\nabla f(x))^T \end{align*} where $(\partial_i g)(x)$ is $\partial_i$ applied to the matrix $g$ elementwise.

Answer 2

Let's use a convention in which an uppercase Latin letter denotes a matrix and a lowercase letter denotes a vector.

Since you've told us nothing about the functions $(f,G)\,$ I'll assume that you are able to calculate their gradients without any difficulties $$\eqalign{ g &= {\rm vec}(G) \\ A &= \frac{\partial f}{\partial x} &\quad\implies\quad &df = A\,dx,\quad \\ B &= \frac{\partial g}{\partial x} &\quad\implies &dg = B\,dx,\quad \\ }$$ The product of the two function is a vector, whose gradient can be calculated (with the aid of the Kronecker product).
$$\eqalign{ h &= Gf \;\,=\; (f^T\otimes I_n)\,g \\ dh &= G\,df + (f^T\otimes I_n)\,dg \\ &= GA\,dx + (f^T\otimes I_n)B\,dx \\ \frac{\partial h}{\partial x} &= GA + (f^T\otimes I_n)B \\ }$$