Taylor expansion with respect to a matrix argument

Question

Suppose $J:\mathbb{R^{m \times n}} \rightarrow \mathbb{R} $ and that $\overline{X}, \delta X \in \mathbb{R^{m \times n}}$. What is the Tyalor series expension of $$J(\overline{X}+\delta X)$$? I'm looking for a similar expression as the following. Suppose $f:\mathbb{R^{m}} \rightarrow \mathbb{R} $. Then we can write $$f(\overline{x}+\delta x)=f(\overline{x})+\nabla{f}(\overline{x})^T \delta x+\frac{1}{2} \delta x ^T \nabla{f}^2(\overline{x})\delta x +....$$ However, for a scalar J with matrix argument, I clearly cannot write $$J(\overline{X}+\delta X)=J(\overline{X})+\nabla{J}(\overline{x})^T\delta X+....$$ because it does not make sense dimensionally. Should I use the Frobenius product?

Answer 1

Yes, you should use the Frobenius product.

The gradient of the function in component form can be written $$G_{ij} = \frac{\partial J}{\partial X_{ij}}$$ The Taylor expansion in component form is $$\eqalign{ dJ &= \sum_{ij} G_{ij}\,dX_{ij} \cr }$$ or in matrix notation $$\eqalign{ J(X+dX) &= J(X) + G:dX \cr }$$ To incorporate the Hessian (which is a 4th order tensor), you would write $$\eqalign{ J(X+dX) &= J(X) + G:dX + \frac{1}{2}dX:{\mathbb H}:dX \cr\cr }$$ Extending this idea in another direction. If $X$ were a 3rd order tensor rather than a matrix, then you would have to use triple-dot products. The Taylor expansion would become $$\eqalign{ G_{ijk} &= \frac{\partial J}{\partial X_{ijk}}\cr dJ &= \sum_{ijk} G_{ijk}\,dX_{ijk} \cr J(X+dX) &= J(X) + G \therefore dX \cr }$$