Differentiation of a function defined over $\mathbb{R}^{m^{2}}$ (matrix spaces).

Question

For $1\leq i,j\leq m$, let $f_{ij}:\mathbb{R}^{m^{2}} \to \mathbb{R}$ defined by $f_{ij}(X)V = (X^{2})_{ij}$. For any $X,V \in \mathbb{R}^{m^{2}}$, show that $df_{ij}(X)V$ is the $ij$-th element of the matrix $XV + VX$. Get a similar result with $X^{3}$. Generalize.

I don't want an answer. Actually, I didn't understand how this function works. I would like an explanation to understand this function and a small hint to start solving.

Answer 1

Recall the definition of the differential. $df_{ij}(X) : \mathbb{R}^{m^2} \to \mathbb{R}$ is the unique linear map such that

$$\lim_{H \to 0} \frac{\left|f_{ij}(X + H) - f_{ij}(X) - df_{ij}(X)H\right|}{\|H\|} = 0$$

where $\|\cdot\|$ is the euclidean norm on matrices.

It suffices to verify that $V \mapsto (XV + VX)_{ij}$ satisfies the above limit.

\begin{align} \lim_{H \to 0} \frac{\left|f_{ij}(X + H) - f_{ij}(X) - (XH + HX)_{ij}\right|}{\|H\|} &= \lim_{H \to 0} \frac{\left|((X + H)^2)_{ij} - (X^2)_{ij} - (XH + HX)_{ij}\right|}{\|H\|}\\ &= \lim_{H \to 0}\frac{\left|(X^2 + XH + HX + H^2)_{ij} - (X^2)_{ij} - (XH + HX)_{ij}\right|}{\|H\|}\\ &= \lim_{H\to 0} \frac{\left|(H^2)_{ij}\right|}{\|H\|}\\ &\le \lim_{H\to 0} \frac{\|H^2\|}{\|H\|}\\ &\le \lim_{H\to 0} \frac{\|H\|^2}{\|H\|}\\ &= \lim_{H\to 0} \|H\|\\ &= 0 \end{align}

Therefore $df_{ij}(X)V = (XV + VX)_{ij}$.

Answer 2

We can actually calculate $df_{ij}(X)\cdot V$. Remember that $df_{ij}(X)\cdot V=\langle\text{grad } f_{ij}(X),V\rangle=\dfrac{\partial f_{ij}}{\partial V}(X)$, then

\begin{align*} df_{ij}(X)\cdot V&=\dfrac{\partial f_{ij}}{\partial V}(X)=\lim_{t\to0}\dfrac{f_{ij}(X+tV)-f_{ij}(X)}{t}\\&=\lim_{t\to0}\dfrac{(X^2+tXV+tVX+t^2V^2)_{ij}-(X^2)_{ij}}{t}\\ &=\lim_{t\to0}\dfrac{t(XV+VX)_{ij}+t^2(V^2)_{ij}}{t}=(XV+VX)_{ij} \end{align*}

It's analogous for $f_{ij}(X)=(X^3)_{ij}$, you just need to observe that $$([X+V]^3)_{ij}-(X^3)_{ij}=t(X^2V+XVX+VX^2)_{ij}+t^2(XV^2+VXV+tV^2)_{ij}$$

The geral answer can be found after suspecting for $g_k(X)=X^k$ we have $$dg_k(X)\cdot V=\displaystyle\sum_{i=1}^kX^{i-1}VX^{k-i}$$ Using induction : \begin{align*} dg_{k+1}(X)\cdot V=\dfrac{\partial g_{k+1}(X)}{\partial V}&=\dfrac{\partial(XX^k)}{\partial V}=\dfrac{\partial (Xg_k(X))}{\partial V}\\&=\dfrac{\partial X}{\partial V}\cdot g_k(X)+X\dfrac{\partial g_k(X)}{\partial V}\\&=VX^k+Xdg_k(X)\cdot V\\&=VX^k+X\displaystyle\sum_{i=1}^kX^{i-1}VX^{k-i}=\sum_{i=1}^{k+1}X^{i-1}VX^{k+1-i}\end{align*}