Matrix function and its argument

Question

Suppose that $f$ is a function from $\mathbb R^n$ to $\mathbb R^n$ given by $$f(x) = \left(\begin{array}{c} f_1(x) \\ f_2(x) \\ \vdots \\ f_n(x) \end{array}\right).$$ If the $i$th component of $x$ is changed, I am interested in the change in $f_i(x)$. This is straight forward. I now want to transfer the framework to functions $F$ from $\mathbb R^{n\times n}$ to $\mathbb R^{n\times n}$. For simplicity, assume that $X$ and $Y= F(X)$ can be decomposed into $X = P^{-1}\Lambda P$ and $Y = Q^{-1}\Psi Q$ with $\Lambda$, $\Psi$ diagonal. Now I want to identify a change in the $i$th component of $\Lambda$ with one of the elements in $\Psi$. Is this possible at all? If yes, how?

Answer 1

Since you find the vector case "straight forward", why not transform the matrix problem into a vector problem, i.e. $$\eqalign{ x &= {\rm vec}(X) \\ y &= {\rm vec}(Y) &= f(x) \\ J &= \frac{\partial y}{\partial x} \quad&\big({\rm via\,\,straighforward\,\,technique}\big) \\ }$$ Given the decompositions of $(X,Y)$ in terms of $(\Lambda,\Psi)$, you can use the Khatri-Rao product $(\boxtimes)$ to expand $(x,y)$ as matrix-vector products. $$\eqalign{ x={\rm vec}(P^{-1}\Lambda P) &= (P^T\boxtimes P^{-1})\,{\rm diag}(\Lambda)\;\doteq A\,\lambda \\ y={\rm vec}(Q^{-1}\Psi Q) &= (Q^T\boxtimes Q^{-1})\,{\rm diag}(\Psi) \;\doteq B\,\psi\\ }$$ Then use $J$ to calculate the desired gradient. $$\eqalign{ dy &= J\,dx \\ B\,d\psi &= J\big(A\,d\lambda\big) \\ \frac{\partial\psi}{\partial\lambda} &= B^{-1}JA \qquad= \left(Q^{-T}\boxtimes Q\right)JA \\ \frac{\partial\lambda}{\partial\psi} &= A^{-1}J^{-1}B \qquad\big({\rm assuming\,}J^{-1}{\,\rm exists}\big) \\ }$$