Given a matrix $\text{A} \in \mathbb{R}^{m \times n}$, what are the derivatives of the squared Frobenius norm of its left and right Moore-Penrose pseudo-inverse with respect to the $\text{A}$?
- $\frac{\partial}{\partial \text{A}} \left \Vert \left(\text{A}^T \text{A} \right)^{-1}\text{A}^T \right\Vert_F^2$
- $\frac{\partial}{\partial \text{A}} \left \Vert \text{A}^T \left(\text{A} \text{A}^T \right)^{-1} \right\Vert_F^2$
Let's use a colon to denote the trace/Frobenius product, i.e. $$\eqalign{ X:Y = {\rm tr}(X^TY) \cr }$$ Let's also define the symmetrization function $$\eqalign{ {\rm sym}(X) = \frac{1}{2}(X+X^T) \cr }$$ and some new matrix variables $$\eqalign{ S &= A^TA &\implies S^T = S \cr B &= S^{-1}A^T &\implies BA = I \cr }$$ Write the first function in terms of these new variables, then find its differential and gradient $$\eqalign{ \beta &= \|B\|_F^2 = B:B \cr\cr d\beta &= 2B:dB \cr &= 2B:\Big(S^{-1}\,dA^T - (S^{-1}\,dS\,S^{-1})A^T\Big) \cr &= 2S^{-1}B:dA^T - 2S^{-1}BAS^{-1}:dS \cr &= 2B^TS^{-1}:dA - 2S^{-2}:2\,{\rm sym}(A^T\,dA) \cr &= 2AS^{-2}:dA - 4\,{\rm sym}(S^{-2}):A^T\,dA \cr &= -2AS^{-2}:dA \cr &= -2A(A^TA)^{-1}(A^TA)^{-1}:dA \cr\cr \frac{\partial\beta}{\partial A} &= -2A(A^TA)^{-1}(A^TA)^{-1} \cr\cr }$$ The second function can be handled similarly $$\eqalign{ R &= AA^T \cr C &= A^TR^{-1} \cr \gamma &= C:C \cr \frac{\partial\gamma}{\partial A} &= -2(AA^T)^{-1}(AA^T)^{-1}A \cr }$$