I have the following equation:
$$ \begin{equation} x=(A^TA)^{-1}A^Tb \end{equation} $$
and I need to calculate the derivative of k-th entry of x with respect to A:
$$ \begin{equation} \frac{\partial x(k)}{\partial A} \end{equation} $$
I really don't know how to solve it! Can someone help me? Thank you!
Straightforward calculation shows that \begin{align} dx(k) &= d\left(e_k(A^TA)^{-1}A^Tb\right)\\ &= e_k^T d(A^TA)^{-1}A^Tb + e_k^T (A^TA)^{-1} dA^Tb\\ &= -e_k^T (A^TA)^{-1} d(A^TA) (A^TA)^{-1}A^Tb + e_k^T (A^TA)^{-1} dA^Tb\\ &= -e_k^T (A^TA)^{-1} (dA^T A + A^TdA) (A^TA)^{-1}A^Tb + e_k^T (A^TA)^{-1} dA^Tb\\ \end{align} Therefore, the $(i,j)$-th entry of $\frac{\partial x(k)}{\partial A}$ is given by $$ -e_k^T (A^TA)^{-1} (E_{ji} A + A^TE_{ij}) (A^TA)^{-1}A^Tb + e_k^T (A^TA)^{-1} E_{ji}b. $$