Convoluted Derivatives

Question

I'm trying to compute the first two derivatives of the map $$ \Psi:X\mapsto (X^TX + f(X))^{-1}X^Ty, $$ should be; where $y$ is a fixed vector and $X$ is a vector also. Wehere, here $f(X)$ is a matrix-valued twice continuously differentiable function. But I'm not very experienced in matrix calculus.

Answer 1

Following my solution in Matrix Derivative of Tichonov Regularization Operator

Let $f_{ij}' = \partial f/\partial X_{ij}$ and $f_{ij,k\ell}'' = \partial^2 f/\partial X_{ij}\partial X_{k\ell}$. Then $$ \phi_{ij} := \frac{\partial(X^T X+f(X))^{-1}}{\partial X_{ij}} = -(X^T X+f(X))^{-1}(X^T J^{ij} + J^{ji}X + f''_{ij})(X^T X+f(X)). $$ and $$ \psi_{ij} := \frac{\partial(X^T X+f(x))^{-1}X^T y}{\partial X_{ij}} = \phi_{ij}^TX^T y + (X^T X+f(x))^{-1}(y_j\cdot e_i). $$

The second-order derivative can be similarly computed using chain rule: $$ \frac{\partial^2(X^T X+f(x))^{-1}}{\partial X_{ij}\partial X_{k\ell}} = -\phi_{k\ell}^T\left[(X^T J^{ij}+J^{ji}X+f_{ij}')(X^TX+f(X))^{-1}X^T y-(y_i\cdot e_j)\right]\\ -(X^T X+f(X))^{-1}\left[(J^{\ell k}J^{ij}+J^{ji}J^{k\ell} + f_{ij,k\ell}'')(X^T X+f(X))^{-1}X^T y+(X^T J^{ij}+J^{ji}X+f_{ij}')\psi_{k\ell}\right]. $$