How to compute the Jacobian of a function with an Hadamard division?

Question

I have a function

$$ f: \mathbb{R}^n \mapsto \mathbb{R}^n \\ f(x) = A^{-1}(b \ \oslash \ x) $$

where $A \in \mathbb{R}^{n\times n}$ and $b, x \in \mathbb{R}^n$, and I cannot figure out how to compute the Jacobian of $f$ by using Matrix calculus

Answer 1

$ \def\o{{\tt1}} \def\d{{\rm diag}} \def\D{{\rm Diag}} \def\c#1{\color{red}{#1}} $There is a well known trick for replacing the Hadamard product of vector by the regular matrix product with a diagonal matrix generated by the vector, and for recovering the vector from the diagonal matrix by multiplying by the all-ones vector. $$\eqalign{ B &= \D(b) \\ Bc &= b\odot c \\ b &= B\o \\ }$$ This trick can be extended to Hadamard division by using the matrix inverse (assuming that the vector does not have any components equal to zero). $$\eqalign{ c\oslash b \;=\; B^{-1}c \;=\; \left(B^{-1}C\right)\o }$$ Applying this trick to the current problem yields $$\eqalign{ f &= A^{-1}(b\oslash x) \\&= A^{-1}BX^{-1}\o \\ df &= A^{-1}B\,\c{dX^{-1}}\o \\ &= A^{-1}B\c{\left(-X^{-1}\,dX\,X^{-1}\right)}\o \\ &= -A^{-1}BX^{-2}\,dx \\ \frac{\partial f}{\partial x} &= -A^{-1}BX^{-2} \\ }$$ where the part in red is a well-known result from matrix calculus. Also the fact that diagonal matrices like $(B,X)$ commute with one another has been utilized in several steps.

Answer 2

So let's break this down, you have $f=g\circ h$ where

\begin{align} &g:{\bf R}^n\rightarrow{\bf R}^n\;\;\;g(x) = A^{-1}x\\ &h:{\bf R}^n\rightarrow{\bf R}^n\;\;\;h(x) = b\oslash x \end{align}

By the chain rule, $f' = (g'\circ h)\cdot h'$, where $g'$ and $h'$ are both $n\times n$ matrices, and where $f'$ will also be an $n\times n$ matrix.

Since $g'$ is a constant, this reduces to $f'=g' \cdot h'$, where $h'$ will be a diagonal matrix since $\oslash$ acts coordinate-wise on $x$, can you take it from here?