Derivative of a function involving diagonal matrix

Question

Let $A$ be a $n\times n$ matrix, $\text{diag}(x)$ is the diagonal matrix with $x$ on the diagonal. How can I find $dF(x)$ for $F(x) = \text{diag}(x)Ax$?

Thank you very much in advance!

Answer 1

I assume that $x = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ is a column vector so $F \colon \mathbb{R}^n \rightarrow \mathbb{R}^n$. The differential $dF(x)$ must satisfy

$$ \lim_{h \to 0} \frac{F(x + h) - F(x) - dF(x)h}{\| h\|} = 0. $$

We have

$$ F(x + h) - F(x) = \operatorname{diag}(x + h)A(x + h) - \operatorname{diag}(x)Ax = \\ (\operatorname{diag}(x) + \operatorname{diag}(h))(Ax + Ah) - \operatorname{diag}(x)Ax = \\ \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax + \operatorname{diag}(h)Ah.$$

The term $\operatorname{diag}(h)Ah$ is quadratic in $h$ so we can guess that $dF(x)h = \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax$ and prove it:

$$ \frac{\| F(x+h) - F(x) - \operatorname{diag}(x)Ah + \operatorname{diag}(h)Ax \|}{\| h \|} = \frac{\| \operatorname{diag}(h)Ah \|}{\| h \|} \leq \frac{ \| \operatorname{diag}(h) \| \| A \| \| h \|}{\| h \|} = \\ \| \operatorname{diag}(h) \| \| A \| \xrightarrow[h \to 0]{} 0.$$

Answer 2

$ \def\d#1{\operatorname{Diag}\left(#1\right)} $If you use the Hadamard (aka elementwise) product, then you can get rid of the diag operation, which will make it easier to find the differential and jacobian of the function. $$\eqalign{ f &= x\circ(Ax) \\ &= \d{Ax}\,x \;=\; \d{x}\,Ax \\ df &= \d{Ax}\,dx + \d{x}\,A\,dx \\ &= \Big(\d{Ax} + \d{x}\,A\Big)\,dx \\ \frac{\partial f}{\partial x} &= \d{Ax} + \d{x}\,A \\\\ }$$ Special properties of the Hadamard product were used in several steps, i.e. $$\eqalign{ x\circ y & = y\circ x \\ &= \d{x}\,y \\ &= \d{y}\,x \\ }$$