How can I derive the matrix?

Question

I have questions about matrix differentiation problem.

x∈ℜⁿ, A∈ℜ^m∗n

x is an n*1 vector, A is an m*n matrix.

How to derive the following equation?

$$\frac{dAx}{dA}$$

not the equation $$\frac{dAx}{dx}$$

Thanks for reading my questions.

Answer 1

The function $$P_x: A\longmapsto Ax$$ is obviously linear, so the Freched differential of $P_x$ in any point is again $P_x$: $$DP(A_0) = P_x.$$

Answer 2

You have a function

$$ f: \mathbb{R}^{m \times n} \rightarrow \mathbb{R}^n : A \mapsto Ax $$

with fixed $x$. Hence, your derivative must be a mapping

$$ Df: \mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \rightarrow \mathbb{R}^n , (A_0, H) \mapsto Df(A_0)[H], $$

which is linear in $H$, and that satisfies the 1st order Taylor equation

$$ f(A_0 + dA) = f(A_0) + Df(A_0)[dA] + o(dA) . $$

If we insert our function we get

$$ f(A_0 + dA) = A_0 x + dA x = f(A_0) + dA x. $$

So $Df(A_0)[dA] := dA x$ does the job. If you are looking for the derivative written as tensor you can write the last formula via Einstein summation:

$$ Df(A_0)[dA] _{i} := dA_{i,k} x_{k} = \delta_{i,j} dA_{j,k} x_k = (\delta_{i,j} x_k) dA_{j,k} $$

So the derivative $Df(A_0)$ can be identified with the 3-dimensional tensor $T \in \mathbb{R}^{m \times m \times n}$ given by

$$T_{i,j,k} = \delta_{i,j} x_k. $$

Hereby $$\delta_{i,j}:= \begin{cases} 1, &i=j\\ 0, &i\neq j \end{cases}.$$