What's $\frac{\partial}{\partial A}A?$

Question

What's $\frac{\partial}{\partial A}A?$

In this Python App it says that it's $I\otimes I$, but how can it be?

$\frac{\partial}{\partial A}A=\left[\frac{\partial}{\partial A_{ij}}A\right]=[\frac{\partial}{\partial A_{ij}}A_{lk}]$

that equals to 1 only when $l=i$ and $k=j$,i.e.

$\frac{\partial}{\partial A_{ij}}A$ is a matrix of $A$'s dimensions, but only one entry is 1. All the others are zero...

Answer 1

The python app cannot handle higher-order tensors, so it uses vectorization to flatten everything into vectors and proceeds as follows. $$\eqalign{ A &= IAI \cr {\rm vec}(A) &= (I\otimes I)\,{\rm vec}(A) \cr a &= (I\otimes I)\,a \cr da &= (I\otimes I)\,da \cr \frac{\partial a}{\partial a} &= I\otimes I \cr }$$ The tensor gradient is actually very easy to calculate with index notation. $$\eqalign{ \frac{\partial A_{ij}}{\partial A_{kl}} &= \delta_{ik}\delta_{jl} }$$ which simply says that if the index pair $(i,j)$ is equal to the pair $(k,l)$ then the derivative is one, otherwise it's zero.

Answer 2

Consider that the underlying mapping is$f(A)=A$ hence the identity mapping on the vector space $V$ of all matrices. It is clearly linear, hence its derivative at $A$ in direction $\delta A$ is $ f'(A)\delta A=\delta A, $ which is given by $$ f'(X) = I. $$ Note, that both $f$ and $f'(A)$ are linear mappings from $V$ to $V$. The mapping $f'$ is a mapping from $V$ to $L(V,V)$.

The kronecker product Python is returning maybe because of the fact that the block matrix formed contains all $1$'s and that Python potentially thinks that in general you are asking, a speacial case of $\frac{\partial A}{\partial B}$ for general matrices $A, B \in \mathbb{R}^n$.