Recently I want to pratice matrix calculus, and I face the following questions. I have a kronecker product and I don't know how to differentiate it. Could you please help me? $$ \frac{\partial(\theta\theta^T\otimes I_{n*n})}{\partial\theta} $$ where $\theta$ is a vector and $I_{n*n}$ is a identity matrix.
My thought is that $$ \frac{\partial(\theta\theta^T\otimes I_{n*n})}{\partial\theta}=(I\otimes\theta+\theta\otimes I)\otimes I_{n*n} $$ And is it right? Thanks a lot.
$ \def\l{\lambda}\def\t{\theta}\def\e{{\large e}} \def\o{{\tt1}}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\fracLR#1#2{\LR{\frac{#1}{#2}}} \def\gradLR#1#2{\LR{\grad{#1}{#2}}} $Unfortunately, you've picked a difficult practice problem because a matrix-by-vector gradient produces a third-order tensor which cannot be represented using standard matrix notation. However, the component-wise gradients are simply matrices, so let's calculate those.
First, recall that the gradient of a vector with respect to one of its own components is the corresponding cartesian basis vector, i.e. $$\eqalign{ \t = \sum_k\t_k\e_k \qiq \grad{\t}{\t_k} = \e_k \\ }$$ Applying this to your matrix-valued function yields a set of matrix-valued gradients $$\eqalign{ F\; &= \;\t\t^T\otimes I \\ \grad{F}{\t_k} &= \LR{\e_k\t^T+\t\e_k^T}\otimes I \\ }$$