Suppose I have a vector to vector valued function of the form $$ f_1: Z \mapsto (ZZ^T)^nAZ , \mbox{ and } f_2: Z \mapsto (ZZ^T)^nA $$ where $Z \in \mathbb{R}^n$ and $A$ is an $n$ by $n$ matrix, how would I calculate the derivative of $f_1$ and $f_2$ with respect to $Z$?
I've looked up the matrix cookbook but I am having difficulty...
On
For convenience, define
$$\eqalign{
\alpha &= z^TAz \cr
\lambda &= z^Tz \cr
}$$
with differentials
$$\eqalign{
d\alpha &= z^T(A+A^T)\,dz \cr
d\lambda &= 2z^T\,dz \cr\cr
}$$
Now let's calculate the differential and gradient of that first function $$\eqalign{ f_1 &= (zz^T)^nAz \cr &= zz^T(zz^T)^{n-1}Az \cr &= z(z^Tz)^{n-1}z^TAz \cr &= \alpha\lambda^{n-1}z \cr \cr df_1 &= \lambda^{n-1}z\,d\alpha + \alpha z(n-1)\lambda^{n-2}\,d\lambda + \alpha\lambda^{n-1}\,dz \cr &= \lambda^{n-1}z\,z^T(A+A^T)\,dz + \alpha z(n-1)\lambda^{n-2}\,2z^T\,dz + \alpha\lambda^{n-1}\,dz \cr \cr \frac{\partial f}{\partial z} &= \lambda^{n-1}zz^T(A+A^T) + 2\alpha z(n-1)\lambda^{n-2}z^T + \alpha\lambda^{n-1}I \cr &= \lambda^{n-2}\Big(\lambda zz^T(A+A^T) + 2(n-1)\alpha zz^T + \alpha\lambda I\Big) \cr\cr }$$ The second function can be handled in a similar fashion.
Hint: $ (ZZ^T)^n A Z = (Z^T Z)^{n-1} (Z^T A Z) Z $