My question is: Given a matrix $A$ and its eigenvector $v$ which corresponds to $A$'s maximum eigenvalue, is there a closed form formula to calculate the derivative
$$\frac{\partial(u^Tv)}{\partial A}$$
where $u$ is an unrelated vector?
Please help! Thanks in advance!
Update:
I check matrix cookbook which shows
$$\partial v = (\lambda I-A)^\dagger\partial(A) v$$
where $\lambda$ is the corresponding eigenvalue, $\dagger$ is the symbol of pseudo-inverse. But I still don't know how to calculate the desired derivative.
Start with the eigenvalue equation, and take differentials $$ \eqalign { 0 &= (A-\lambda I)v \cr &= dA\,v + (A-\lambda I)dv \cr dA\,v &= (\lambda I-A)dv } $$ For notational convenience, let $M=(\lambda I-A)^{\dagger}$.
Then proceeding to the least squares solution, we obtain the cookbook result $$ \eqalign { dv &= M\,dA\,v \cr } $$ From there, we can pre-multiply by $u^T$ to obtain $$ \eqalign { u^Tdv &= u^TM\,dA\,v \cr d(u^Tv) &= p^TdA\,v \cr &= pv^T:dA \cr } $$ where $p=M^Tu\,\,$ and the colon represents the Frobenius product, $\,X\!:\!Y=tr(X^TY)$.
Since $df = (\frac{\partial f}{\partial A}):dA$, the derivative must be $$ \eqalign { \frac{\partial\,(u^Tv)}{\partial A} &= pv^T \cr &= M^{T}uv^T \cr &= (\lambda I-A^T)^{\dagger} \, uv^T } $$