Given a scalar objective function,
$\mathrm{argmin}_P$ $f\left(P\right) = ||P^TAP||^2_F+\mathrm{tr}\left(P^TBP\right)$
where $P$ is an unknown matrix, A and B are known.
A stationary point $P$ of $f\left(P\right)$ should be a matrix of $d$ smallest eigenvectors of $M\left(P\right)$, where
$M\left(P\right)=APP^TA+\frac{1}{2}B$
How to prove the above statement? What is the relation between the stationary point and the eigenvectors?
Thank you!
Calculate the gradient $$\eqalign{ f &= P^TAP:P^TAP + B:PP^T \\ df &= 2P^TAP:(P^TA\,dP+dP^TAP) + B:(P\,dP^T+dP\,P^T) \\ &= (2A^TPP^TAP + 2APP^TA^TP +BP+B^TP):dP \\ \frac{\partial f}{\partial P} &= 2A^TPP^TAP + 2APP^TA^TP +BP+B^TP \\ }$$ If $A$ and $B$ are symmetric then this can be simplified to $$\eqalign{ \frac{\partial f}{\partial P} &= 4MP \\ }$$ Set the gradient to zero and solve $$\eqalign{ MP &= 0 \\ }$$ Obviously, $P=0$ is the global minimum. Other stationary points might be characterized as follows:
Assume $P={\tt[}\,p_1\;p_2\;\ldots\,{\tt]}\;$ then the columns of $P$ satisfy $\,Mp_k = 0.\;$
So the $p_k$ are eigenvectors of $M$ associated with the eigenvalue $\lambda = 0.$
NB: In several steps above, a colon is used to denote the trace/Frobenius product, i.e. $$\eqalign{ A:B = {\rm Tr}(A^TB) = {\rm Tr}(B^TA) = B:A \\ }$$