I am trying to solve the following matrix maximization problem
$\max_\Theta trace (A H (\Theta, P))-ln(det(H (\Theta, P)))-ln(det(P))$ ,
where $A, \Theta, P, F$ are all matrices and $P$ is implicitly defined by the matrix equation
$P=F H (\Theta, P)F^{T}$,
where $H (\Theta, P)$ is a non-linear function in both $P$ and $\Theta$.
I think it would not be difficult to solve it using the Matrix Cookbook if $P$ wasn't implicitly defined.
Thank you.
Matrix Cookbook is as a jukebox: if you don't find the required song, then you are stark naked.
Necessarily $A,H\in M_n,F\in M_{m,n},P\in M_m,\Theta\in M_{p,q}$ and $H,P$ have $>0$ determinant. Since $P=FHF^T$, we can assume that $P$ is a function of $\Theta=[\theta_{i,j}]$ ; then each $\dfrac{\partial P}{\partial \theta_{i,j}}$ satisfies a linear equation: $\dfrac{\partial P}{\partial \theta_{i,j}}=F(\dfrac{\partial H}{\partial \theta_{i,j}}+\dfrac{\partial H}{\partial P}(\dfrac{\partial P}{\partial \theta_{i,j}}))F^T$. From a numerical point of view, for every $i,j$, we can calculate $\dfrac{\partial P}{\partial \theta_{i,j}}$.
Let $f:\Theta\rightarrow tr(AH)-\ln(\det(H))-\ln(\det(P))$. Then $\dfrac{\partial f}{\partial \theta_{i,j}}=tr(A(\dfrac{\partial H}{\partial \theta_{i,j}}+\dfrac{\partial H}{\partial P}(\dfrac{\partial P}{\partial \theta_{i,j}})))-tr((\dfrac{\partial H}{\partial \theta_{i,j}}+\dfrac{\partial H}{\partial P}(\dfrac{\partial P}{\partial \theta_{i,j}}))H^{-1})-tr(\dfrac{\partial P}{\partial \theta_{i,j}}\;P^{-1})$.
Beware, $\dfrac{\partial H}{\partial P}(\dfrac{\partial P}{\partial \theta_{i,j}})$ is not a product because $\dfrac{\partial H}{\partial P}:M_m\rightarrow M_n$.
Then it "remains" to solve the system: for every $i,j$, $\dfrac{\partial f}{\partial \theta_{i,j}}=0$.