I asked a related question yesterday and got a brilliant answer from Ross B. However I still have difficulties. I have the following analog of multivariate loglikehood function (minus 2*log-likehood to be precise) $$ L(C)=Tr(XX^TC + \log(C))~~~~~~~~~(1) $$
This is the fully matrix representation of multivariate log-likelihood. My goal is to minimize it with respect to $C$. Now $C$ is inverse of covariance matrix I am looking for and it is parametrized as follows: $$ C=(D + UFU^T)^{-1}~~~~~~~~~~~~~~~~~ (2) $$ Here $C$ is symmetric nxn matrix. $D$ is a strictly positive diagonal, $U$ is $nxp$ mattrix , $p<n$ and $F$ is $pxp$ symmetric positive definite matrix. $D$ and $F$ are parameters. It is easy to see that in global minimzation over $C$ without parametrization we have $$ L^\prime_c(C)=XX^T-C^{-1}~~~~~~~~~~~~~~~~~(3) $$ So the optimal covariance achieving the derivative $L^\prime=0$ is $\Sigma = C^{-1} =XX^T$ But within (2) the problem is more difficult. Ross.B helped to get derivative of $L(C(D,F)^\prime_F$ and I calculated myself the derivative of $L(C(D,F)^\prime_D$: $$ L(C(D,F))^\prime_F=-U^TC(XX^T-C^{-1})CU \\ L(C(D,F))^\prime_D=-diag(C(XX^T-C^{-1})C) \\~~~~~~~~~(4) $$ Everything seems pretty much nice and symmetric (I hope they are correct), but I cannot solve the system $$ L(C(D,F))^\prime_F=0\\ L(C(D,F))^\prime_D=0 $$
From other considerations I suspect that the optimal solution $D^*$ and $F^*$ is defined as follows: $$ F^*=U^{+} XX^T U^{+T} $$ CORRECTION: $$ D^*=diag((I-UU^+)XX^T(I-UU^+)) $$ or may be $$ D^*=diag((XX^T-UFU^T)=diag(XX^T-UU^+XX^TU^{+T}U^T) $$ But I cannot prove or disprove it. Any help would be very appreciated.