Suppose we have to positive definite matrices $M$ and $K$. I want to optimize the following expression \begin{align} \max_{A} \mathrm{Trc} \left((I-A \cdot B) \cdot M \cdot (I-A \cdot B)^T + A KA^T \right) \end{align} where $I$ is the identity matrix and $B$ some other matrix (assume we don't know much about $B$)
In the scalar the case the solution is given by \begin{align} a= \frac{bm}{b^2m+k} \end{align}
Thank you for your help.
Write the function in terms of the Frobenius (:) Inner Product and find its differential $$\eqalign{ f &= (AB-I)M:(AB-I) + AK:A \cr\cr df &= dA\,BM:(AB-I) +(AB-I)M:dA\,B + dA\,K:A + AK:dA \cr &= dA:(AB-I)M^TB^T +(AB-I)MB^T:dA + dA:AK^T + AK:dA \cr &= \Big((AB-I)M^TB^T +(AB-I)MB^T + AK^T + AK\Big):dA \cr\cr }$$ Since $df=\Big(\frac{\partial f}{\partial A}:dA\Big),\,$ the gradient must be $$\eqalign{ \frac{\partial f}{\partial A} &= (AB-I)M^TB^T +(AB-I)MB^T + AK^T + AK \cr &= A(B(M^T+M)B^T + K^T + K) - (M+M^T)B^T \cr\cr }$$ Setting the gradient equal to zero and solving for A yields $$\eqalign{ A(B(M+M^T)B^T + K^T + K) = (M+M^T)B^T \cr A = (B(M+M^T)B^T + K^T + K)^{-1} (M+M^T)B^T \cr }$$