How to get the Riesz representative of the derivative of $L(K):=\text{tr}(\Lambda^* K A)$

Question

$\DeclareMathOperator{\tr}{tr}K,\Lambda, A$ here are appropriate matrices. The question is not completely accurate as I can differentiate it, but I would prefer it to be in the form $⟨DL,h⟩$ for some $DL$, with the inner product defined below; the notes I am following claims that $DL = \Lambda A^*$ but I don't see how this is true.

We first write $L$ as an inner product induced by the Hilbert (Frobenius) norm, $$ ‖ K‖^2 := \tr(KK^*) \implies ⟨ A, B⟩ = \tr AB^*$$ which means $$L(K) = ⟨ \Lambda^*,(KA)^*⟩ = ⟨ \Lambda^*,A^*K^*⟩$$ so The mindless calculation: $$ L(K+h) - L(K) = ⟨ \Lambda^*,A^*(K+h)^*⟩ - ⟨ \Lambda^*,A^*K^*⟩ = ⟨ \Lambda^*,A^*h^*⟩$$ Which is the derivative...but not in the form I want and I don't see how to continue.

---

For context, I am trying to prove Gauss-Markov via Lagrange multipliers; i.e.

Suppose there is an known matrix $A$ and an unknown true $x$ and $b$ satisfying $Ax = b$. Then given the noisy observations of b,(read: random vector) $y = b + \eta = Ax + \eta$ where $\eta\sim N(0,Q)$ is Gaussian noise such that $A^*Q^{-1}A$ is invertible, the estimator $K=Ky=K(y)$ (by abuse of notation) that is unbiased ($⇔ KA = I)$ and minimises the mean square error $$K = \text{argmin}_{\kappa:\kappa A=I}\mathbb{E}\|\kappa y - x‖^2 $$ is $K = (A^* Q^{-1} A)^{-1}A^*Q$. This choice of $K$ also minimises the covariance matrix $\mathbb{E}(Ky-x)(Ky-x)^*$, in the sense of positive definite matrices i.e. $A\leq B$ iff $B-A$ is positive definite, with minimising value (i.e. matrix) $$ \mathbb{E}(Ky-x)(Ky-x)^* = (A^*Q^{-1}A)^{-1}$$

The steps leading to the above problem:

1. rewrite the objective function $E‖Ky - x‖^2 = \tr KQK^* ( = ‖KQ^{1/2}‖^2 )$
2. formulate the Lagrangian $$ \mathcal{L}(K,\Lambda) = \tr KQK^* - \tr \Lambda(KA-I) $$ Also, for reference the derivative of $\mathcal{L}$ wrt. $K$ is supposed to be represented by $$ ∇_K\mathcal{L}(K,\Lambda) = 2KQ - \Lambda A^* $$

I also don't know why this is the correct formulation of the Lagrangian but I suspect this is enough for one question.

Answer 1

1. If you consider the Gauss-Markov theorem, then your matrices are real and why the notation $A^*$ and not $A^T$ ? Here you are lucky because your derivative is linear over $\mathbb{C}$, that is not the case in gneral. One has $DL_K(h)=tr(\Lambda^*hA)=tr(hA\Lambda^*)=<h,\Lambda A^*>$; according to Riesz, we identify $DL_K$ and $\Lambda A^*$.

2. Assume that the matrices are real and $Q$ is symmetric; let $L=tr(KQK^T)-tr(\Lambda(KA-I))$. Then $DL_K(H)=2tr(KQH^T)-tr(\Lambda HA)=2tr(H^TKQ)-tr(H^T\Lambda^TA^T)=<H,2KQ-\Lambda^T A^T>$.

using the inner product $<U,V>=tr(U^T,V)$. Under this hypothesis, we identify $DL_K(H)$ and $2KQ-\Lambda^T A^T$.