Suppose two matrices $A$ and $B$ with eigenvalues $\lambda^{a}_i$ and $\lambda^{b}_i$ and eigenvector matrices $U^a=\left(v^a_0, v^a_1, ...\right)$ and $U^b=\left(v^b_0, v^b_1, ...\right)$.
Suppose some number $Q=\left(\sqrt{\lambda^a_0},\sqrt{\lambda^a_1}...\right)U^a U^{b\dagger}\left(\sqrt{\lambda^b_0},\sqrt{\lambda^b_1}...\right)^\dagger$, which I need to get maximized (that would play a role later)
Where $\dagger$ means hermitian conjugation ($\dagger=*T$)
I tried to write out $Q^\dagger Q$
$Q^\dagger Q=(\left(\sqrt{\lambda^b_0},\sqrt{\lambda^b_1}...\right)U^{b})(U^{a\dagger}\left(\sqrt{\lambda^a_0},\sqrt{\lambda^a_1}...\right)^\dagger)(\left(\sqrt{\lambda^a_0},\sqrt{\lambda^a_1}...\right)U^a) (U^{b\dagger}\left(\sqrt{\lambda^b_0},\sqrt{\lambda^b_1}...\right)^\dagger)$
Using associativity and condition $U^\dagger U=I$
$Q^\dagger Q=(\left(\sqrt{\lambda^b_0},\sqrt{\lambda^b_1}...\right)U^{b})(\lambda^a_0+...)(U^{b\dagger}\left(\sqrt{\lambda^b_0},\sqrt{\lambda^b_1}...\right)^\dagger)$
Similarly
$Q^\dagger Q=(\lambda^a_0+...)(\lambda^b_0+...)$
Using property $tr(A)=\sum \lambda_i$
$Q^\dagger Q=tr(A)\cdot tr(B)$
If $Q$ is maximized, then $Q^\dagger Q$ is maximized as well, so differentiating $Q^\dagger Q$ wrt. to A and B and making it equal to zero, we can find the maximum. However, this results $tr(A)=tr(B)=0$, which is obviously not true? Have I made a mistake? If so, where it is?
Edit: I suppose $tr(A^\dagger A)=const$ and same for $B$