Proof of a variational trace inequality

212 Views Asked by At

The property I refer to is this one. It says that for any positive semidefinite $\gamma$ with $\text{Tr}(\gamma) = 1$ and self-adjoint $H$, the following inequality holds

$\text{Tr}(\gamma H) - \text{Tr}(\gamma\log\gamma) \leq \log\text{Tr}\exp(H)$ with equality if and only if $\gamma = \frac{\exp(H)}{\text{Tr}(\exp(H))}$

To prove this for positive definite $\gamma$ is easy. The left hand side is concave in $\gamma$ and differentiable. Taking the derivative and setting it to zero gives the desired result.

However, the result is claimed for positive semidefinite matrices and in this case, the term $\text{Tr}(\gamma\log\gamma)$ is not differentiable as answered in the comments of this question.

How does one complete the proof?

1

There are 1 best solutions below

0
On BEST ANSWER

I am assuming that $\gamma$ is Hermitian and hence diagonalisable (most authors only define positive semi-definteness for Hermitian matrices, but some use alternative definitions).

Define $\text{Tr}(\gamma\log\gamma)=\sum \lambda_i\log\lambda_i$ where the $\lambda_i$ are the eigenvalues of $\gamma$ (some of these eigenvalues may be repeated) and $0\log 0=\lim_{\epsilon\rightarrow 0}\epsilon\log\epsilon=0$

Diagonalize $\gamma$ as $\gamma=UDU^{-1}$ and define $\gamma_{\epsilon}=U(D+\epsilon I)U^{-1}$. Then $\gamma_{\epsilon}$ is positive definite for all $\epsilon>0$.

We will define $f(\epsilon)=\text{Tr}(\gamma_\epsilon H)-\text{Tr}(\gamma_\epsilon \log \gamma_\epsilon)-\log \text{Tr}\exp( H)$.

If we can prove that $f(\epsilon)$ is left-continuous at zero, then $f(\epsilon)\leq 0$ for $\epsilon>0$ implies $f(0)\leq 0$. To prove continuity of the term $\text{Tr}(\gamma_\epsilon \log \gamma_\epsilon)$, note that $\lim_{\epsilon \rightarrow 0} (\lambda_i+\epsilon) \log (\lambda_i+\epsilon)=\lambda_i \log \lambda_i$ .