For a convex function $f: \mathbb{R}^{n \times n} \to \mathbb{R}$, if $X^{*} = \underset{ X\succeq 0 }{ \operatorname{arg min}}f(X)$, then we have $$\nabla f(X^{*})\succeq 0, \qquad X^{*} \succeq 0, \qquad \left\langle \nabla f(X^{*}),X^{*} \right\rangle = 0$$ How can we show $\nabla f(X^{*})$ is symmetric?
I have proved everything else (see below). However, I'm running out of techniques to show that $\nabla f(X^{*})$ is symmetric, I have tried to use orthogonal component of skew symmetric matrix, and some other techniques. My intuition is that this is quite easy, and can be proven by contradiction by finding a direction to decrease in the positive semidefinite range. However, I have not come up with anything solid.
Proof: For $\nabla f(X^{*}) \succeq 0$, suppose not, $\exists u$ s.t. $u^{T} \nabla f(X^{*})u = u^{T}\otimes u^{T}vec(\nabla f(X^{*})) < 0$. $u^{T}\otimes u^{T} = uu^{T}$ which is a positive semidefinite matrix. Therefore, $\exists t > 0$ s.t. $f(X^{*}+tuu^{T}) < f(X^{*})$, which contradicts that $X^{*}$ is the global minimum. Next we show $\langle \nabla f(X^{*}),X^{*} \rangle = 0$. Suppose not, if $\langle \nabla f(X^{*}),X^{*} \rangle < 0$, $\exists t> 0$, s.t. $f(X^{*}+tX^{*}) < f(X^{*})$. Similarly, if $\langle \nabla f(X^{*}),X^{*} \rangle > 0$, $\langle \nabla f(X^{*}),X^{*} \rangle < 0$, since we know for $0 < t < 1$, $tX^{*} \succeq 0$, we know $\exists 0<t<1$ s.t. $f(X^{*}-tX^{*})) < f(X^{*})$. Both cases contradict that $X^{*}$ is the global minimum.
Let $A= \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}$ and $f(X) = {1 \over 2} \| X-A\|_F^2$. Then $\nabla f(X) = X-A$.
A straightforward computation shows that $\min_{X \ge 0 } f(X) $ is attained at $X^*= \begin{bmatrix} 1 & {1 \over 2} \\ {1 \over 2} & 1 \end{bmatrix}$, and we see that $\nabla f(X^*) = \begin{bmatrix} 0 & -{1 \over 2} \\ {1 \over 2} & 0 \end{bmatrix}$.