EDIT: It turns out, somewhat embarrassingly, that I misread the source exercise. The true claim is that
$|A|$ is the smallest Hermitian transformation that commutes with $A$ and satisfies $|A|\ge\pm A$.
Now, commuting Hermitian matrices are simultaneously diagonalizable, so the result is fairly immediate. I leave below my original post that led to an interesting characterization by user1551.
This is an exercise of Halmos (1958, §83, exc. 16(a)). Let $A$ be a self-dual matrix (meaning Hermitian in the complex case, symmetric in the real case). Write $A=\operatorname{diag}(\sigma_1, \ldots, \sigma_n)$ in some orthonormal basis. Then define $$ |A| = \operatorname{diag}(|\sigma_1|, \ldots, |\sigma_n|). $$ We can also say that $A=\sqrt{A^2}$ where $\sqrt{\cdot}$ is the unique positive semidefinite square root. The claim is:
$|A|$ is the smallest Hermitian transformation (in the "definiteness" ordering of self-dual transformations) that is greater than both $A$ and $-A$.
I'm stuck on showing the minimality here. So let $B$ be self-dual with $B+A$ and $B-A$ both positive semidefinite, and I want to show that $B-|A|$ is positive semidefinite.
In the case where $A$ and $B$ are simultaneously diagonalizable, the result is straightforward by comparing eigenvalues. But I can't figure out how to extend from here to the general case.
Another strategy would be to use a previous exercise, where it shown that $0\le A\le B \implies \sqrt A\le\sqrt B$. With this it would be enough to show that $A\le B \implies A^2\le B^2$ and then take square roots. (To finish this reasoning, we also need $B\ge 0$, but that is fairly straightforward as well). I really feel like this should be true, but I'm stuck on it too.
Any hints appreciated :)
This is simply false. E.g. by Sylvester’s criterion, $$ B-A:=\pmatrix{2&\sqrt{2}\\ \sqrt{2}&2}-\pmatrix{1&0\\ 0&-1} =\pmatrix{1&\sqrt{2}\\ \sqrt{2}&3}\ge0 $$ and similarly $B+A\ge0$, but $$ B-|A|=B-I=\pmatrix{1&\sqrt{2}\\ \sqrt{2}&1} $$ is indefinite.
However, in general, $|A|$ is indeed the smallest in the sense that $B\ge \pm A\implies\|B\|_F\ge\big\|\,|A|\,\big\|_F$, and equality holds if and only if $B=|A|$.
It is also the smallest in the sense that if $|A|\ge B\ge\pm A$, we must have $B=|A|$. Clearly, $B$ is positive semidefinite because $B=\frac12(B+B)\ge\frac12(A+(-A))=0$. Therefore the condition $|A|\ge B$ implies that $\big\|\,|A|\,\big\|_F\ge\|B\|_F$. By the result in last paragraph, we must have $B=|A|$.
Alternatively, as $B\ge0$, the condition $|A|\ge B\ge\pm A$ implies that $\ker B=\ker A$. So, by considering the restrictions of $A$ and $B$ on $\operatorname{range}(A)=(\ker(A)\big)^\perp$, we may assume that $|A|$ and $B$ are positive definite. By replacing $B$ with $|A|^{-1/2}B|A|^{-1/2}$, we may further assume that $A=\pmatrix{I\\ &-I}$. Now partition $B$ accordingly as $\pmatrix{X&Y^\ast\\ Y&Z}$. From $I\ge \pmatrix{X&Y^\ast\\ Y&Z}\ge\pm\pmatrix{I\\ &-I}$, we obtain $X=I,\,Z=I$ and $Y=0$. Hence $B=|A|$.