Bound for the norm of expectation of positive semi-definite operators composition

Question

Let $\mathcal X: \mathbb R^{m\times n}\to \mathbb R^{m\times n}$ be a random positive semi-definite operator such that $\|\mathcal X\|\leq a$ where $\|\mathcal X\| = \max_A \|\mathcal X(A)\|_F/\|A\|_F$ is the spectral norm, and $a$ is some deterministic constant. My question is whether the following inequality holds: $$ \|E[\mathcal X^2]\| \leq \|E[\|\mathcal X\| \cdot \mathcal X]\| \leq a\cdot\|E[\mathcal X] \|. $$ By the Cauchy-Schwarz inequality, it is immediate that $\|E[\mathcal X^2]\| \leq E[\|\mathcal X\|\|\mathcal X\|] \leq a E[\|\mathcal X\|]$. However, this is weaker than the desired inequality.

In a way, my question is if the scalar inequality $x\cdot y \leq x\cdot \max(y)$ for $x\geq 0$ generalizes, in some sense, to the positive semi-definite matrix case?

Answer 1

It is true. The given conditions imply that $\mathcal X^2\preceq\|\mathcal X\|\mathcal X\preceq a\mathcal X$ in positive semidefinite partial ordering. Hence $E(\mathcal X^2)\preceq E(\|\mathcal X\|\mathcal X)\preceq E(a\mathcal X)$ and in turn, $\|E(\mathcal X^2)\|\le\big\|E\big(\|\mathcal X\|\mathcal X\big)\big\|\le\|E(a\mathcal X)\|$.

Answer 2

I am going to prove that for any distribution on positive semidefinite matrices $A$,

$$\lVert \mathsf{E} A^2\rVert\le \left\lVert \mathsf{E}\left[\lVert A\rVert A\right]\right\rVert,$$

where $\lVert\cdot \rVert$ is the spectral norm.

---

Fix some positive semidefinite matrix $A$. Let $u_1, \ldots, u_n$ be an orthonormal basis of eigenvectors of $A$ corresponding to eigenvalues $\lambda_1, \ldots, \lambda_n$. Then, for all $x=\sum_{i=1}^n \alpha_i u_i$,

$$\langle x,A x\rangle=\sum_{i=1}^n \alpha_i^2 \lambda_i$$

and since $0\le \lambda_i\le \lVert A\rVert$,

$$\langle x,A^2 x\rangle=\sum_{i=1}^n \alpha_i^2 \lambda_i^2\le \lVert A\rVert \langle x, Ax\rangle.$$

---

By linearity of expectation, it follows that

$$\langle x, (\mathsf{E} A^2) x\rangle=\mathsf{E} \langle x, A^2x\rangle\le \mathsf{E}\left[\lVert A\rVert \langle x, Ax\rangle\right]=\langle x, \mathsf{E} \left[\lVert A\rVert A\right] x\rangle.$$

And the result follows by taking the supremum over $x$.