Question: Let $A,B\in\mathbb{R}^{n\times n}$ be two symmetric and positive definite matrices with $AB=BA$ and $\left(A\underline{x},\underline{x}\right)\geq\left(B\underline{x},\underline{x}\right)\,\,\forall\underline{x}\in\mathbb{R}^{n}$. Prove that $\left\Vert A\underline{x}\right\Vert _{2}\geq\left\Vert B\underline{x}\right\Vert _{2}\forall\underline{x}\in\mathbb{R}^{n}$.The symbols denote the Euclidean inner product and the Euclidean vector norm.
My attempt (probably not very good)
Consider $\lambda_{i}$ as the eigenvalues of the matrix $A$ and $v_{i}$ as the eigenvectors of $A$ corresponding to $\lambda_{i}$. Since $A$ is symmetric positive definite, its eigenvalues are all positive . Therefore, we get the following:
$Av_{i} =\lambda_{i}v_{i} $
$A\underline{x}=\underset{i}{\sum}\left(\lambda_{i}v_{i}\underline{x}_{i}\right)\\$
We get the same for the matrix B when we consider $\mu_{i}$ and $u_{i}$ to be the eigenvalues and eigenvectors respectively giving us the following:
$Bu_{i}=\mu_{i}u_{i}\\$
$B\underline{x}=\underset{i}{\sum}\left(\mu_{i}u_{i}\underline{x}_{i}\right)\\$
Now we consider the inequality from the question
\begin{align*}
\left(A\underline{x},\underline{x}\right) & \geq\left(B\underline{x},\underline{x}\right)\\
\end{align*}
\begin{align*} \left(A\underline{x},\underline{x}\right)-\left(B\underline{x},\underline{x}\right) & \geq0\\ \end{align*}
\begin{align*} \left(\underset{i}{\sum}\left(\lambda_{i}v_{i}\underline{x}_{i}\right),\underline{x}\right)-\left(\underset{i}{\sum}\left(\mu_{i}u_{i}\underline{x}_{i}\right),\underline{x}\right) & \geq0\\ \end{align*}
\begin{align*} \left(\underset{i}{\sum}\lambda_{i}\left(v_{i}\underline{x}_{i},\underline{x}\right)\right)-\left(\underset{i}{\sum}\mu_{i}\left(u_{i}\underline{x}_{i},\underline{x}\right)\right) & \geq0\\ \end{align*}
\begin{align*} \underset{i}{\sum}\lambda_{i}\left(v_{i}\underline{x}_{i},\underline{x}\right)-\mu_{i}\left(u_{i}\underline{x}_{i},\underline{x}\right) & \geq0\\ \end{align*}
Since both $A$ and $B$ are symmetric we are able to simplify their inner products as such:
\begin{align*} \left(v_{i}\underline{x}_{i},\underline{x}\right) & =\left(\underline{x}_{i},v_{i}\underline{x}\right)=\left(\underline{x}_{i},v_{i}\right)^{T}\underline{x}\\ \end{align*}
and
\begin{align*} \left(u_{i}\underline{x}_{i},\underline{x}\right) & =\left(\underline{x}_{i},u_{i}\underline{x}\right)=\left(\underline{x}_{i},u_{i}\right)^{T}\underline{x}\\ \end{align*}
We now substitute this into the above inequality to obtain the following:
\begin{align*} \underset{i}{\sum}\lambda_{i}\left(\left(\underline{x}_{i},v_{i}\right)^{T}\underline{x}\right)-\mu_{i}\left(\left(\underline{x}_{i},u_{i}\right)^{T}\underline{x}\right) & \geq0\\ \end{align*}
Simplifying, we have :
\begin{align*} \underset{i}{\sum}\left(\left(\lambda_{i}-\mu_{i}\right)\left(\underline{x}_{i},v_{i}-u_{i}\right)^{T}\underline{x}\right) & \geq0\\ \end{align*}
Since $\left(\underline{x}_{i},v_{i}-u_{i}\right)^{T}$ is a scalar, we are able to rewrite the inequality as follows:
\begin{align*} \underset{i}{\sum}\left(\left(\lambda_{i}-\mu_{i}\right)\left(\underline{x}_{i},v_{i}-u_{i}\right)^{T}\right)\underline{x} & \geq0\\ \end{align*}
Now consider some vector $\underline{y}=\underset{i}{\sum}\underline{x}_{i}\left(v_{i}-u_{i}\right),$a linear combination of the differences between the eigenvectors of A and B, weighted by the components of $\underline{x}$.
Notice that we can rewrite this as $\underline{y}^{T}\underset{i}{\sum}\left(\lambda_{i}-\mu_{i}\right)\underline{y}\geq0$.
Since both $A$ and $B$ are SPD, we know $\lambda_{i},\mu_{i}>0$ we find that $\lambda_{i}-\mu_{i}>0\,\,\forall i$. As a result, we can say that $\underset{i}{\sum}\left(\lambda_{i}-\mu_{i}\right)$ is some scalar that we will denote as $\kappa$.
Therefore we have $\underline{y}^{T}\kappa\underline{y}\geq0$.
Further, since $\kappa>0$ and $\underline{y}$ is a non-zero vector, we can conclude that $\underline{y}^{T}\kappa\underline{y}>0$
Finally, we consider the norms:
\begin{align*} \left\Vert A\underline{x}\right\Vert _{2} & =\left\Vert \underset{i}{\sum}\left(\lambda_{i}v_{i}\underline{x}_{i}\right)\right\Vert _{2}=\left\Vert \underline{y}\right\Vert _{2}\\ \end{align*}
\begin{align*} \left\Vert B\underline{x}\right\Vert _{2} & =\left\Vert \underset{i}{\sum}\left(\mu_{i}u_{i}\underline{x}_{i}\right)\right\Vert _{2}\\ \end{align*}
Since $\underline{y}$ is a non-zero vector and $\left\Vert \underline{y}\right\Vert _{2}>0$, we conclude $\left\Vert A\underline{x}\right\Vert _{2}\geq\left\Vert B\underline{x}\right\Vert _{2}\forall\underline{x}\in\mathbb{R}^{n}$
First proof The result is true for bounded positive and commuting operators on a Hilbert space. The assumption is equivalent to $\|A^{1/2}y\|\ge \|B^{1/2}y\|.$ Hence$$ \|Ax\|\ge \|B^{1/2}A^{1/2}x\|=\|A^{1/2}B^{1/2}x\|\ge \|Bx\|$$
Another proof In the finite dimensional case let $V_\lambda$ denote the subspace of all eigenvectors of $A$ corresponding to the eigenvalue $\lambda.$ Then $B(V_\lambda)\subset V_\lambda.$ Indeed for $v\in V_\lambda $ we have $$ABv=BAv=\lambda Bv$$ The entire space is the orthogonal sum of subspaces $V_{\lambda}$ corresponding to the eigenvalues of $A.$ It suffices to show the inequality for $x\in V_\lambda.$ By assumptions we have $$\lambda \|x\|^2=\langle Ax,x\rangle \ge \langle Bx,x\rangle$$ Therefore the eigenvalues of $B$ restricted to $V_\lambda$ are nonnegative and less than $\lambda.$ Hence the operator norm of $B$ restricted to $V_\lambda$ is less than or equal $\lambda.$ Therefore $$\|Bx\|\le \lambda \|x\|=\|Ax\|$$