I have a very simple question that is confusing me at one point. Suppose that $A$ is a real, non-negative definite matrix (hence automatically square and symmetric). Then, is it true that: $$|y^\top A x| \leq \|A\|_{\textrm{sp}}|y^\top x|~? $$ Here, $\|A\|_\textrm{sp}$ denotes the spectral norm of $A$.
I tried along the following lines:
Let $A = P^\top D P$ be the spectral decomposition of $A$. Let $X = Px$ and $Y=Py$. Then, $$|y^\top A x| = |Y^\top D X| = \left|\sum_i D_{ii}X_iY_i\right|~.$$ I cannot proceed further from the last step, since the $X_i$s and $Y_i$s can be positive or negative (although the $D_{ii}$s are non-negative). Can anyone help me?
This isn't true. Given any two mutually orthogonal nonzero vectors $x$ and $y$ in $\mathbb R^n$, we can construct an inner product $\langle\cdot,\cdot\rangle$ such that $\langle x,y\rangle\ne0$. It follows that if $A$ is the matrix representation of this inner product with respect to the standard basis, then $A$ is positive definite and $|y^\top Ax|=|\langle x,y\rangle|>0=\|A\||y^\top x|$.
E.g. suppose that $u=e_1=(1,0)$ and $v=e_1+e_2=(1,1)$ form an orthonormal basis with respect to the new inner product. Then \begin{cases} \langle e_1,e_1\rangle=\langle u,u\rangle=1,\\ \langle e_1,e_2\rangle=\langle u,v-u\rangle=-\langle u,u\rangle=-1,\\ \langle e_2,e_2\rangle=\langle v-u,v-u\rangle=\langle v,v\rangle+\langle u,u\rangle=2. \end{cases} Therefore $$ A=\pmatrix{1&-1\\ -1&2} $$ is positive definite and $|e_2^\top Ae_1|=|-1|>0=\|A\||e_2^\top e_1|$.