Proposition: If $A \in \mathbb{R}^{n \times n}$ a symmetric matrix then $||A||= \sup \{ ||Ax||_2: ||x||_2=1\}= \sup \{ |\langle x, Ax \rangle|: ||x||_2=1\}$.
Proof:
It suffices to show that $||A|| \leq \sup \{ |\langle x,Ax \rangle|: ||x||_2=1\}$.
(The other inequality has already been shown.)
There is a $x \in \mathbb{R}^n$ with $||x||_2=1$ such that
$||A||^2= ||Ax||_2^2= \langle Ax, Ax \rangle= \langle x, A^T A x \rangle= \langle x, A^2 x \rangle \overset{\text{ Lemmas 1,2}}{\leq} \sup \{ |\langle x, Ax \rangle|: ||x||_2=1\}^2$.
The lemmas 1 and 2 are the following:
Lemma 1: Let $A \in \mathbb{R}^{n \times n}$ be a symmetric matrix, $A=(a_{ij})$. The vectors with norm 1 that maximize or minimize the function $f(x)=\langle x, Ax \rangle (||x||_2=1)$ are eigenvectors of the matrix $A$.
Lemma 2: Let $A \in \mathbb{R}^{n \times n}$ be a symmetric matrix. Then $\delta$ or $-\delta$ is an eigenvalue of $A$ iff $\delta^2$ is an eigenvalue of $A^2$.
Could you explain me how from them we deduce that $\langle x, A^2 x \rangle \leq \sup \{ |\langle x, Ax \rangle|: ||x||_2=1\}^2$ ?
you have $\langle x, A^2x \rangle \leq \sup\{\langle x, A^2x\rangle, \|x\| = 1\}$. Now let $y$ be the vector where the maximum is attained and let $\delta^2$ be the corresponding eigenvalue. We then have
$$ \langle x, A^2x \rangle \leq \sup\{\langle x, A^2x\rangle, \|x\| = 1\} = \langle y, \delta^2y\rangle = \delta^2 \langle y, y\rangle = \delta^2 \langle y, y\rangle^2 = (\langle y, \delta y\rangle)^2 = (\langle y, Ay\rangle)^2 \leq (\sup\{\langle x, Ax\rangle, \|x\| = 1\})^2 $$