Show that for M symmetric positive definite have $\sup_{||x||=1}\langle x, Mx \rangle$=$\lambda_1=:$ the largest eigenvalue of M

Question

I know that if I can reach the step $$\sup_{||x||=1}\langle x, Mx \rangle=\langle x,\lambda_1 x\rangle$$ then clearly $\lambda_1$ follows. But how can we bridge this gap?

Answer 1

By the spectral theorem we can take an $\textit{orthonormal basis}$ of $\mathbb R^n$ consisting of eigenvectors of $M$, say $v_1,\ldots,v_n$ with corresponding eigenvalues $\lambda_1,\ldots,\lambda_n$. Then, for any $\Vert v\Vert=1$ we can write $v=c_1v_1+\cdots+c_nv_n$. Try expanding $\langle Mv,v\rangle$ using linearity of the inner product, the fact that $\langle v_i,v_j\rangle=\delta_{i,j}$, and the fact that $Mv_i=\lambda_iv_i$ to show that $\langle Mv,v\rangle\leq\lambda^*$ where $\lambda^*$ is your max eigenvalue. Then can you exhibit a specific $v$ with norm $1$ for which equality holds (think eigenvectors)?