proving that $\max Q(x)=\lambda_\max$

Question

Let $Q(x)$ be quadratic form. Prove that $\max_{\|x\|=1}Q(x)=\lambda_\max$.

$Q$ is symmetric so it can be presented as $$\langle Ax,x\rangle$$ where $A$ is matrix which on its diagonal appears the eigenvalues. I don't really know from where to continue. How can I prove it?

Answer 1

This is discussed in my lecture notes, at http://rutherglen.science.mq.edu.au/math133s213/notes/Quadratic%20forms2013.pdf

Answer 2

This is a special case of the Min-max Theorem, and it is proved relatively easily.

I will assume that you are working on a finite-dimensional inner product space, say $\Bbb{R}^{n}$.

If you assume the spectral theorem, then the proof is quite straightforward: Choose an orthogonal matrix $P$ such that $A = P^{T}DP$, where $D$ is a diagonal matrix. Then

$$ \max_{\|x\|=1} Q(x) = \max_{\|x\|=1} (Ax, x) = \max_{\|x\|=1} (DPx, Px) = \max_{\|y\|=1} (Dy, y) = \lambda_{\max}. $$

Here, $y = Px$ and we utilized the fact that $\|y\| = 1$ if and only if $\|x\|=1$. But we can also prove with rather directly as follows. (Actually, you can use this argument repeatedly to prove the spectral theorem for symmetric matrices.)

Step 1. We prove that $\displaystyle \max_{\|x\|=1} Q(x) \geq \lambda_{\max}$.

Let $v$ be a unit eigenvector corresponding to $\lambda_{\max}$. Then $Q(v) = (Av, v) = (\lambda_{\max}v, v) = \lambda_{\max} $. Thus $\max_{\|x\|=1} Q(x) \geq \lambda_{\max}$.

Step 2. We prove that $\displaystyle \max_{\|x\|=1} Q(x) \leq \lambda_{\max}$.

Let us denote $\lambda = \max_{\|x\|=1} Q(x)$. Then the inequality follows once we prove that $\lambda$ is an eigenvalue of $A$. Since $Q$ is continuous on the compact set $S^{n-1} = \{ x \in \Bbb{R}^{n} : \| x\| = 1 \}$, we can choose a unit vector $v$ that maximizes $Q$ on $S^{n-1}$: $Q(v) = \lambda$.

Our aim is to prove that $v$ is an eigenvector corresponding to the eigenvalue $\lambda$. To this end, choose any other unit vector $x \in S^{n-1}$ such that $x \perp v$. Then the function $f(t) = Q(v \cos t + x \sin t)$ achieves a local maximum when $t = 0$. Thus we must have $f'(0) = 0$. But since

\begin{align*} f(t) &= (A(v \cos t + x \sin t), v \cos t + x \sin t) \\ &= Q(v)\cos^{2}t + 2(Av, x)\cos t \sin t + Q(x) \sin^{2} t, \end{align*}

we must have $ 0 = f'(0) = 2(Av, x)$. This implies that $Av \perp x$ whenever $v \perp x$, hence $Av$ and $v$ must be parallel. Therefore $Av = (Av, v)V = \lambda v$ and $\lambda$ is an eigenvalue of $A$ as desired. ////