Min-Max Principle with matrices - Understanding $\lambda_2$

Question

Related to the question Min-Max Principle $\lambda_n = \inf_{X \in \Phi_n(V)} \{ \sup_{u \in X} \rho(u) \}$ - Explanations, I am trying to do the same with $$A= \begin{bmatrix} 2 & 0 & 0\\ 0 & 7 & -1\\ 0 & -1 & 7 \end{bmatrix}.$$ I'd like to understand the Min-Max Principle (see here) on $\lambda_2$, i.e. $$\lambda_2 = \min_{X \in \mathbb{R}^3} \{\max_x \{R_A(x) : x \in X, x \not = 0 \text{ and } \dim X = 2\}\}.$$ In the mickep's answer, it is rather simple to understand that if we work on $\lambda_1$, the subspaces $X$ are simply lines. So the Rayleigh quotient $R_A$ is constant in each $X$ because $$\rho(tu)= \frac{\langle Atu,tu\rangle}{\langle tu,tu\rangle}= \frac{\langle Au,u\rangle}{\langle u,u\rangle}= \langle Au,u\rangle \text{ with }\|u\|=1.$$ I would like to simplify my problem in such way, but I don't know if I can do it. Is there anyone could tell me if there exists an equivalent way to do it with the $2$-dimensional subspace $X \subset \mathbb{R}^3$?

A thing I know so far is there exists an orthonormal basis $\{w_1, w_2\}$ is each $2$-dimensional subspace $X$, but I just don't know if it could be useful for the problem. Any help would be appreciated, thanks!

P.S. If my question is unclear, please let me know. Furthermore, Jeb's answer is well, but I would like to get another answer if possible. I would have like that someone use the explicit formula to respond in my question. In fact, the important thing I want to understand is the notion of subspace that Jeb didn't use in his answer, i.e. something with image, geometric.

Answer 1

Another way of writting Jeb's solution is the following.

Let $v_1,v_2,v_3$ be an orthonormal basis of $\mathbb{R}^3$ of eigenvectors of $A$ associated to the eigenvalues $\lambda_1\leq\lambda_2\leq\lambda_3$. Let $V=\text{span}\{v_2,v_3\}$.

If $X$ is a 2-dimensional subspace of $\mathbb{R}^3$ then $\dim(X\cap V)\geq 1$. Let $u\in X\cap V$ with norm 1. Thus $u=av_2+bv_3$, $a^2+b^2=1$.

Now, $R_A(u)=a^2\lambda_2+b^2\lambda_3\geq \lambda_2$, since $a^2+b^2=1$.

So $\max_x \{R_A(x) : x \in X, x \not = 0 \text{ and } \dim X = 2\}\geq \lambda_2$, for every $X$. Hence, $\min_{X \subset \mathbb{R}^3}\{\max_x \{R_A(x) : x \in X, x \not = 0 \text{ and } \dim X = 2\}\}\geq \lambda_2$.

Now, if we choose $X= \text{span}\{v_1,v_2\}$, we can prove in a similar way that $\max_x \{R_A(x) : x \in X, x \not = 0 \text{ and } \dim X = 2\}= \lambda_2$.

Therefore, $\min_{X \subset \mathbb{R}^3}\{\max_x \{R_A(x) : x \in X, x \not = 0 \text{ and } \dim X = 2\}\}\leq \lambda_2$.

Answer 2

Seems like you're jumping the gun with this question... I'd like to take a few steps back then work up to whats going on here. If $A$ is a hermitian matrix ( or symmetric for real matrices) then all the eigenvalues of $A$ are real (why?) and the eigenvectors are orthogonal (also why?). This allows us to rewrite our basis in terms of these eigenvectors. i.e. if $x \in \mathbb{R}^3$, then $$ x = c_1 \vec { \lambda_1} + c_2 \vec{ \lambda_2} + c_3 \vec{\lambda_3}\quad c_1,c_2,c_3 \in \mathbb{R}$$ with $\vec{\lambda_i}$(the eigenvectors) normalized. Now it's easy to see our inner product with $A$ produces $$ (x, A x ) = c_1^2 \lambda_1 + c_2^2 \lambda_2 + c_3^2 \lambda_3 $$ let's restrict ourself to $x$ being of unit length as well, i.e. $$ ||x||^2 = c_1^2 + c_2^2+c_3^2 =1 $$ Now we see $$ \frac{ (x, A x )}{||x||^2 } = c_1^2 \lambda_1 + c_2^2 \lambda_2 + c_3^2 \lambda_3 $$ It's easy to see that the max is obtained at the largest eigenvalue since if $a<1$, we have $a^2 <a$. Thus $$ \max_{||x||=1} \frac{ (x, A x )}{||x||^2 } = \max_i \lambda_i $$ If we take out the linear subspace generated by $ A x - \max_i \lambda_i x=0$, i.e. $\vec \lambda_i$, we see the resulting subspace contains elements of the form above but missing the $\vec \lambda_i$ term. Now you repeat how we found $ \max_i \lambda_i$, to find the next largest eigenvalue.

Edit: Let's see how this pans out for the exact matrix you've given. One may easily check that the eigenvalues and vectors of your matrix are $$ \lambda = 8,6,2 \quad \vec \lambda = \begin{pmatrix} 0 \\ -1 \\ 1 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}$$ You know that any element in $\mathbb{R}^3$ may be written as $$ x = c_1 \begin{pmatrix} 0 \\ -1 \\ 1 \end{pmatrix} + c_2 \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix}+ c_3 \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} $$ and we have that $$ (x,Ax) = 8 c_1^2 + 6c_2^2 + 2 c_3^2 $$ Now if we restrict to the unit sphere, we see $$\max_{||x||=1} (x,Ax) = 8$$ if we restrict to the sphere and cut out the subspace generated by $(0,-1,1)$, we have $$ \max_{||x|| =1 \setminus span \vec \lambda_1} (x,Ax) = 6 $$ and if we cut off $\vec \lambda_2$ as well, the vectors we're allowed look like $$x = c_3 \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} $$ Thus $$ \max_{ ||x|| =1 \setminus span \vec \lambda_1,\vec \lambda_2} (x,Ax) = 2 $$ notice this works from the top down, you can reverse the procedure but taking the minimum on the sphere, but then remove vectors to move up as well.