Using Lagrange multiplier to find eigenvector for the minimum eigenvalue in a symmetric matrix

Question

Without the usage of Spectral theorem, with $f:\mathbb{R^n}\to\mathbb{R}$ and $g:\mathbb{R^n}\to\mathbb{R}$ where $f(\mathbf{x}) = \mathbf{x} \bullet A \mathbf{x}$ and $g(\mathbf{x}) = |\mathbf{x}|^2 -1$ and $A$ is a symmetric $n{\times}n$ matrix, I used Lagrange multiplier to try to prove that for any $\mathbf{y}$ that is a solution satisfying the multiplier, then $A\mathbf{y} = \lambda_1\mathbf{y}$ where $\lambda_1 := \min_{|\mathbf{x}|=1}\mathbf{x}\bullet A\mathbf{x}$.

However, I'm stuck after simplifying the equations in Lagrange to narrow it down to: $$2A\mathbf{x} + 2\lambda_1\mathbf{x} = 0$$ $$|\mathbf{x}|^2=1$$ Can anyone tell me if there are steps that I did wrong, and if there are any hints moving forward to find the solution?

Answer 1

(I'm assuming that $A$ and $Q$ are the same matrix here...)

Well, you could divide the first equation by two and move the second term to the other side, getting

$$ Ax = -\lambda_1 x $$ which should not have a minus-sign, and suggests you made a computational error earlier.

But also: there are multiple solutions to the lagrange multiplier problem: both the eigenvectors for the min and the max eigenvalues, for instance. (Unless you're also checking second derivatives, of course.)

You might want to work through the case $$ A = \pmatrix{2 & 0 \\ 0 & -3} $$ and see where your argument goes wrong.

Answer 2

The derivative of $\langle x,Ax\rangle-\lambda(\langle x,x\rangle-1)$ in $v$ is $$\langle v,Ax\rangle+\langle x,Av\rangle-\lambda\cdot2\langle x,v\rangle.$$ Using the symmetry of $A$ it's equal to $$2\langle Ax,v\rangle+2\langle -\lambda x,v\rangle =2\langle Ax-\lambda x,v\rangle.$$ That equals zero if $Ax=\lambda x$ and we have $$\langle x,Ax\rangle =\langle x,\lambda x\rangle=\lambda\langle x,x\rangle=\lambda\cdot1.$$