Without the use of Spectral Theorem, assume that $A$ is a symmetric $n\times n$ matrix, and define $f: \mathbb{R^n}\to\mathbb{R}$, $g_0: \mathbb{R^n}\to\mathbb{R}$, and $g_1: \mathbb{R^n}\to\mathbb{R}\;$ by $$f(\mathbf{x}) = \mathbf{x}\cdot A\mathbf{x},\ \ \ \ \ g_o(\mathbf{x}) = |\mathbf{x}|^2-1, \ \ \ \ \ g_1(\mathbf{x}) = \mathbf{y}\cdot\mathbf{x}$$ where $\mathbf{y}$ is any solution of the minimization problem where we minimize $f(\mathbf{x})$, subjecting to the constraint $g_o(\mathbf{x}) = 0$.
I proved $A\mathbf{y} = \lambda_1\mathbf{y}$ where $\lambda_1$ is the smallest eigenvalue. However, I am having trouble proving the second smallest eigenvalue $\lambda_2$ with its eigenvector where if $\mathbf{z}$ is any solution of the minimization problem where we minimize $f(\mathbf{x})$, subjecting to the constraint $g_o(\mathbf{x}) = 0$ AND $g_1(\mathbf{x}) = 0$, then $A\mathbf{z} = \lambda_2\mathbf{z}$.
Notice that $P=I-yy^T$ projects $x$ onto $y_{\perp}$. So the minimization problem is equivalent to minimizing $xP^TAPx$. It’s easy to see that $M:=P^TAP$ is symmetric. So it will be the smallest eigenvector/eigenvalue of $M$. Can you finish from here? As a hint: $Px=x$ whenever $(x,y)=0$