I was watching this video about Camera Calibration and I have a doubt regarding the solution of the following Constrained Least Squares problem $$ \underset{\mathbf{p}}{\operatorname{min}}\Vert A\mathbf{p}\Vert\ \ \ s.t. \ \Vert\mathbf{p}\Vert^2=1\\ $$ Which is equivalent to $$ \underset{\mathbf{p}}{\operatorname{min}}\left(\mathbf{p}^T A^TA\mathbf{p}\right) \ \ \ s.t. \ \mathbf{p}^T\mathbf{p}=1 $$ So we define a Loss function $\mathcal{L}(\mathbf{p},\lambda)$ $$ \mathcal{L}(\mathbf{p},\lambda) = \mathbf{p}^T A^TA\mathbf{p} -\lambda(\mathbf{p}^T\mathbf{p}-1) $$ Taking the derivative w.r.t. $\mathbf{p}$ and find out that we're in front of the well-known eigen-value problem $$ \frac{\partial \mathcal{L(\mathbf{p},\lambda)}}{\partial\mathbf{p}} = 2A^TA\mathbf{p}-2\lambda\mathbf{p} = 0\\ A^TA\mathbf{p}=\lambda\mathbf{p} $$ The solution of the problem apparently is the eigenvector $\mathbf{p}$ with the smallest eigen-value $\lambda$ (minute 7:22 of the video).
My question is: why does the solution has to be the eigenvector $\mathbf{p}$ with the smallest eigen-value? Wouldn't any eigen-vector work as well?
As your matrix $A^TA$ is by construction real, symmetric, and positive semidefinite their eigenvalues are all non-negative real numbers and it can be diagonalized in an orthonormal basis of eigenvectors $e_1,\dots,e_n$ associated with eigenvectors $\lambda_1\geq\dots\geq\lambda_n$. Now we can write any vector as $$x=\mu_1e_1+\dots+\mu_ne_n$$ With $\mu_1^2+\dots+\mu_n^2=1$. In particular, we have $$p^tA^tAp-e_k^tA^tAe_k=\lambda_1\mu_1^2+\dots+\lambda_n\mu_n^2-\lambda_k\geq \lambda_k\left(\mu_1^2+\dots+\mu_n^2-1\right)=0$$ Therefore as we know the minimum is an eigenvector and this eigenvector will be the one that minimizes the difference $$\lambda_1\mu_1^2+\dots+\lambda_n\mu_n^2-\lambda_k$$ And clearly, this will be the eignenvector $e_q$ with minimum eigenvalue $\lambda_q$.