Given the function $G(X,y)$ = $y^T$$X$$y+$$b^T$$b$, where $y$ is a vector in $R^n$, $b$ is a constant vector in $R^n$ and $X$ is a constant symmetric, square, and invertible matrix, what value of $y$ such that $||y||=1$ minimizes the function $G$?
To minimize the function, I first thought to take the partial derivative wrt $y$ and set it equal to 0, so $y^T$$(A^T+A)=0$. But with the restriction of the magnitude of $y$, I'm not sure how to approach the rest. Any help would be appreciated!
On
Let $w$ be an unconstrained vector and define $y$ in terms of it $$y = \frac{w}{\sqrt{w^Tw}}$$ Consider the function $$\eqalign{ \lambda &= y^TXy = \frac{w^TXw}{w^Tw} \cr d\lambda &= \bigg(\frac{2Xw-2\lambda w}{w^Tw}\bigg)^Tdw \cr \frac{\partial\lambda}{\partial w} &= \frac{2}{w^Tw}\Big(Xw-\lambda w\Big) \cr }$$ Setting this gradient to zero yields the eigenvalue equation $$Xw = \lambda w$$ So, in terms of the original variables: $\,y$ is simply the (normalize) eigenvector corresponding to the smallest eigenvalue of $X$, and $G=\lambda+b^Tb$.
I take it that the problem is $$ \min_{y} \quad f(y) = y^T A y \\ \text{s.t.}\quad \|y\|_2 = 1 \tag{1} $$ for $A$ symmetric, invertible. If this is correct, then consider using the eigendecomposition of $A$ using the Spectral Theorem. Can you see what we might want $y$ to be? We can disregard $b$ since it is independent of $y$.
As a start, we have $y^{\top} A y = \sum_{i=1, k=1}^n y_i\, \left( \sum_{j=1}^n \lambda_j \, [v_j]_i \, [v_j]_k \right) \, y_k$ $= \sum_{i=1, j=1, k=1}^n y_j\, y_k\, \lambda_i \, [v_i]_j \, [v_i]_k$.
Then from the last expression in the equality, we see that if we let $y_j = [v_i]_j$, that is, we let the $j$-th component of $y$ be the $j$-th component of the (unit length) eigenvector $i$, then we recover $\lambda_i$. So, to minimize $f$, we can set $y$ equal to the eigenvalue associated with the smallest eigenvector.