Find optimal parametr

Question

Let $y = X{\beta_{0}} + \varepsilon, $ where $X$ - nonsingular constant matrix and $\varepsilon$ - constant vector with $||\varepsilon||_{2}^{2} = \delta.$

Answer 1

The minimization has nothing to do with $\epsilon$.

Hint: use the fact that for any vector $u$, $$ \Vert u\Vert_{2}^{2}=u^{\intercal}u $$ to write the minimization in terms of dot products. This will give you an expression of the form $$ \min_{\beta}f(\beta). $$ Recall from calculus that extrema occur at critical points (points where the derivative is zero). Therefore, next, find point(s) $\beta$ which satisfy the equation $$ \nabla f(\beta)=0. $$ You can use the identities on this page to compute the gradient.