How can I apply least squares solution to the problem min = ||Av - uv|| where v is given?

Question

Here is the screenshot: https://i.stack.imgur.com/QDwQN.jpg I tried to find the least squares solution of the problem at first, and i couldn't solve it. So I asked that part only, but if showing the Rayleigh quotient part is necessary to understand the part I asked, you are welcome to answer the whole question.

Answer 1

$$ \min_{\mu}\vert A \vec v- \mu \vec v\vert^2 = \min_{\mu} \vert A\vec v\vert^2-2\mu < A\vec v, \vec v > + \mu^2 \vert\vec v\vert^2 $$

deriving regarding $\mu$ the minimum condition gives

$$ -2< A\vec v, \vec v >+2\mu \vert\vec v\vert^2 = 0 \Rightarrow \mu = \frac{< A\vec v, \vec v >}{\vert\vec v\vert^2} $$

Answer 2

$$\begin{array}{rl} f(\mu) &= \| A\nu - \mu\nu \|^2 = (A\nu - \mu\nu)^{\top} (A\nu - \mu\nu) \\ &\,= \nu ^\top A^\top A\nu + \nu^\top\mu^\top \mu\nu - \nu ^\top A^\top \mu\nu - \nu^\top\mu^\top A\nu \end{array}$$

Differentiating with respect to $\mu$,

$$\nabla_ \mu f(\mu)= 2 \mu \nu^\top \nu -2 \nu^\top A \nu $$

The if $(\nu^\top\nu)$ is invertible then the derivative is null for,

$$\mu^* = \frac{\nu^\top A \nu }{\nu^\top\nu}$$

Another way to look at it assuming you already know the formula of the least square solution,

$$ \underset{\beta}{\mathrm{argmin}} \| Y-X\beta \| = (X^\top X)^{-1}X^ \top Y $$

Then,

$$ \underset{\beta}{\mathrm{argmin}} \| Y^\top - \beta^\top X^\top \| = (X^\top X)^{-1}X^ \top Y $$

$Y^\top = A\nu,\ X^\top=\nu,\beta^\top=\mu $

And you can retrieve your optimization problem.