Find the minimum $\min\limits_x\Vert y-Ax\Vert_2^2$ given $A$'s singular value decomposition

63 Views Asked by At

Let $A\in \mathbb R^{5\times 3}$ with left singular vectors $u_1,\ldots,u_5$, right singular vectors $v_1,\ldots,v_3$ and singular values $\sigma_1,\sigma_2,\sigma_3=4,2,1$

  • Suppose $y=2u_1+3u_2+4u_3+5u_5$. Find the minimum $ \min\limits_x\Vert y-Ax\Vert_2^2$

  • Find $\hat x=\operatorname*{arg min}\limits_x \|y-Ax\|_2^2$. Write $\hat x$ as a linear combination of the vectors $v_j$

  • Is the vector in part (b) the unique minimum? Explain

I know that $A$ can be written as $A=\left(\begin{matrix}|&|&|&|&|\\u_1&u_2&u_3&u_4&u_5\\|&|&|&|&|\end{matrix}\right)\left(\begin{matrix}4&0&0\\0&2&0\\0&0&1\\0&0&0\\0&0&0\end{matrix}\right)\left(\begin{matrix}-v_1^*-\\-v_2^*-\\-v_3^*-\end{matrix}\right)$ but that's about it. I was lamenting that A cannot be written in the form $\sum_i\sigma_iu_iv_i^*$. I wrote $y-Ax=2u_1+3u_2+4u_3+5u_5-U\Sigma V^*x$, then I tried to compute $U\Sigma V^*x=4v_1^*xu_1+2v_2^*xu_2+v_3^*xu_3$, and then equating the terms, to get $\begin{cases}2=4v_1^*x\\3=2v_2^*x\\4=v_3^*x\end{cases}$, but not sure this helps. Anyone have any ideas?