I have a system of equation of the following form:
$$ {\bf W} \vec{x} = \vec{y} $$
W is a thin matrix (i.e. has more rows than columns) and $\vec{x}$, $\vec{y}$ are vectors of appropriate size.
I solved this equation using the pseudo-inverse:
$$ \vec{x} = ({\bf W})^\dagger \ \vec{y} $$
We know that by definition the pseudo-inverse gave the solution that minimized: $$|| {\bf W} \vec{x} - \vec{y}||$$
However, now that I found the values of my vector $\vec{x}$, I would like to find the maximum amount of change that I could apply to the vector $\vec{y}$ that would keep the values of $\vec{x}$ constant and would minimize the distance $|| {\bf W} \vec{x} - \vec{y}|| = || {\bf W} \vec{x} - \vec{y}||$ further.
Is there any way that I could find an analytical solution to this problem?
Clarification:: Let $\epsilon = || {\bf W} \vec{x} - \vec{y}||$.
I have two steps to my problem: First I would like to find my response variable $\vec{x}$ which I simply find by calculating $({\bf W})^\dagger \ \vec{y}$. Now, my problem arises here.
After this step and getting my value $\vec{y}$, I would now like to change the values of $\vec{y}$ to reduce $\epsilon$ while keeping the values of $\vec{x}$ constant.
Edit 1: fixed a typo, the solution to the system is simple the pseudo-inverse of ${\bf W}$ and not the ${\bf W}^T$.
Edit 2: added a clarification.
You can make sure that the residual doesn't change at all simply by moving along a level set. Assume this path is of the form y+f(tau), f(0)=0, simply calculate the gradient of the residual with respect to tau and set to 0: grad ((y+f)'(y+f) - 2(y+f)'Wx + x'W'Wx) = 0 => 2f_t'f + 2f_t'y - 2f_t'Wx = 0 => f_t'(f+y-Wx) = 0
So simply find any vector valued function of tau such that the above is satisfied, then you'll have a level set and the residual will be unchanged.