Singular Value Decomposition-noisy data

Question

I have a system of the form

$$Ay=f,$$

where $A$ is a $N\times4$ matrix, $y$ a 4-element array of unknows and $f$ an $N$-element array.

I add Gaussian noise in my data. I tested the following cases:

1. $A$ noise-free and $f$ noisy
2. $A$ noisy and $f$ noise-free
3. $A$ and $f$ both noisy.

For the first case where there is noise only in matrix $f$ everything looks fine. For the other two cases where there is a noise in matrix $A$ I have a systematic error in my results and I can't understand why this happened. I was wondering if anyone have any idea about what is going wrong and give me a hint.