Assume that you have the following convex optimization problem:
$\min_{M} \|b+A\ M\ v\|_2$
subject to :
$\|M\|_{2}<1$ (maximum singular value less than 1)
where M is a suare matrix (n by n), A is a tall matrix (m by n) where $m>n^2$ and b and v are vectors.
Now assume that you find the optimum matrix M (M is square) for two sets of given vectors b and v and matrix A. In other words, $M^{const.}_1$ solves the problem for given $b_1$,$v_1$ and $A_1$.
Similarly you find another solution for the second set of given parameters: $M^{const.}_2$ solves the problem for given $b_2$,$v_2$ and $A_2$.
Now consider the unconstrained problem which is nothing but:
$\min_{M} \|b+A\ M\ v\|_2$
again you find another two matrices namely $M^{unconst.}_1$ and $M^{unconst.}_2$ which solves the unconstrained problem for the two sets of {$b_1$,$v_1$ and $A_1$}, and {$b_2$,$v_2$ and $A_2$.} respectively.
The question here is, can we say something like this?
$\|M^{const.}_2-M^{const.}_1\|<\|M^{unconst.}_2-M^{unconst.}_1\|$
In other words, the distance of the solutions in constrained problem is smaller than the distance of the two in unconstrained problem.
How about this approach? I use @davcha 's idea to convert the matrix to vector m. so we have the following unconstrained optimization problem:
$\min_{m} \|b+V\ m\|_2$ where V denotes $(v^{T}*A)$.
Now by decomposing matrix $V=\tilde{V}\ T$, where $\tilde{V}^{T}\tilde{V}=I$, leads to
$\min_{m} \|b+\tilde{V}\ \tilde{m}\|_2$
where $\tilde{m}=T\ m$.
So now by this transformation, the cost function contours are spherically symmetric (they are circles rather than ellipse). Thus, here the following inequality should be valid:
$\|\tilde{m}^{const.}_2-\tilde{m}^{const.}_1\|<\|\tilde{m}^{unconst.}_2-\tilde{m}^{unconst.}_1\|$.
am I right?