Suppose I have two matrices A, B
$$ A \in \mathbb{R}^{m \times n}, B \in \mathbb{R}^{m \times n} $$ Then on what conditions on A and B will the followig ineqiality hold: $$ ||A+B||_2 \geq ||A||_2 $$
I for some reason feel that this would hold when the spaces spanned by the columns these matrices are different. But maybe I am wrong, or I cannot mathematically articulate it.
You are wrong. In particular, if we take $$ A = \pmatrix{1&100\\0&0}\\ B = \pmatrix{-1&-100\\1/100&1} $$ Verify that the column space of the two matrices is different, but $\|A + B\| < \|A\|$.