The Eckart–Young theorem provides for us the unique $k$ rank approximation matrix such that,
$$ A_{k} = \arg \min ||A- B|| $$
and
$$||A- A_{k}||_{2} = \sigma_{k+1} $$
Where $A_{k} = \sum_{n=1}^{k} \sigma_{k}u_{k}v_{k}^{H}$. I seem to have found a counterexample, consider the case $k=1$ and the matrix
$$A_{*} = (\sigma_{1}-\sigma_{2})u_{1}v^{H}_{1}$$
Does this not also result in
$$||A-A_{*}||_{2} = \sigma_{2}? $$