Let $A$ be a real $n \times n$ matrix, and suppose that $\text{rank}(A)=k$. Consider $$ r_0=\sup \{r>0 \,\, | \,\, |B-A| \le r \Rightarrow \text{rank}(B) \ge k \}.$$
Does $r_0$ depend only on the smallest non-zero singular value $\sigma_{\min}$ of $A$?
Clearly, $r_0 \le \sigma_{\min}$, since if we allow $|B-A| >\sigma_{\min}$, we can replace $\sigma_{\min}$ by zero. (keeping all the other components of the SVD the same).
Is it true that $r_0= \sigma_{\min}$? Even if this is false, I would still like to know whether $r_0$ depends only on $\sigma_{\min}$.
If you choose the $||.||_2$, then your conjecture is true, that is,
If $||B-A||_2<\sigma_k$, then $rank(B)\geq k$. Moreover, there is a matrix $B$ s.t. $rank(B)=k-1$ and $||B-A||_2=\sigma_k$.
cf. Eckart-Young-Mirsky Theorem
https://en.wikipedia.org/wiki/Low-rank_approximation#Proof_of_Eckart%E2%80%93Young%E2%80%93Mirsky_theorem_(for_spectral_norm)