I keep coming accross this statement that the set of rank-$r$ $m \times n$ matrices form a manifold. In both sources referenced below, the authors say "it is known", but I am having trouble finding the justification.
The statement can be seen in Section 3 eq. 1 here, and on slide 11 here.
This is a standard exercise (see, for example, exercise 13 on p. 27 of Guillemin and Pollack's Differential Topology). It is an application of the preimage theorem. If you assume the matrix is of the form $$X=\left[\begin{array}{c|c}A & B \\\hline C & D \end{array}\right],$$ where $A$ is $r\times r$ invertible, then show that $X$ has rank $r$ if and only if $D-CA^{-1}B = 0$. (So you need to show that $0$ is a regular value of the map $M_{m\times n} \to M_{(m-r)\times (n-r)}$.)