Consider two matrices $A$ and $B$ that get stacked to form a (tall) matrix $J$,
$$ J = \left[\begin{array}{l} A\\ B \end{array} \right]. $$
Assume that $\text{rank}(J) = \text{rank}(A) + \text{rank}(B)$ holds (and usually $J$ is full rank). I am trying to show that there always exists a linear transform $T$ such that $JT$ has a kind of block-diagonal structure, i.e. that all columns of $AT$ with non-zero entries are stacked on top of columns with all zero entries of $BT$, and vice versa.
So far, I tried to explicitly construct this transformation from the left singular vectors of $A$ and $B$, but I couldn't yet get this to work. Any hint is much appreciated!
The fact that $\operatorname{rank}(J) = \operatorname{rank}(A) + \operatorname{rank}(B)$ tells you that the rows of $A$ are linearly independent to the rows of $B$. In other words, for all column vectors $x,y$ of the correct size: $A^\top x + B^\top y = 0$ implies that $A^\top x = B^\top y = 0$ (where $A^\top$ denotes the transpose of $A$). By applying row operations to $J^\top$, we can find an invertible matrix $S$ such that $SJ^\top$ is in reduced row-echelon form. Because of the independence condition, we can further conclude that $$ SJ^\top = \pmatrix{R_1 & 0\\0 & R_2}, $$ where $\pmatrix{R_1\\0}$ has the same shape as $A^\top$, $\pmatrix{0\\R_2}$ has the same shape as $B^\top$, and both $R_1,R_2$ are in reduced row-echelon form. Taking the transpose of both sides, we see that $$ JS^\top = \pmatrix{R_1^\top & 0\\0 & R_2^\top}, $$ which is to say that $S^\top$ brings the matrix $J$ into the desired form.