I tried it out on a $3\times 3$ and a $4\times 4$ matrix and saw that from the third line on all lines are multiples of the second line, so only the first two lines are unique.
E.g. for a $3\times 3$ matrix until $3^2$: $$\left( \begin{matrix}1 & 2 & 3\\ 4 & 5 & 6\\ 7 & 8 & 9 \end{matrix} \right) \Rightarrow \left( \begin{matrix}1 & 2 & 3\\ 0 & 3 & 6\\ 0 & 6 & 12 \end{matrix} \right).$$
However, I don't understand why it is the case and how it would be possible to prove the above statement.
Thank you very much for your help.