What exactly does it mean when someone says a matrix may be intrepreted as a set of vectors? As in: "A matrix can be considered a set of vectors, organised as rows or columns"
It seems it would only be a set of vectors if the matrix was multiplied by a unit vector.
It means that you can think of a $n \times m$ matrix as something like this:
$A = \begin{bmatrix} A_1 & A_2 & \cdots & A_m\end{bmatrix}$ = $\begin{bmatrix}B_1 \\B_2 \\ \vdots \\ B_m\end{bmatrix}$ where each $A_i$ is an $n$-dimensional column vector and each $B_i$ is an $m$-dimensional row vector.
For example:
$\begin{bmatrix}1 & 9 & -13 \\20 & 5 & -6 \end{bmatrix}$
You can think of it as $\begin{bmatrix}A_1 \\A_2 \end{bmatrix}$ where $ A_1=\begin{bmatrix}1 & 9 & -13\end{bmatrix}$ and $A_2 = \begin{bmatrix}20 & 5 & -6 \end{bmatrix}$ or you can think of it as $\begin{bmatrix} B_1 & B_2 & B_3 \end{bmatrix}$ where $B_1 = \begin{bmatrix}1 \\20 \end{bmatrix}$, $B_2 = \begin{bmatrix}9 \\5 \end{bmatrix}$ and $B_3 = \begin{bmatrix}-13 \\-6 \end{bmatrix}$.