It is stated in this answer that conceptualizing linear algebraic systems with column vectors has a minor notational advantage. However, I just noticed that thinking in these terms may conceal an important geometric connection.
Consider the linearly dependent system
$x + 2y = 3$
$4x + 8y = 12$
The slope of both equations in the $xy$-plane is $-\frac{1}{2}$. In $\mathbb{R^2}$ vector space, the two column vectors associated with the LHS of the system uninterestingly span the line $y = 4x$. By contrast, the row vectors span the line $y = 2x$, which is orthogonal to a line with slope $-\frac{1}{2}$. This is not a coincidence; in the linearly dependent case, it seems that row vectors of $2 \times 2$ systems are orthogonal to the graphs of the equations themselves, while column vectors feel unhelpfully disconnected from the original system's geometry.
We can't make this exact claim for linearly independent systems, since all of $\mathbb{R^2}$ would be spanned by either column or row vectors. Nonetheless I wonder, is some important geometric/conceptual connection between the graph of a system's equations and the graph of its vectors lost in the column vector view, perhaps evident in higher dimensional systems?
If you agree that points in $\mathbb{R}^2$ should be column vectors, then it is entirely natural for the $(1, 2)$ and $(4, 8)$ vectors in the system $$\begin{aligned} x + 2y &= 3\\ 4x + 8y &= 12 \end{aligned}$$ to be row vectors. That is because the two things on the left hand side of the equals signs are functions $f_1, f_2: \mathbb{R}^2 \to \mathbb{R}$ where $f_1(x, y) = x +2y$ and $f_2(x, y) = 4x + 8y$, and we are trying to find all the points where $f_1(x, y) = 3$ and $f_2(x, y) = 12$. Functions operating on column vectors are naturally row vectors: we could write the above system like $$ \begin{aligned} \begin{pmatrix}1 & 2 \end{pmatrix} \begin{pmatrix}{x \\ y} \end{pmatrix} &= 3 \\ \begin{pmatrix}4 & 8 \end{pmatrix} \begin{pmatrix}{x \\ y} \end{pmatrix} &= 12 \end{aligned} $$