Given the solution space V,
$$ \left\{ \begin{array}{c} x_1+2x_2-x_3-5x_4=0 \\ -x_1+3x_3+5x_4=0 \\ x_1+x_2+ax_3+bx_4=0 \end{array} \right. $$
By applying Gaussian Jordan Elimination, I get from $$ \left[ \begin{array}{cccc|c} 1&2&-1&-5&0\\ -1&0&3&5&0\\ 1&1&a&b&0 \end{array} \right] $$
to $$ \left[ \begin{array}{cccc|c} 1&0&-3&-5&0\\ 0&1&1&0&0\\ 0&0&a+2&b+5&0 \end{array} \right] $$ If dim(V) = 2, I can assume that a+2 = 0 and b+5 = 0, to give 2 pivot columns, hence a = -2 and b = -5.
However, if dim(V) = 1, how do I go about solving for a and b, as I can't seem to find a way to further reduce the above matrix? Also, is it possible if dim(V) take on other values e.g. 0 and 3?
The dimension of the column space and the dimension of the row space of a matrix are always the same. Since the first two columns are independent from each other, the column space will always have dimension at least 2, and since there are only 3 rows, the row space can have dimension at most 3. Thus we can say the rank is at least 2 and at most 3.