Column and row independence (or dependence).

Question

Can someone please explain why the number of independent columns equals the number of independent rows? I know that the number of independent columns (or rows) gives the rank of a matrix, but I want a deeper sense of the underlying physical significance of "columns" and "rows" and how they are related.

Answer 1

Matrices are just re-arrangements of linear relationships.

For example, the linear relationship:

$ a_1 x + b_1 y = c_1$

$ a_2 x + b_2 y = c_2$

Is equivalent to the relationship:

$ \begin{pmatrix} a_1 & b_1 \\ a_2 & b_2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} c_1 \\ c_2 \end{pmatrix} $.

Let's make some quick abbreviations:

$ \underline{\mathbf{A}} = \begin{pmatrix} a_1 & b_1 \\ a_2 & b_2 \end{pmatrix} $, $ \mathbf{x} = \begin{pmatrix} x \\ y \end{pmatrix} $, and $\mathbf{c} = \begin{pmatrix} c_1 \\ c_2 \end{pmatrix}$.

And, now we have reached (pretty much) the entire point of the creation of matrices: the ability to write down a relationship (that may relate many, many variables through many, many equations) in one line - saving lots of space and copying effort. Ta-da: $\underline{\mathbf{A}} \mathbf{x} = \mathbf{c}$ !

Then, the columns of the matrix $ \underline{\mathbf{A}}$ describe the influence of the variables, and the rows of the matrix $\underline{\mathbf{A}}$ describe the equations within the relationship.

Now we have a couple of distinct cases:

UNDER-DETERMINED CASE: If you have more variables than equations, the relationships don't actually eliminate enough of the possible values of the variables to narrow the variables down to meaningful possibilities.

OVER-DETERMINED CASE: If you have more equations than variables (and all of the equations include new information), you will have too many requirements on the variables. Applying all of these requirements, removes all of the possible values of the variables.

CONSISTENT CASE: If you have exactly as many equations as there are variables, AND each of these equations includes new information, then you can uniquely solve the linear problem $\underline{\mathbf{A}} \mathbf{x} = \mathbf{c}$ to find $\mathbf{x}$. (This is the case when $\underline{\mathbf{A}}$ has full rank...)

LINEARLY-DEPENDENT CASE: If you have exactly as many equations as there are variables, BUT some of these equations just repeat the information that you already know from other equations in the relationship, then you have some unfortunate circumstances. That is, you still cannot get the unique solution of the linear problem $\underline{\mathbf{A}} \mathbf{x} = \mathbf{c}$, because you only have as much information as in the under-determined case; but you might fool yourself into thinking that it is possible to solve the problem, because you think that you are in the consistent case.

In the end, the matrix rank tells you which case your matrix belongs in, by telling you how many variables you can narrow down, given the information available in the relationship. In the under-determined and linearly-dependent cases, the rank will be equal to the number of equations with new information - because you can't find any more variables than those equations. In the over-determined case, the rank will be equal to the number of variables. In the consistent case, the rank will be equal to both the number of variables and the number of equations.