Is there a more intuitive way to understand matrices and linear algebra?

Question

When learning linear algebar I often blindly follow algorithms to perform operations with matrices without fully understanding concepts behind them. It helped a lot when I started dealing with rotations and got some associations in those terms.

Could you give some intuitive examples illustrating operations with matrices? A good example from the other field is graphing a function point by point to understand its nature.

Answer 1

For matrix multiplication, an example is a plumbing system where water flows in and out of multiple pipes. Let the $p\times q$ matrix $\mathbf{A}$ represent one system of $p$ pipes in and $q$ pipes out where each cell $(i,j)$ represents the fraction of water coming in on pipe $i$ that will leave on pipe $j$. Given a $q\times r$ matrix $\mathbf{B}$ representing another system of $q$ pipes in and $r$ pipes out, the overall system is represented by the $p\times r$ matrix $\mathbf{AB}$.

This is because cell $(i,j)$ in $\mathbf{AB}$ is the dot product of the $i$th row of $\mathbf{A}$ and $j$th column of $\mathbf{B}$ (i.e. the sum of the products of the corresponding elements of the row and column vectors). The $i$th row of $\mathbf{A}$ represents outflows for pipe $i$ of the first system into each of the $q$ pipes of the second system. And the the $j$th column of $\mathbf{B}$ represents the outflows from the $j$th pipe of the second system given inflows into the $q$ pipes of the second system.

Answer 2

The multiplication of a matrix $A$ with a vector $x$, $Ax$ can be seen as a (linear) function or transformation of the vector. Matrix multiplication of two matrices $AB$ can be seen as the composition of these linear functions: First we apply the transformation $B$ to $x$, and get a new vector $Bx$. Then, we apply the transformation $A$ to get $ABx$.

There is a class of quadratic matrices which maintain the length of the vector, whose action can be visualized in a nice way, imho. These matrices are called orthogonal matrices; for an orthogonal matrix $Q$ and any vector $x$, we have $\Vert Qx \Vert = \Vert x \Vert$. You already know some of these transformations: rotations and reflections are among them, but for $3 \times 3$ matrices, there are also other length-preserving transformations, e.g. putting the value of the first coordinate into the place of the second, the second in place of the third, and the third in place of the first.

Answer 3

I liked examples about finite state games.

For example imagine you have a game where there are three lights. These lights can only be on or off; and you have 3 buttons which will do different things to the lights.

We represent a light being on with the number $1$ and off with $0$.

We can then represent the action of pushing the buttons onto our configuration of lights with vectors inside $\mathbb{Z}_2^3$, that is vectors with $3$ components which have entries respecting arithmetic modulo $2$. You can see how this works in the following way. Imagine we have $3$ lights and they are side by side. The first light is on, the second is off, and the third is on. We can write this down like so:

$$\text{Light starting config}:=\mathbf{s}=\begin{pmatrix}1\\0\\1\end{pmatrix}.$$ Suppose further that we have access to a button that will switch the first and middle lights (by switch I mean if that light is on after pressing it the light will be off), and does nothing to the last light. This can be represented as: $$\text{button}:=\mathbf{b}=\begin{pmatrix}1\\1\\0\end{pmatrix}.$$ To see why this is true if we add $\mathbf{b}$ to our light configuration what do we get: $$\mathbf{b}+\mathbf{s}=\begin{pmatrix}1\\1\\0\end{pmatrix}+\begin{pmatrix}1\\0\\1\end{pmatrix}=\begin{pmatrix}0\\1\\1\end{pmatrix}=\text{Config after pressing}.$$

As would be expected our representation of pushing the button has changed the configuration of lights so that now the first light is off, the second is on, and the third is on.

What if we have more than one button? Can we make the lights reach a certain configuration with some specific set of buttons and a starting configuration?

We can use the preceding discussion and some more linear algebra to answer these questions. An example will show you what I mean.

So suppose we start with $3$ lights in the same position as before:

$$\mathbf{s}=\begin{pmatrix}1\\0\\1\end{pmatrix}.$$

And three buttons:

$$\mathbf{b}_1=\begin{pmatrix}1\\0\\0\end{pmatrix},\mathbf{b}_2=\begin{pmatrix}1\\1\\0\end{pmatrix},\mathbf{b}_3=\begin{pmatrix}1\\0\\1\end{pmatrix}$$

And we want to know is it possible that after pushing some sequence of buttons we can arrange the lights to be in the configuration: $$\mathbf{d}=\begin{pmatrix}0\\1\\0\end{pmatrix}.$$

If you think about it it doesn't matter which order we push the buttons in and so it matters only if we push the button at all. Therefore what we are asking in terms of our model is whether the following is true:

$$\text{Does there exist $c_1,c_2$ and $c_3$ in $\mathbb{Z}_2$ such that:}\quad c_1\mathbf{b}_1+c_2\mathbf{b}_2+c_3\mathbf{b}_3+\mathbf{s}=\mathbf{d}.$$

Here $c_1,c_2$ and $c_3$ represent if we push buttons $1,2$ and $3$ respectively. We can rewrite the equation as:

$$c_1\mathbf{b}_1+c_2\mathbf{b}_2+c_3\mathbf{b}_3=\mathbf{d}-\mathbf{s}$$

$$\downarrow$$ $$\begin{pmatrix}c_1 & c_2 & c_3\\0 & c_2 & 0\\0 & 0 & c_3\end{pmatrix}=\begin{pmatrix}1\\1\\1\end{pmatrix}$$

After solving this in the usual way (except remember now we are doing arithmetic modulo $2$) you can see if we push each button once we will have put the lights into their desired configuration. If this matrix had no solution then we would not be able to put them into the given configuration.

Answer 4

Matrices represent linear transformations. One of the clearest ways to imagine these transformations is from a geometric perspective. Rotations and reflections are among the easiest transformations to understand this way (as you suggested), but many of the other concepts in linear algebra can be visualized in the same manner.

For example, any similarity transformation relies upon a change of basis, a choice of new axes if you will.

The kernel can be imagined as an infinite line, plane, or volume containing the origin that the transformation maps only to the origin.

Eigenvectors describe lines through the origin that are invariant (within a scaling factor, which is the eigenvalue). Transformations with complex eigenvalues typically describe "eigenplanes" instead: whole planes that do not change under the transformation, even though every individual vector in those planes does.

Row operati9ons are also just fancy ways of doing change of basis.

If you have an understanding of vector calculus, then finding the trace can be understood as finding the divergence of a vector field.

The determinant can be understood the "eigenvalue" corresponding to an "eigenvolume". The infinite volume of the space is dilated or shrunk by some scaling factor under the transformation. That scaling factor is the determinant.