We have been going through how to solve the system of equations known as $Ax=b$, where $A$ is a matrix, $x$ is a vector and $b$ is a vector. I understand that if we have $A$ and $b$ we must find out what $x$ is, this happens via Gauss-Jordan elimination, back substitution, etc.
What does solving the linear system of equations actually mean though? Is it the point in space where all of the vectors of $A$ intersect, and what will this be useful for? Real-life examples are appreciated!
On
Given a matrix equation $$A {\bf x} = {\bf y},$$ where $A$ is $m \times n$, $\bf x$ is $n \times 1$ (and so we write ${\bf x} \in \mathbb{R}^n$), and $\bf y$ is $m \times 1$ (${\bf y} \in \mathbb{R}^m$), there are several interpretations, including:
If we think of the matrix $A$ as the map $\mathbb{R}^n \to \mathbb{R}^m$ defined by ${\bf u} \mapsto A{\bf u}$, then a solution is any vector ${\bf x} \in \mathbb{R}^n$ that the function maps to $b$.
If we think of the columns of $A$ as vectors in $\mathbb{R}^n$, a solution is a vector ${\bf x} = (x_1, \ldots, x_n)^T$ whose components are coefficients in a linear combination of the columns $({\bf a}_j)$ of $A$ that give $\bf b$, or more precisely, are such that $${\bf b} = x_1 {\bf a}_1 + \cdots + x_n {\bf a}_n.$$
If we think of the rows of $A$ as (row) vectors $(a_{i1}, \ldots, a_{in})$ in $\mathbb{R}^m$, and write ${\bf b} = (b_1, \ldots b_m)$, then the $i$th row of the linear system is a (scalar) equation $$a_{i1} x_1 + \cdots + a_{in} x_n = b_i,$$ which defines a hyperplane in $\mathbb{R}^n$. A vector ${\bf x}$ is a solution of this scalar equation if it lies in this hyperplane, and so it is a solution of the original system (which we now view as consisting of $m$ separate equations) precisely if it lies in the intersection of these hyperplanes.
It's hard to overstate the important of solving linear equations, and linear algebra in general. They are utterly ubiquitous, both because
Some applications (this list could not hope to be exhaustive, or even representative) include:
And really,
On
Ok, so suppose you want to solve $\begin{bmatrix} 1 & 2 & 5 \\ 2 & 0 & 9 \\ 0 & 1 & 1 \\ \end{bmatrix} \begin{bmatrix} x \\ y \\ z \\ \end{bmatrix} = \begin{bmatrix} 4 \\ 8 \\ 7 \\ \end{bmatrix}$. As you know, we are trying to find the vector $\begin{bmatrix} x \\ y \\ z \\ \end{bmatrix}$ that makes this equation work.
Well, if you worked out the multiplication in $Ax$ and then rearranged a little, you would see that the product on the left is just:
$$x \begin{bmatrix} 1 \\ 2 \\ 0 \\ \end{bmatrix} + y \begin{bmatrix} 2 \\ 0 \\ 1 \\ \end{bmatrix} + z\begin{bmatrix} 5 \\ 9 \\ 1 \\ \end{bmatrix}$$
which gives the equation
$$x \begin{bmatrix} 1 \\ 2 \\ 0 \\ \end{bmatrix} + y \begin{bmatrix} 2 \\ 0 \\ 1 \\ \end{bmatrix} + z\begin{bmatrix} 5 \\ 9 \\ 1 \\ \end{bmatrix} = \begin{bmatrix} 4 \\ 8 \\ 7 \\ \end{bmatrix}.$$
So what we are doing when solving $Ax = b$ is finding the scalars that allow $b$ to be written as a linear combination of the columns of $A$. In other words, how is $b$ represented in the span of the columns of $A$ (i.e., the column space of $A$)?
If there is no solution to the equation, then it must be that $b$ is not in the column space of $A$.
On
A system $$Ax=b\tag{1}$$ of equations in unknowns $x_1$, $x_2$, $\ldots$, $x_n$ implicitly defines the subset $$S:=\{x\in{\mathbb R}^n\>|\>Ax=b\}\quad\subset{\mathbb R}^n\ .$$ "Implicitly" means that for any given $x\in{\mathbb R}^n$ it is easy to test whether it is an element of $S$ or not (just compute $Ax$ and check whether this is $=b$); but you don't have a-priori a complete overview over this set $S$.
Solving the system $(1)$ (or a similar system containing equations of a more complicated nature) means obtaining such an overview. If $S$ is in fact the empty set you want a proof of this fact; if $S$ contains just finitely many points you want an explicit list of these points, etc.
In linear algebra the favorite case is when $S$ is a one-element set $\{a\}$; we then call $a$ "the" solution of $(1)$. But it very often happens that $S$ is an infinite set; say, a two-dimensional plane embedded in ${\mathbb R}^n$. In such a case you want an explicit "production scheme" with a certain number of free variables, in other words: a parametric representation of $S$, which generates every point of $S$ exactly once. In the case where $S$ is a two-dimensional plane such a parametric representation looks like $$S:\quad(u,v)\mapsto x:=a+up+vq \qquad\bigl((u,v)\in{\mathbb R}^2\bigr)\ ,\tag{2}$$ whereby the vectors $a$, $p$, $q$ have to be computed from the data $A$ and $b$. (Note that the same $S$ has many different representations $(2)$.)
There are productive ways to think about the solutions of the linear system $Ax = b$ geometrically. There is already a link above on this.
However most applications of linear systems are not geometric. They come up everywhere we have any kind of mathematical modeling: physics, chemistry, biology, medicine, epidemiology, computer science, all types of engineering, as well as finance and economics. Linear systems also appear in fields as disparate as political science, geography or literature, where recent statistical models have been built to determine if a text was written by Shakespeare. And of course, it is also a useful tool in all kinds of higher mathematics.
Knowing how to solve $Ax = b$ is a basic tool in your quantitative toolbox.