What are $\vec{x}$ and $\vec{b}$ in $A \vec{x} = \vec{b}$?

Question

The condition of a matrix gives a bound on how inaccurate the solution $\vec{x}$ will be after approximation. So given $A \vec{x} = \vec{b}$, how much does $\vec{x}$ change when there is a change in $\vec{b}$. Here is an example:

$$\pmatrix{1&1\\1&1.001} \pmatrix{x\\y} = \pmatrix{2\\2} \implies x=2, y=0$$ $$\pmatrix{1&1\\1&1.001} \pmatrix{x\\y} = \pmatrix{2\\2.001} \implies x=1, y=1$$

This is an ill-conditioned matrix because a small change in $\vec{b}$ lead to a big change in $\vec{x}$. But here's where I am confused. The matrix $A$ can be considered as observed values or input data. However, I am not sure what $\vec{x}$ and $\vec{b}$ are in this case.

Answer 1

This question isn't really about condition number at all, but rather "how do I interpret a system of linear equations". In general, $A$ is a matrix of coefficients, $\vec{x}$ is a vector of unknowns, and $\vec{b}$ is a vector of constants. The single matrix equation you have above relates to two linear equations in two unknowns. Specifically:

$$x + y = 2$$

and

$$x + 1.001y = 2$$ (or 2.001, depending on which version you are using)

So one (geometric) interpretation of this matrix equation is to find the intersection points between the two lines $y = -x + 2$ and $y = \frac{-1}{1.001}x + 2$. In this sense, the vector $\vec{b}$ represents the y intercepts. Of course, this assumes that your quantities are in $\mathbb{R}^2$ and that you are able to use this interpretation for useful results.

Otherwise, "what do these quantities represent" is answered by "whatever they need to for your application".

Answer 2

I don't see how the matrix $A$ constitutes either observed values or input data in an example like this. A matrix in a context like this is a relationship between two sets of values

Often $A$ is a model of how we think two things are related. These two things could be the input to some process and the output of the process. So if you are able to observe the output of the process but not the input, and you want to estimate what the input might be, you could set the values of $\vec b$ to the observed output values and let $\vec x$ be the unknown input values, and then you would want to solve for the unknown $\vec x$ in the system $$ A \vec x = \vec b.$$

Anyway, if you're looking for two distinct things called "observed values" and "input data" in the equation $A \vec x = \vec b$, my first guess would be that $\vec b$ is the "observed values" and $\vec x$ is the "input data."