Solve $Ax \approx b$ for A based on multiple noisy (!) samples of x and b

Question

I'm looking for a way to solve an equation system $A\vec{x} = \vec{b}$ for $A$ based on known values for $\vec{x}$ and $\vec{b}$.

I know that this is undetermined for one sample of $\vec{x}$ and $\vec{b}$ because I have 9 unknowns in $A$ and only 3 equations.

But what if I had three different samples of each $\vec{x}$ and $\vec{b}$? Then I'd have something like:

$$ A\vec{x}^{(1)} = \vec{b}^{(1)}\\ A\vec{x}^{(2)} = \vec{b}^{(2)} \\ A\vec{x}^{(3)} = \vec{b}^{(3)} $$

That way I'd have 9 equations for 9 unknowns and from what I know the system should generally be solvable, is that correct?

What steps do I need to take in order to solve the system for $A$?

Update 22.1.18:

In theory the vectorization approach should work.

In practice, unfortunately, what I did not consider is that the vectors that I have are noisy and since both vector sets come from different sources the noise in $x$ is different from the noise in $b$. So, unfortunately

$$ Ax^{(1)}-b^{(1)} \neq Ax^{(2)} -b^{(2)} \neq 0 $$

So I have to somehow approximate $A$.

Answer 1

The system $$ \pmatrix{ a_{1,1} &a_{1,2} & a_{1,3} \\ a_{2,1} &a_{2,2} & a_{2,3} \\ a_{3,1} &a_{3,2} & a_{3,3} } \pmatrix {x^1_1 \\ x^1_2 \\ x^1_3 } = \pmatrix {b^1_1 \\ b^1_2 \\ b^1_3 } $$

can be rewritten, vectorizing the matrix, as

$$ \pmatrix { x^1_1 & x^1_2 & x^1_3 & 0 & 0& 0& 0& 0& 0\\ 0& 0& 0 &x^1_1 & x^1_2 & x^1_3 & 0 & 0& 0\\ 0& 0& 0 &0& 0& 0 &x^1_1 & x^1_2 & x^1_3 \\ } \pmatrix {a_1 \\ a_2\\ a_3\\ a_4\\ a_5\\ a_6\\ a_7\\ a_8 \\ a _9 } =\pmatrix {b^1_1 \\ b^1_2 \\ b^1_3 } $$

or

$$ \pmatrix { {\bf x^1} & {\bf 0}& {\bf 0}\\ {\bf 0}& {\bf x^1} & {\bf 0}\\ {\bf 0}& {\bf 0}& {\bf x^1} } {\bf a} =\pmatrix { {\bf b^1}} $$

Where ${\bf x^1}$ and ${\bf 0}$ are $1 \times 3$, ${\bf a}$ is $9 \times 1$ and ${\bf b^1}$ is $3 \times 1$

Completing it with the other equations we get

$$ \pmatrix { {\bf x^1} & {\bf 0}& {\bf 0}\\ {\bf 0}& {\bf x^1} & {\bf 0}\\ {\bf 0}& {\bf 0}& {\bf x^1} \\ {\bf x^2} & {\bf 0}& {\bf 0}\\ {\bf 0}& {\bf x^2} & {\bf 0}\\ {\bf 0}& {\bf 0}& {\bf x^2} \\ {\bf x^3} & {\bf 0}& {\bf 0}\\ {\bf 0}& {\bf x^3} & {\bf 0}\\ {\bf 0}& {\bf 0}& {\bf x^3} \\ } {\bf a} = {\bf X} {\bf a} = \pmatrix { {\bf b^1} \\{\bf b^2} \\{\bf b^3} } $$

which we can solve as $ {\bf a} = {\bf X}^{-1} {\bf b} $, provided ${\bf X}$ is nonsingular.

Answer 2

Here is one way to look at it: If you use the $x_i$ as basis vectors, then the $b_i$, expressed in that basis, are the columns of $A$, expressed in that basis. Then you "just" need to transform $A$ back to the standard basis.

PS. It's possible you can save a step here calculation-wise, because the $b_i$ are transformed to the new basis first, and then back again as the columns of $A$.