How to find $A$ from $Ax=b$?

Question

I am aware of the inverse problem of the form $Ax=b$ where matrix $A$ and vector $b$ are known and we need to estimate the vector $x$. Is there any formal methods to find matrix $A$ given $b$ and $x$? I can understand how ill-posed it may be, but is there any studies about it?

Specifically, is there specific significance of matrix $A$ with highest $L_p$ norm, or sparsest matrix $A$ or even $A$ with specified singular values?

Answer 1

It's just a linear system (underdetermined) in the coefficients $A_{ij}$.

Answer 2

We have a linear system in $\mathrm A \in \mathbb R^{m \times n}$

$$\mathrm A \mathrm x = \mathrm b$$

where $\mathrm x \in \mathbb R^{n}$ and $\mathrm b \in \mathbb R^{m}$ are given. Vectorizing the left-hand side, we obtain the linear system

$$(\mathrm x^T \otimes \mathrm I_m) \, \mbox{vec} (\mathrm A) = \mathrm b$$

which is written in a more familiar form. We have $m n$ unknowns and only $m$ equations, i.e., if $n > 1$ then we have an underdetermined linear system. The null space of $\mathrm x^T \otimes \mathrm I_m$ has dimension at least $m (n-1)$. Assuming that $n > 1$, then there are infinitely many solutions.

We can search for the matrix $\mathrm A$ with the minimum Frobenius norm via $2$-norm minimization

$$\begin{array}{ll} \text{minimize} & \|\mbox{vec} (\mathrm A)\|_2\\ \text{subject to} & (\mathrm x^T \otimes \mathrm I_m) \, \mbox{vec} (\mathrm A) = \mathrm b\end{array}$$

or we can search for a sparse matrix $\mathrm A$ via $1$-norm minimization

$$\begin{array}{ll} \text{minimize} & \|\mbox{vec} (\mathrm A)\|_1\\ \text{subject to} & (\mathrm x^T \otimes \mathrm I_m) \, \mbox{vec} (\mathrm A) = \mathrm b\end{array}$$

In MATLAB, use function kron to do the Kronecker product $\otimes$ and reshape to vectorize (and un-vectorize).

Answer 3

The simplest solution is surely a diagonal matrix A whose diagonal elements can immediately be computed - when no zeros in $x$ are involved.

However, if zeros in $b$ or in $x$ are involved $A$ may not be possible to make diagonal.

In some extreme case: where $b$ is completely zero but not $x$, then a special solution would be needed; for instance that $A$ is so, that $x$ is an eigenvector of $A+ \lambda \cdot I$