Say I want to solve $$\min_{\bf x}\|\bf Ax-b\|_2^2$$ This problem is well defined and easy to solve using linear least squares. But what if we suddenly have
$$\min_{\bf x,A}\|\bf Ax-b\|_2^2$$
Looking at $\bf x$ and $\bf A$ separately this is as simple as the above problem, but what about now when we allow both to vary?
Own work
A naive approach I have tried which seems to work well enough in some cases is to simply do alternatingly solving:
$${\bf x}_k \text{ from } \min_{\bf x}\|{\bf Ax-b}\|_2^2\hspace{1cm} {\bf A}_k \text{ from } \min_{\bf A}\|{\bf Ax-b}\|_2^2$$
Maybe if we assume "sufficient niceness" in some sense of $\bf b$ then this would guarantee to give optimum? Or if we take small enough steps we can refer to the idea of continuation? In other words regularizing
$${\bf x}_k \text{ from } \min_{\bf x}\|{\bf Ax-b}\|_2^2 + \epsilon \|{\bf x-x}_{k-1}\|_2^2$$
And analogously for $\bf A$?
Note that, $$ A = \begin{bmatrix} | & 0 & \cdots & 0 \\ b & \vdots & & \vdots \\ | & 0 & \cdots & 0 \end{bmatrix} ,~~ x = \begin{bmatrix} 1 \\ 0 \\ \vdots \\0 \end{bmatrix} $$ satisfies $Ax = b$.
More generally, whenever $x\neq0$ you you can always find $A$ so that $Ax=b$ but having one of the columns of $A$ be an appropriately scaled version of $b$. However, since you have this as an interpediate step in your current approach, perhaps I have not understood what you are trying to do.