I seek to find a matrix $\mathbf{X}$ which minimizes
$g(||\mathbf{A}_1-\mathbf{XB}_1||_F, ||\mathbf{A}_2-\mathbf{XB}_2||_F)$,
where $g$ is some convenient norm (e.g. $L^1$-norm, $L^2$-norm or $L^\infty$-norm).
Suppose, it is hard to minimize $g$ directly, but easy to find $\mathbf{X}_1$ which minimizes $||\mathbf{A}_1-\mathbf{XB}_1||_F$ and, separately, also easy to find $\mathbf{X}_2$ which minimizes $||\mathbf{A}_2-\mathbf{XB}_2||_F$.
In this context, is it possible to compute an interpolation $\mathbf{X}_3 = f(\mathbf{X}_1, \mathbf{X}_2)$, such that $\mathbf{X}_3$ could be shown to minimize $g$.
If so, what would the interpolation function $f(\mathbf{X}_1, \mathbf{X}_2)$ be? I am looking for a principled solution, rather than for a heuristic which seems to work.