I have two sequences of matrices which are not diagonal: let say $A_n$ is $m\times p$ and $B_n$ is $p\times k$.
I know that the product $A_n B_n$ converges to $AB$, that is $\Vert A_n B_n - AB\Vert = O( \delta_n)$ with $\delta_n \rightarrow 0$ as $n\rightarrow \infty$ for some norm $\Vert \cdot \Vert$, for example the Froebenuis norm.
I also know that $\Vert A_n \Vert$ and $\Vert B_n \Vert$ are bounded and for identifiability $A'A= I $.
Is it true that, under some conditions, $\Vert A_n - A\Vert = O(\delta_n) $ and $\Vert B_n - B\Vert = O(\delta_n) $? How can I show it?
I will try to answer my own question. For simplicity, I will assume that $B$ and all $B_n$ are orthogonal. Also all matrices are bounded in norm.
The problem is not unique. For example if the matrices $A_n$ and $B_n$ converges,the product can converge to $AB$ but there is no guarantee that $A_n$ converges to $A$ and $B_n$ to $B$. Indeed, any transformation $A_nD$ and $D^{-1}B_n$ will converge for an invertible $D$: $\lim \Vert A_n D D^{-1} B_n - AB\Vert =O(\delta_n)$.
So we can reformulated the problem. Instead, we want to know if there are matrices $\tilde A_n= A_n D$ and $\tilde B_n= D^{-1} B_n$ such that $\Vert \tilde A_n - A\Vert = O(\delta_n)$ and $\Vert \tilde B_n - B\Vert = O(\delta_n)$.
Let's take $D$ as the minimizer of $\Vert D^{-1} B_n- B \Vert$, which gives $D= B_n B'(B B')^{-1} $. Since $B$ is orthogonal, $D$ should be orthogonal as well and thus $D= B_n B'$. We also see that: $\tilde B_n B' = B B'$.
$\Vert \tilde A_n - A\Vert = \Vert (\tilde A_n - A) B B' \Vert$ $ = \Vert \tilde A_n \tilde B_n B' - AB B' \Vert = \Vert \tilde A_n \tilde B_n - AB \Vert \Vert B \Vert = O(\delta_n) $ since $\Vert B \Vert$ is bounded.
For $\tilde B$, we have $\Vert \tilde B_n - B\Vert = \Vert A \tilde B_n - AB \Vert O(1) = \Vert \tilde A_n \tilde B_n - A \tilde B_n \Vert + \Vert \tilde A_n \tilde B_n - AB \Vert O(1)\\ = O(1) \Vert \tilde B_n \Vert \Vert \tilde A_n - A \Vert + O(\delta_n) = O(\delta_n)$.