Uniqueness of matrix up to row permutation

Question

Let $A, B \in \text{Mat}_{n \times m}(\mathbb{R})$ be such that for every vector $x \in \mathbb{R}^m$ it holds that $A x = B x$ up to a permutation of the entries (this permutation may depend on the specific $x$). Can we conclude that $A = B$ up to a permutation of the rows?

Answer 1

I think this should work.

Let $\Sigma$ be the (finite) family of permutations of an $m$-dimensional vector. Partition $\mathbb{R}^m$ into $$\mathbb{R}^m = \bigcup_{\sigma \in \Sigma} \mathcal{X}_{\sigma},$$ where $\mathcal{X}_{\sigma} = \{x \in \mathbb{R}^m : Ax = \sigma(Bx)\}$. Then there must exist at least one $\sigma_0 \in \Sigma$ such that $\mathcal{X}_{\sigma_0}$ contains an open subset of $\mathbb{R}^m$. We can therefore pick $m$ linearly independent elements $x_1, \dots, x_m \in \mathcal{X}_{\sigma_0}$ and construct the invertible matrix $\mathbf{X} = [x_1, \dots , x_m]$. Let $E_{\sigma_0}$ denote the matrix that permutes a vector according to $\sigma_0$. Then, $$A \mathbf{X} = E_{\sigma_0} B \mathbf{X} \quad \Rightarrow \quad A = E_{\sigma_0} B,$$ and we are done.