If $A^TA = B^TB$ where $B$ is non-square, does $\exists$ matrix $P$ such that $A=PB$?

Question

This question has been answered before for the case where $A$ and $B$ are both square: $A^TA=B^TB$. Is $A=QB$ for some orthogonal $Q$?

I didn't really follow the algebraic explanation there but the geometric explanation in terms of hyperspheres is really nice.

Unfortunately, if $A$ is not square, then we can't use the hypersphere argument since $A$ and $B$ will map to spaces of different dimensionality. Furthermore, $P$ will not be orthogonal as it is in the above case.

So, let's suppose $A$ has dimension $k \times k$ and $B$ has dimension $d \times k$ with $k<d$. Is is still true that there exists a matrix $P$ such that $A=PB$? $P$ will need to have dimension $k \times d$.

Answer 1

Let $B\in\mathbb R^{d\times k}$, where $k<d$. Let $M$ be a subspace of $\mathbb R^d$ such that $\text{im}B\subset M\subset\mathbb R^d$ and $\dim M = k$. Then there is a unitary operator $W : M\to\mathbb R^k$. Extend $W$ by setting $Wx =0$ for $x\in M^\perp$. Then $W : \mathbb R^d\to\mathbb R^k$ such that $W^TWy = y$ for $y\in M$. *)

Consider $C = WB\in\mathbb R^{k\times k}$. Then $C^TC = B^TW^TWB = B^TB$ and therefore $WB = C = Q(C^TC)^{1/2} = Q(B^TB)^{1/2}$. Hence, $B = W^TQ(B^TB)^{1/2}$. Setting $V = W^TQ$, you have $B = V(B^TB)^{1/2}$ with an isometry $V\in\mathbb R^{d\times k}$ (i.e., $V^TV = I_k$). Hence $$ A = U(A^TA)^{1/2} = U(B^TB)^{1/2} = UV^TB. $$

---

*) To see this, let $y\in M$ and $x\in \mathbb R^d$, $x = u+v$ with $u\in M$ and $v\in M^\perp$. Then $$ (W^TWy,x) = (Wy,Wx) = (Wy,Wu) = (y,u) = (y,u+v) = (y,x). $$ Thus, $W^TWy = y$.

Answer 2

The symmetric positive semi-definite matrix $A^TA=B^TB$ has a minimal diagonalization $V\Sigma^2V^T$, minimal in the sense of leaving out zero eigenvalues. Then $V$ is isometric, $V^TV=I_k$ and $Σ$ is diagonal of full rank $k$.

As in the classical construction of the SVD, where we aim at a factorization $A=U_AΣV^T$, $B=U_BΣV^T$, construct the matrices $U_A=AVΣ^{-1}$, $V_B=BVΣ^{-1}$, then $U_A^T U_A^{\,}=I_k$, $U_B^TU_B=I_k$ so that also these matrices are also isometric. To actually be the missing factors of the SVDs, one would have to prove that $A=[AVΣ^{-1}]ΣV^T=AVV^T$, that is, that the projector $(I-VV^T)$ has its image in the kernel of $A$, which is a standard fact for the orthogonormal eigenbasis of $A^TA$.

In conclusion, with $P=U_AU_B^T$ we get $$ PB=U_AU_B^T(U_BΣV^T)=U_AΣV^T=A. $$

Answer 3

Since you like the geometrical argument, I want to point out that there actually is an argument with hyperspheres and relate this geometrical viewpoint with the other answers.

Consider a matrix $B \in \mathbb{R}^{d\times k}$, which can be regarded as a a linear map $B\colon\mathbb{R}^k \to \mathbb{R}^d$. By the SVD, $B$ maps the $(k-1)$-dimensional hypersphere $S^{k-1}$ in $\mathbb{R}^k$ onto a $(n-1)$-dimensional ellipsoid in $\mathbb{R}^d$, where $n$ is the number of non-zero singular values of $B$.

• Case 1. $k \leq d$. $B(S^{k-1})$ is an ellipsoid lying in an $k$-dimensional subspace $M$ of $\mathbb{R}^d$. If $B$ is full-rank, all singular values are non-zero and $\mathrm{im}\,B = M$. The map $B$ is an embedding of the sphere into $\mathbb{R}^d$. If $B$ doesn't have full rank, $\mathrm{im}\,B$ is a subspace of $M$. Note that in amsmath's answer the map $W$ describes how $M$ lies in $\mathbb{R}^d$.

• Case 2. $k > d$. Also in this case there is an interpretation. The intersection of $S^{k-1}$ and $(\ker B)^\perp$ is a subsphere. This subsphere is mapped onto an ellipsoid in $\mathbb{R}^d$. So $B$ can be regarded as a kind of projection of the hypersphere onto a subsphere, and this subsphere is then mapped onto an ellipsoid.

Now let's look at your question. By the condition $A^T A = B^T B$, the ellipsoids $A(S^{k-1}) \subset \mathbb{R}^k$ and $B(S^{k-1}) \subset \mathbb{R}^d$ have axes with equal lengths. So the matrix $P$ maps the axes of the ellipsoid $B(S^{k-1})$ onto the corresponding axes of $A(S^{k-1})$. And this is exactly what the map $P$ does. Using Lutzl's notations, $P=U_A U_B^T$. Now, $U_B^T$ maps unit vectors in the directions of the axes of $B(S^{k-1})$ to the standard basis in $\mathbb{R}^k$. Consequently $U_A$ maps this basis onto the unit vectors in the directions of the axes of $A(S^{k-1})$.