Given tall matrix $A \in \mathbb R^{n \times k}$ (where $n \gg k$), is there a way to solve the following system of matrix equations in $X \in \mathbb R^{n \times k}$?
$$\begin{aligned} X X^T A &= A\\ X^T X &= I_k\end{aligned}$$
Matrix $A$ is probably of rank $k$.
On
From $X^TX = I$, we know that the matrix $X$ has $k$ mutually orthogonal bases. Also, the matrix $A$ has $k$ vectors maybe linearly independent.
To minimize the distance between the two sets of vectors, since the length of the column vectors of $X$ is 1, the spanning space of column vectors of $X$ should be the same as $A$.
Thus there exists a $k \times k$ invertible matrix $H$, makes $X = AH$, and $X - A = AH - A$, which lefts $H$ to optimization for: $$ \min_{H} \| AH - A\|_F^2 $$ with the constraint $AH^THA^T = I$. The whole thing might become easier for the Lagrange multiplier method and further numerical optimization.
On
According to the comments posted by @Jean-ClaudeArbaut, we can do the SVD $\mathbf A = \mathbf{USV}^\top$, then set $\mathbf X = \mathbf U \mathbf V^\top$. It is easy to check that it is indeed a solution. $$ \mathbf X^\top\mathbf X = \mathbf V \mathbf U^\top \mathbf U \mathbf V^\top = \mathbf I $$ $$ \mathbf{XX}^\top \mathbf A = \mathbf{UV}^\top \mathbf{VU}^\top \mathbf{USV}^\top = \mathbf{USV}^\top = \mathbf A $$
The optimization problem you posted in the comment is minimizing the Euclidean norm on the space of matrices of given form.
Now the entries of $X^TX$ are polynomials of total degree two in the entries of $X$, so you have a subset of $\mathbb{R}^{n \cdot k}$ given by polynomial equations, this is relatively nice and you should be able to get lots of properties of this set.
There are lots of results on how to find the distance between a set and a given point in an Euclidean space. Thus, I think going down this road will give you better results than trying to solve the matrix equation.