I am trying to optimize a function of a matrix $X \in \mathbb{R}^{m \times n}$
$$ f(X) = \frac{1}{2} \left\lVert AX - B \right\rVert_F^2 $$
s.t. $X^T X = I_n$ (identity matrix).
As I've been watching few tutorials on the subject I have a rough idea of how to do this. Optimization when dealing with manifold contraints, at least for first order optimization, require an extra operator called "retraction" and because the manifold I have is embedded my understanding is that I should iterate till convergence the two following steps
- Perform normal gradient descent step
- Project onto the manifold by using retraction (in my case this would be the $Q$ matrix coming from $QR$ factorization).
Initialization could be provided by just solving the unconstrained linear least square problem.
As I said I have very little knowledge on optimization on manifold, but I have a problem (similar to the one I gave in this question) and I wonder if in principle my approach is correct.