Given $y_1,\ldots,y_n\in \mathbb{R}$,$w\in \mathbb{R}^d$, and $x_1,\ldots x_n\in \mathbb{R}^D$, how do we solve the following optimization problem \begin{align} \min_A \sum_{i=1}^n (y_i-w^TA^Tx_i)^2\\ \text{subject to:}~~~~~ A^TA=I_d \end{align} where $I_d$ is $d$ dimensional identity matrix, and $A\in \mathbb{R^{D\times d}}$, with $D>> d$.
My hope is that a solution to the above problem is possible, using only spectral arguments.
According to the real singular value decomposition we can write $A=U I_{Dd} V$, where U is an orthogonal DxD-matrix, V is an orthogonal dxd-matrix and $I_{Dd}$ is the Dxd-matrix with 1s on the diagonal and zeros elsewhere. Now calculate $$w^TA^Tx_i=w^TU I_{Dd} Vx_i=(w^TU\oplus0)Vx_i=(w^T\oplus0)(U\oplus I)Vx_i.$$ Now we see that optimizing over $U$ and $V$ is the same as optimizing over a single orthogonal matrix, i.e. replacing $w^TA^Tx_i$ by $(w^T\oplus 0)O^Tx_i$ with $O$ orthogonal DxD. Hence the problem is equivalient to \begin{align} \min_e \sum_{i=1}^n (y_i-||w||e^Tx_i)^2 \end{align} optimized over all unit vectors $e\in \mathbb{R}^D$. Now you can use well known linear least squares results.