I have the following problem to solve, but can't.
$X^T(Xb-y)=0$ where $X$ is an $n$-by-$p$ unknown non-zero matrix, $b$ is a known vector in $\mathbb{R}^{p}$, and $y$ is also a known vector in $\mathbb{R}^{n}$. I am trying to find what $X$ could be. Note that $Xb-y \neq 0$. The only thing that comes to my mind is that $Xb-y$ must be in the nullspace of $X^T$, but I don't know how to proceed from there since there is no way to find the nullspace of X since it is unknown.
Any help is appreciated!
If $y=0$, then $X^T(Xb-y)=0$ iff $Xb=0$, i.e. iff the rows of $X$ are orthgonal to $b$.
If $b=0$, then $X^T(Xb-y)=0$ iff $X^Ty=0$, i.e. iff the columns of $X$ are orthgonal to $y$.
If both $y$ and $b$ are nonzero, pick two orthogonal matrices $U$ and $V$ whose first columns are $\frac{y}{\|y\|}$ and $\frac{b}{\|b\|}$ respectively. Let $A=\sqrt{\frac{\|b\|}{\|y\|}}UXV$. Denote the first vectors in the standard bases of $\mathbb R^n$ and $\mathbb R^p$ by $\mathbf e_1^{n}$ and $\mathbf e_1^{p}$ respectively. Then $X^T(Xb-y)=0$ if and only if $$ A^T(A\mathbf e_1^{p}-\mathbf e_1^{n})=0.\tag{1} $$ So, if you write the first column of $A$ as $\pmatrix{a_{11}\\ \mathbf a}$ where $\mathbf a\in\mathbb R^{n-1}$, then equation $(1)$ implies that $a_{11}^2-a_{11}+\|\mathbf a\|^2=0$. Therefore the general solution to $(1)$ is given by an arbitrary $A$ such that $\|\mathbf a\|\le\frac14$, $a_{11}^2-a_{11}+\|\mathbf a\|^2=0$ and all but the first columns of $A$ are orthogonal to $\pmatrix{a_{11}-1\\ \mathbf a}$. Having chosen $A$, we get $X=\sqrt{\frac{\|y\|}{\|b\|}}U^TAV^T$.