This question is mainly to confirm my intuition (since my proof is counter intuititive).
Suppose I have a least squares formulation, $AX=B$ where $A \in \mathbb{R}^{d \times d}$, $X \in \mathbb{R}^{d \times n}$ and $B \in \mathbb{R}^{d \times n}$ (here n < d).
I want to solve for $X$, but here is the catch $A$ is both idempotent and singular. So not invertible. And the pseudo-inverse isn't clearly defined either since $A^TA = A$ is also singular.
I try to take advantage of the idempotent property and do the following
$AX = B$
$AAX = AB$
$AX = AB$
$A(X-B) = 0$
which will give tell me that $X=B$. This also implies that $AB = B$.
This doesn't seem quite right to me, would love to hear anyone's thoughts on this.
EDIT: thanks to @Qiaochu Yuan, its clear why the above proof is wrong. The follow up question is, does $X$ have a well-defined solution?