I need to find the distance of some $3\times 3$ matrix vector, $B$ (over $\mathbb R$) to a subspace defined as $U=\lbrace A\in M_{3\times 3}(\mathbb R)|trace(A)=0\rbrace$.
The classic way is to apply Gram-Schmidt to find an orthonormal basis for $U$, and then calculate $|| B-Pr_{U}(B)||$. But such basis will consist of 8 vectors, so Gram-Schmidt will be quite tedious.
But I thought I could reduce to $W\subset U$, where $W=\lbrace D\in M_{3\times 3}(\mathbb R)|trace(D)=0\rbrace$ and $D$ is diagonal. it's basis has only 2 vectors and since it is contained in $U$, then $|| B-Pr_{U}(B)||$=$|| B-Pr_{W}(B)||$. Is my assumption right?
Another observation: the inner product is defined as $<A,B>=trace(AB^{t})$, so I can also use the fact that $U=\lbrace A\in M_{3\times 3}(\mathbb R)|A\perp \alpha I_{3} , \alpha\in\mathbb R\rbrace$, but I don't see how that helps me.
Your last paragraph is the most useful observation. Think about how you would project a vector onto a hyperplane $\{v \in \mathbb{R}^n : v \perp b\}$ for some fixed $b$. The story is the same here.
I think the projection can be written as $A - \frac{1}{3} \langle A, I_3 \rangle I_3$, i.e. project $A$ onto the orthogonal direction $I_3$, and subtract from $A$.