Can someone point out what I am fundamentally doing wrong in this question?
Consider the vector space $\mathbb{R}^3$ with the standard inner product (dot product) and let $H=span\left\{(2,1,0),(0,1,2)\right\}$ be a subspace of $\mathbb{R}^3$. If $(4,12,8)=u+v$, with $u$ in H and $v\epsilon$H$^\perp$, then $||v||$ equals:
$(A)$ $\sqrt6$
$(B)$ $1$
$(C)$ $3$
$(D)$ $\sqrt{24}$
$(E)$ $\sqrt3$
So my reasoning was that I needed what $||v||$ was. $v$ is the orthogonal complement of $H$. Since the vectors in $H$ are written as rows, we can find the Null space of $H$ in order to determine the orthogonal complement and hence $v$.
Thus:
$ \left[ \begin{array}{ccc|c} 2 & 1 & 0 & 0 \\ 0 & 1 & 2 & 0 \\ \end{array} \right] $
$...$
$ \left[ \begin{array}{ccc|c} 1 & 0 & -1 & 0 \\ 0 & 1 & 2 & 0 \\ \end{array} \right] $
Therefore, $x_1=t,x_2=-2t,x_3=t$ and then:
$v=span\left\{(1,-2,1)\right\}$
Thus, $||v||=\sqrt{1^2+(-2)^2+1^2}$ $||v||=\sqrt6$
However, the answer says it should be $(D)$, $\sqrt{24}$. Why is that?
You don't get $\|v\|$ by just finding some vector orthogonal to the two given vectors. The formula for $v$ is $v=\frac {|\langle (4,12,8), (1,-2,1) \rangle |} {\ {\|(1,-2,1\|}}$ which works out to $\sqrt {24}$.
When you write the given vector $(4,12,8)$ in terms of the span of $(2,1,0), (0,1,2)$ and its orthogonal complement you will get $(4,12,8)=a(2,1,0)+b(0,1,2)+c(1,-2,1)$ and $v =c((1,-2,1)$. You are missing the coefficient $c$ in your calculation. To find the value of $c$ all you have to do is take the inner product of both sides with $(1,2,1)$. That gives you the formula for $v$.