$M = span\{\begin{pmatrix}8\\0\\-6\end{pmatrix}, \begin{pmatrix}8\\6\\-6 \end{pmatrix}\}$
I am trying to calculate $v_1$ and $v_2$ when $v=\begin{pmatrix}2 \\ 4 \\ 6 \end{pmatrix}$.
I know that since $v_1 \,\in \, M$ and $v_2 \, \in \, M^{\perp}$, $v_1\cdot v_2 = 0$.
Let $v_1 = a_1\begin{pmatrix}8\\0\\-6\end{pmatrix} + a_2\begin{pmatrix}8\\6\\-6 \end{pmatrix}$. The orthogonal basis of $M = \begin{pmatrix}8\\0\\-6\end{pmatrix},\begin{pmatrix}0\\1\\0\end{pmatrix}$. From this, we can say that $v=a_1\begin{pmatrix}8\\0\\-6\end{pmatrix} + a_2\begin{pmatrix}8\\6\\6 \end{pmatrix}+a_3\begin{pmatrix}8\\0\\-6\end{pmatrix}+a_4\begin{pmatrix}0\\1\\0\end{pmatrix}$
But if we put this into an augmented matrix, it gives an inconsistent system (which is clearly wrong). What is the correct way to solve this?
It is quite easy to see that $w=\begin{bmatrix}6\\0\\8\end{bmatrix}$ is orthogonal to the basis vectors given for $M$. Thus $w \in M^{\perp}$. Moreover $\text{dim}(M)=2$ and $M \subset \Bbb{R}^3$, so $\text{dim}(M^{\perp})=1$. This means we can say that $\{w\}$ is a basis for $M^{\perp}$.
So we want to solve for $a,b,c$ such that $$\begin{bmatrix}2\\4\\6\end{bmatrix}=\underbrace{a\begin{bmatrix}8\\0\\-6\end{bmatrix}+b\begin{bmatrix}8\\6\\-6\end{bmatrix}}_{v_1 \in M}+\underbrace{c\begin{bmatrix}6\\0\\8\end{bmatrix}}_{v_2 \in M^{\perp}}.$$ This yields the system \begin{align*} 4a+4b+3c&=1\\ 3b&=2\\ -3a-3b+4c&=3 \end{align*} Upon solving this, we get $a=-\frac{13}{15}, b=\frac{2}{3}$ and $c=\frac{3}{5}$. Thus $$v_1=\color{red}{\frac{-13}{15}}\begin{bmatrix}8\\0\\-6\end{bmatrix}+\color{red}{\frac{2}{3}}\begin{bmatrix}8\\6\\-6\end{bmatrix},$$ and $$v_2=\color{red}{\frac{3}{5}}\begin{bmatrix}6\\0\\8\end{bmatrix}$$