I have a point $(9,5,0)$ and a triangle with points $(1,1,0), (3,3,1), (6,1,0)$, let's label them as $A,B,C$ respectively. In order to get the normal vector, I do the cross product of two vectors. If I do $AB \times AC$ I get $n = \left\langle 0,-5,10\right\rangle$. With this I determine the equation of the plane is $5y - 10z - 5 = 0$. If I project my point onto the plane with this normal vector in mind, the point on the plane I find is $(9, 21/5, 8/5)$. HOWEVER, if I initially choose to cross two different vectors, I get something different.
Example: If I instead choose to do $BA \times BC$ I get $n = \left\langle 0,5,-10\right\rangle$. Now this in a way makes sense to me since it is the inverse of the other normal I found. Basically the other half of the line through the plane, right? If I continue, I find the equation of the plane to be $-5y + 10z + 5 = 0$. This leads me to a projected point of $(9, -29/5, -8/5)$. Why do I get two different points based on which normal vector I choose to solve with? I know the correct answer to this problem, one is right and one is wrong. I am trying to program a general solution to this type of problem. How do I choose the correct normal vector?
Please let me know if I am doing something wrong.
$\vec{AB} = <2,2,1> \quad \quad \vec{AC} = <5,0,0>$.
So, $\vec{AB} \times \vec{AC} = <0,5,-10>$
Then, the plane equation is something like $5y -10z = C$ for some constant $C$.
Plug (for example) the point $(1,1,0)$ into the plane equation to get $C=5$.
Hence, the plane equation is $5y -10z -5 =0$ or equivalently $-5y +10z+5=0$ which is slightly different from the one you found.
Edit: Let's continue with a different approach. The vector $\vec{n} = <0,5,-10>$ is perpendicular to the plane. The line that passes through our point $(9,5,0)$ which is parallel to $\vec n$ is $$x=9, \quad y=5+5t, \quad z=0-10t, \quad t \in \mathbb R$$
Now let's find where this line cuts the plane. Pick an arbitrary point from the line and put into the plane equation:
$$0= 5y-10z-5 =5(5+5t)-10(0-10t)-5$$
$$125t + 25 -5=0 \quad \implies \quad t=-\frac{4}{25}$$
So, the answer is $(9,5+5t,0-10t)=(9,-\frac{21}{5},\frac{8}{5})$.
Edit2: Now we do the projection. Choose an arbitrary point on the plane, say $A(1,1,0)$ and consider the vector $\vec{AP}$ where $P=P(9,5,0)$. $\vec{AP}=<8,4,0>$.
Then,
\begin{align*} proj_{\ \vec n \ } \vec{AP} &= |\vec{AP}| \cdot \cos \alpha \cdot \frac{\vec n}{| \vec n |} \\ &= |\vec{AP}| \cdot \frac{\vec{AP} \bullet \vec n}{|\vec{AP}| \cdot |\vec n|} \cdot \frac{\vec n}{| \vec n |} \\ &= \text{ calculations ... find the projection vector} \\ &= \vec u \end{align*}
Now, the point you are looking for is the point $K$ such that $\vec{KP} = \vec u$