$\textbf{Question}$: Find all vectors in $\mathbb{R}^3$ that are orthogonal to both $(1,2,3)$ and $(4,5,6)$.
$\textbf{My Attempt}$: We can write two equations: $$x+2y+3z=0\\4x+5y+6z=0$$ And now we can combine and simplify:$$x+2y+3z=4x+5y+6z\\3x+3y+3z=0\\x+y+z=0$$ Which means that all vectors that lie on the plane $x+y+z=0$ are orthogonal to both of the original vectors, or any vector of the form $(-y-z,y,z)$.
Is this solution correct? I feel like I might be going wrong when I say that every vector in the $x+y+z=0$ plane is orthogonal. Can anyone think of a different method of approaching this problem?
You started out on the right track by setting up a system of two linear equations, but then you took a bit of a left turn and didn’t actually solve the system. Each equation defines a plane orthogonal to the corresponding vector, and as the two vectors aren’t colinear, the intersection of these planes should be a line, not a plane as in your solution. This is consistent with a simple dimension check: the span of the vectors is two-dimensional, so its orthogonal complement in $\mathbb R^3$ will be one-dimensional. Specifically, your mistake was concluding that all of the solutions of $x+y+z=0$ are also solutions of the original two equations.
Since we’re working in $\mathbb R^3$, a simple way to solve this problem is to take the cross product of the two vectors. Every scalar multiple of this product is orthogonal to both vectors, and these are the only vectors orthogonal to them both.
Working instead with the equations that you set up, we have $$\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix},$$ i.e., we seek the kernel (null space) of the coefficient matrix. Row-reducing this matrix produces $$\begin{bmatrix}1&0&-1\\0&1&2\end{bmatrix},$$ from which we can read directly that the kernel is spanned by $\begin{bmatrix}1&-2&1\end{bmatrix}^T$.