I have two vectors in 3D Cartesian coordinate space such that each vector is described by components .
I would like to find the angle between these two vectors in the xy coordinate plane, ie. viewed from an observer located on the z axis.
Can I simply set the z component of each vector to zero and calculate the angle between them using the definition of the dot product? If not, what do I do to project the 3D vectors onto the 2D xy plane?
On
Yes, you can.
Let's look at a more general case, where the plane is perpendicular to some unit vector $\hat{n}$, and your vectors are $\vec{p}$ and $\vec{q}$.
Projecting the vectors to the plane yields $$\vec{p}' = \vec{p} - \hat{n}\left(\hat{n} \cdot \vec{p}\right) \tag{1}\label{1}$$ and $$\vec{q}' = \vec{q} - \hat{n}\left(\hat{n} \cdot \vec{q}\right) \tag{2}\label{2}$$
If we consider $\hat{n} = ( 0, 0, 1 )$ and $\vec{p} = ( x_p , y_p , z_p )$, we find that $\vec{p}' = ( x_p , y_p , 0 )$.
So, yes, projecting to the $xy$ plane is the same thing as zeroing out the $z$ component.
(As a reminder to those who have forgotten, the angle $\theta$ between any two three-dimensional vectors fulfills $$\cos(\theta) = \frac{\vec{p}' \cdot \vec{q}'}{\left\lVert\vec{p}'\right\rVert \left\lVert\vec{q}'\right\rVert} \qquad \text{and} \qquad \sin(\theta) = \frac{\left\lVert\vec{p}' \times \vec{q}'\right\rVert}{\left\lVert\vec{p}'\right\rVert \left\lVert\vec{q}'\right\rVert}$$ which are quite often useful.)
To project $3$D vectors onto $2$D $xy$-plane (also known described by the hyperplane $z=0$), simply set the third coordinate to zero.
Hence the two process that you described are the same.
To show that it is valid to set the third coordinate to zero, we are minimizing $$(\hat{x} - x)^2 + (\hat{y} - y)^2 +(0-z)^2$$
where the variables are $\hat{x}$ and $\hat{y}$. The minimal value is attained at $(x,y,0)$.