Given a 3D directional vector, and a 3D point, is it possible to calculate a 'rotation around the vector' for other points?

Question

Sorry if the title if confusing.

Essentially I have a vertex and a vector (the normal of a plane which the vertex sits on), and would like to be able to calculate the 'angle' along the plane of any other point in 3d space (which may not sit on the plane). I'd have two other points on the plane which could serve as the zero degrees point if need be, but the point being ultimately compared would almost certainly not be on the plane.

As an example: https://i.stack.imgur.com/4JhWe.png

My context is that I have a 3-cornered polygon (a rendering triangle face), and am looking for a cheap way to see whether another given 3d point lies within the 'infinite normal extrusion' of the bounded section of the plane. It also helps to know which edge the point is closest to if it falls outside of the polygon's infinite normal extrusion, hence why calculating an angle seems best.

As an example: https://i.stack.imgur.com/2v7hP.png

Answer 1

This doesn't directly answer your question, but it might solve your problem.

As you say, you are trying to find out whether a given point $P$ lies in the extruded volume formed by sweeping a triangle $ABC$. The first thing I would do is find the equations of the three planes defining this extruded shape. Then, for each of the three planes, check on which side the point $P$ lies. The point $P$ will be inside the volume if it's on the "inside" side of each of the three planes.

Specifically, consider the plane defined by the triangle side $AB$. Its equation is $(X - A) \cdot N = 0$, where $N$ is a vector lying in the plane of $ABC$ that is normal to the side $AB$. The dot denotes a vector dot product (sometimes called a scalar product). You determine inside vs. outside simply by checking whether $(X - A) \cdot N < 0$ or $(X - A) \cdot N > 0$.

Alternatively, project the point $P$ onto the plane of $ABC$, and use well-known tests to determine whether the projected point lies inside the triangle.

I'm not convinced that computing angles is a good approach. Using trig functions will certainly have a nasty impact on performance (if that's a concern).

Answer 2

From what I understand, the situation seems to be thus:

Let $p$ be a point (specified by a vector, of course) in $3$-dimensional Euclidean space, let $d$ be a direction vector, and let $\Pi$ be the plane through $p$ with normal vector $d$. Let $v$ be any arbitrary point in $3$-dimensional space. We want to know if we can have a well-defined notion of the angle $v$ makes relative to $\Pi$.

The answer is yes, but you have to provide and fix a base point in $\Pi$ to represent the vertex point for the angles we consider. Let $a$ denote this base point. Then to every point $x$ in $3$-dimensional space (not necessarily on $\Pi$), we can associate the direction vector $$u_x = \frac{x-u}{\Vert x-u\Vert}.$$

Imagine placing the tail end of $u_x$ at $a$. Now we can speak of the angle between $u_x$ and $\Pi$. This is done by taking the orthogonal projection of $u_x$ into $\Pi$, call it $Pu_x$. Then the angle between $\Pi$ and $u_x$ can be defined by the formula $$u_x\cdot Pu_x = \Vert u_x\Vert\Vert Pu_x\Vert \cos\theta = \Vert Pu_x\Vert\cos\theta.$$ The angle you seek is this $\theta$. ($\Vert u_x\Vert$ was dropped because $u_x$ was chosen to be a unit vector.)

The last concern is how to get the orthogonal projection. The surefire way is to get an orthonormal basis for $\Pi$ (now regarded as a vector subspace of $3$-d space) and apply the Gram-Schmidt process to $u_x$, getting the orthogonal component $v_x$. Then the orthogonal projection is $u_x-v_x$.