This is an illustration of the scenario I have:
I have the plane equation of a plane P $(A_px + B_py + C_pz + D_p = 0)$ and the coordinates of points A $(X_a, Y_a, Z_a)$, B $(X_b, Y_b, Z_b)$ and C $(X_c, Y_c, Z_c)$. The plane P will intersect the triangle ABC which will result in creating intersection points D and E. How can I calculate the coordinates for these two new points with the values I have?
I like using homogeneous coordinates and the Plücker matrix representation of a line for this since that lets you crank out the answer directly. The Plücker matrix of the line through two points with homogeneous coordinate vectors $\mathbf p$ and $\mathbf q$ is $\mathcal L = \mathbf p \mathbf q^T-\mathbf q \mathbf p^T$. If the homogeneous vector that represents the plane is $\mathbf\pi$, then the intersection of the line and plane is simply $$\mathcal L \mathbf\pi = (\mathbf p \mathbf q^T-\mathbf q \mathbf p^T)\mathbf\pi = (\mathbf q^T \mathbf\pi) \mathbf p - (\mathbf p^T \mathbf\pi) \mathbf q.$$ In other words, multiply each point by the dot product of the plane and the other point, and subtract. To convert back to Cartesian coordinates, you’ll of course have to divide through by the last coordinate of the result. (If the last coordinate is $0$, there’s no intersection.) For this problem, we have in homogeneous coordinates the plane $\mathbf P=[A_p:B_p:C_p:D_p]$, the point $\mathbf A=[X_a:Y_a:Z_a:1]$ and so forth.
This gives you the intersection of the line through a pair of points and the plane, but this intersection point might not lie between the endpoints of the segment. A simple way to test for this is to compare the signs of the dot products of the homogeneous coordinates of the two end points with the plane: if they’re different, then the points lie on opposite sides of the plane, so the line segment will intersect it. If either dot product is zero, then that point lies on the plane. You need these two dot products to compute the intersection point, anyway, so this doesn’t involve any extra work, but I recommend making this test before spending unnecessary time on the rest of the intersection computation.