this is a long description but I hope the solution is simple:
I have a 3D pointCloud with a box (displayed at 0,0,0) and I want to drag the box on the XY plane with my mouse cursor (Zero movement on the Z axis) I can access the box transformation matrix and manipulate it and I have all the camera parameters.
My first approach was to keep the element (3,4) of the box's homogeneous transformation matrix fixed and let the other elements of the position vector change freely, it works well in normal conditions (image 1), but when I changed the camera position (image 2) I was no longer able to drag the box "towards" me. So I tried to use a "picker" to get the 3D position of the mouse cursor and move the box to that x,y position,however, the problem is when the camera is close to the level of the box or when it's rotated (same image 2 and image 3), the x,y,z values of the cursor in the images shows why I was unable to move the box using the picker.
The axis directions, the camera position and orientation,different cursor positions(white boxes),and the 3D positions of the cursor are all shown in each image(The values are not entierly exact).
I did some experiments and the best results I could get are from these 2 equations (I got them by trial and error) but the results need further improvements :
// **position[0],[1],[2]** is the (x, y, z) position of the mouse cursor respectively.
// **camOrientation[0],[1],[2]** is the orientation of the camera around (x, y, z)respectively.
camOrientation[1] = 90 - camOrientation[1];// the orientation around the
Y axis(The subtraction gave me best results).
// Calculating the new X, Y position of the box.
// The new X position =The X position of the cursor +[cos(orientation around X axis)*the Z position of the cursor] + [cos(orientation around Y axis)*the Z position of the cursor]
// The same applies for the new Y value but instead of **cos** I used **sin** and the first argumetn is the Y position of the cursor.
double XValue = position[0] + cos(camOrientation[0] * 3.14159 / 180) * position[2] + cos(camOrientation[1] * 3.14159 / 180) * position[2];
double YValue = position[1] + sin(camOrientation[0] * 3.14159 / 180) * position[2] + sin(camOrientation[1] * 3.14159 / 180) * position[2];
// Applying the values to the box matrix.(translate the box to the new x,y position)
BoxMatrix->SetElement(0, 3, XValue);
BoxMatrix->SetElement(1, 3, YValue);
My question, is there a mathematical approach or equation to get the expected results right, because it seems a standard problem but I couldn't find a solution.(I posted my question on the computerGraphics stack exchange but didn't get any answers, I hope I'm in the right forum and that I'm not off-topic)
