I'm not sure if I should ask this question here or on stackoverflow, so forgive me if I'm wrong.
I want to apply a texture (taken from a camera) on a 3D surface, let me explain my problem:
I have acquire a 3D surface through a Kinect camera.
At some intervals I have saved the camera position (A rotation|translation matrix) as well as a 2D image of what the camera is looking at.
Now, what I'm trying to do is to find which 3D points (X,Y,Z) correspond to the 2D point (u,v) in my image, if I know that, I can apply the color of my 2D point on the corresponding 3D point.
I guess I have to start from this formula to solve this issue ? (taken from here)
The first matrix represents the intrinsic parameters of my camera while the second matrix represents my Rotation|Translation matrix. Both matrices are known.
Here is a picture representing what I'm trying to achieve (sorry for my poor drawing skills)
As you can see, the 3D surface is represented with triangles, each point of the triangle is stored in a .obj file, so I can also have access to them. From the formula, I just have to take the 3D point interesting me, apply the formula and find the corresponding 2D point.
However, I don't have any ideas of how to find the 3D points that are in my camera range. If I take a point that is not in my camera range, the corresponding 2D point would not make any sense and so the texture mapping will be wrong.
Any idea of how I can solve this issue ?
Thanks !
All you need to do is to compute the $(u, v)$ coordinates corresponding to the ray from the $xyz$-point to the "eye" (i.e., apply the formula you've written above). If $u$ and $v$ are within the range of valid values (typically either $0 \le u,v \le 1$ or $-1 \le u,v \le 1$, or maybe $0 \le u < 640$ and $0 \le v < 512$, for a $640 \times 512$ image, it all depends on what your software uses as its "device coordinates"), then you apply the texture map; if not, you say "Hey, this point doesn't have an associated texture color."
Nicely written question, by the way -- no need to apologize for your drawing!