I have this code:
rayX = camPlaneOriginX + fx*camRightX - fy*camUpX;
rayY = camPlaneOriginY + fx*camRightY - fy*camUpY;
rayZ = camPlaneOriginZ + fx*camRightZ - fy*camUpZ;
rayNorm = 1/sqrt(rayX*rayX + rayY*rayY + rayZ*rayZ);
theta = acos(rayY*rayNorm);
phi = atan2(rayZ,rayX) + PI;
from a function I found. I want to do the "opposite" (backwards) of what this function does, and I'm almost there. I'm having trouble solving for fx and fy. All the other variables are known by me and PI is the number π.
According to the internet, if phi = atan2(y,x) then sin(phi) = y and cos(phi) = x
If that's correct, then:
sin(phi-PI) = rayZ
cos(phi-PI) = rayX
and I also know that
cos(theta) = rayY * rayNorm
right? But then if a replace rayX, rayY, rayZ and rayNorm, I don't really know how to continue from there as it gets way too complicated for me.
How would I go about solving for fx and fy?
Thank you very much for your time and patience
There seems to be two problems here. First, you do not have a closed-form expression for either rayY or rayNorm. Instead you have two equations, where all other variables are known: $$\text{rayNorm}=\frac{1}{\sqrt{\text{rayX}^2+\text{rayY}^2+\text{rayZ}^2}}$$ and $$ \cos(\theta)=\text{rayY}\times\text{rayNorm}$$
We can solve this problem by squaring both sides of the first equation and cross-multiplying, which gives $$\text{rayX}^2+\text{rayY}^2+\text{rayZ}^2=\frac{1}{\text{rayNorm}^2}=\left(\frac{\text{rayY}}{\cos(\theta)}\right)^2$$ where in the end we substituted a rearrangement of the second equation. We now rearrange this equation to get $$(\text{rayY})^2(1-\frac{1}{\cos^2(\theta)})=-(\text{rayX}^2+\text{rayZ}^2)$$ This has two solutions $$\text{rayY}=\pm\sqrt{\frac{\text{rayX}^2+\text{rayZ}^2}{\tan^2(\theta)}}$$ where I simplified $$(1-\frac{1}{\cos^2(\theta)})=\frac{\cos^2(\theta)-1}{\cos^2(\theta)}=\frac{-\sin^2 \theta}{\cos^2\theta}=-\tan^2(\theta)$$
Now, you have two possible values for rayY; perhaps you know its sign, or perhaps you have two possible answers. The second problem is you now have three linear equations in two unknowns, namely the first three lines of your problem statement. This is called an overdetermined system, and now you have a choice of how to proceed. You could just solve it and hope there is a unique solution, or you could use least squares (as described on the wiki page) to get an approximate solution.