Imagine we have a projection matrix $T$ that has a field of view of $90$ degrees. Imagine we also have a plane of pixels $P$ that contains an arbitrary image that was rendered to using projection $T$. Is there a way/equation to find the rotation needed to turn the camera so that the camera will be facing a specified pixel (for example pixel $(u, v)$ where pixel $(0, 0)$ is the current pixel being faced).
I imagine that this could be worked out first with 1 dimension then apply the same formula to both $u$ and $v$. I attempted this in a program by just changing the rotation by the distance to a pixel. It works for pixels close to the centre but starts to fall short further away. The edges are more spread out than the middle so would this be something like $c * \arctan(d)$ where $c$ is an arbitrary constant and $d$ is the distance in pixels?