Place the camera to match a given camera - object transformation

Question

I know the pose (in camera frame) of an object related to a camera in real world (results are coming from a 6d-pose estimator). In the simulation I want to replicate the same view and same position (in camera frame) of a simulated object related to a simulated camera (for the sake of simplicity I want to keep the object static and rotate the camera - so the object is placed at the origin of the world frame). In other world where should be the camera coordinates in the simulator's world frame to match the same setting in real world.

Answer 1

Well, say in the real world, your object has coordinate $x(t)$, and your camera has coordinate $y(t)$ (where $x$ and $y$ are functions $\mathbb{R} \to \mathbb{R}^n$ which are parametrized by time, and $n$ is the dimension of the space you work in).

If you want your new frame of reference to be the one where $x(t)$ is always equal to $0$ (ie, your object is at the center), then your camera must have coordinate $y(t) - x(t)$. (Keep in mind that for all $t$, $x(t)$ and $y(t)$ are vectors, so this is a difference of vectors).