Dear Math Stackexchange community, how can I compute $R$ rotation matrix in this equation ($P=Rp+t$)?
$p$ consists of 3 points of $p_1,p_2,p_3$. And $p_1,p_2,p_3$ each has 3 coordinates of $x,y,z$. Therefore p is 3x3 matrix.
$p=\begin{bmatrix} p_{1x} & p_{2x} & p_{3x} \\ p_{1y} & p_{2y} & p_{3y} \\ p_{1z} & p_{2z} & p_{3z} \end{bmatrix}$
t is 3x1 vector. $t=\begin{bmatrix} t_x \\ t_y \\ t_z \end{bmatrix}$
P is a 3x3 matrix. $P=\begin{bmatrix} P_{1x} & P_{2x} & P_{3x} \\ P_{1y} & P_{2y} & P_{3y} \\ P_{1z} & P_{2z} & P_{3z} \end{bmatrix}$
R is a 3x3 matrix. $R=\begin{bmatrix} r_{11}&r_{12}&r_{13} \\ r_{21}&r_{22}&r_{23} \\ r_{31}&r_{32}&r_{33} \end{bmatrix}$
Therefore, all in all: $P=Rp+t$
$\begin{bmatrix} P_{1x} & P_{2x} & P_{3x} \\ P_{1y} & P_{2y} & P_{3y} \\ P_{1z} & P_{2z} & P_{3z} \end{bmatrix} = \begin{bmatrix} r_{11}&r_{12}&r_{13} \\ r_{21}&r_{22}&r_{23} \\ r_{31}&r_{32}&r_{33} \end{bmatrix}.\begin{bmatrix} p_{1x} & p_{2x} & p_{3x} \\ p_{1y} & p_{2y} & p_{3y} \\ p_{1z} & p_{2z} & p_{3z} \end{bmatrix}+\begin{bmatrix} t_x \\ t_y \\ t_z \end{bmatrix}$
I tried to take the inverse and calculate R. However; when I put numeric values in it, that method doesn't work with numeric values.
Do you have any suggestion?
edit1: I did the following after DavidK, and Cryo's suggestion:
$P=Rp+t$
$(P-t)=R.p$
$(P-t).p^{-1}=R$
However there is a problem now: last raw of $p$ is always fully zero for my case. And I can't take the inverse of $p$.
Thanks to @David K. I was suspecting that Matlab was doing something convenient, but I don't use it any more, so wanted to avoid guessing.
So you have
$\mathbf{Q}=\left[\mathbf{q}_1\,\mathbf{q_2}\,\mathbf{q_3}\right]=\mathbf{P}-\mathbf{t}=\mathbf{R}.\boldsymbol{\mathcal{P}}=\mathbf{R}.\left[\mathbf{p}_1\,\mathbf{p}_2\,\mathbf{p}_3\right]$
Where $\mathbf{Q}$ is some $3\times 3$ real-valued matrix, which we can also view as 3 column vectors $\mathbf{q}_{1,2,3}$. $\mathbf{R}$ is a real-valued rotation matrix, which means it is orthogonal ($\mathbf{R}^T\mathbf{R}=\mathbf{Id}=identity$) and has determinant of $det\left[\mathbf{R}\right]=+1$. Finally, matrix $\boldsymbol{\mathcal{P}}$ is made up from three 3d real-valued column vectors $\mathbf{p}_{1,2,3}$. The aim is to determine the rotation matrix by choosing the vectors $\mathbf{p}_{1,2,3}$.
If you can pick three linearly independent vectors $\mathbf{p}_{1,2,3}$ you can have a unique solution for $\mathbf{R}$ given $\mathbf{Q}$. You seem to say that all z-components all your three vectors are zero. Then the three vectors are not linearly independent.
Why can you not have three linearly independent vectors? Are there any limitations?
PS: This is not so much an answer as restatement of the problem, but I am still struggling to understand the question