I have three original points $pt_1, pt_2, pt_3$ which if transformed by an unknown matrix $M$ turn into points $gd_1, gd_2, gd_3$ respectively. How can I find the matrix $M$ (all points are in 3-dimensional space)?
I understand that for original points holds $M\cdot pt_i = gd_i$, so combining all $pt_i$ into matrix $PT$ and all $gd_i$ into $GD$ I'd get a matrix equation $M\cdot PT=GD$ with unknown $M$.
However, many math packages solve matrix equations in form of $A\cdot x=B$, where $x$ is unknown.
Is my idea of combining points into matrices correct and if so how can I solve my matrix equation?
Form matrices whose rows are $pt_k$ and $gd_k$. Then $$ \begin{bmatrix}pt_1\\pt_2\\pt_3\end{bmatrix}M=\begin{bmatrix}gd_1\\gd_2\\gd_3\end{bmatrix} $$ Then we have $$ M=\begin{bmatrix}pt_1\\pt_2\\pt_3\end{bmatrix}^{-1}\begin{bmatrix}gd_1\\gd_2\\gd_3\end{bmatrix} $$ So your idea is correct.