My goal is to express the transformation between the black frame F1 and the other one F0 (Green, Red, Violet):
All what I know is the position X,Y Z of four points A, B, C, D wrt F0. My question is how can I know the transformation (3d rotation+ 3d translation) between the two frames?


Translation + rotation can be expressed like this
$$M=\left[\begin{array}{cc}T&0\\0&1\end{array}\right]\left[\begin{array}{cc}R&0\\0&1\end{array}\right]\left[\begin{array}{cc}T^{-1}&x\\0&1\end{array}\right]$$
Where $T$ first is transform that takes us to coordinate system where we rotate. and $R$ is rotation matrix, for example
$$R=\left[\begin{array}{ccc}\cos(\alpha)&\sin(\alpha)&0\\-\sin(\alpha)&\cos(\alpha)&0\\0&0&1\end{array}\right]$$ $$T=\left[\begin{array}{ccc}v_1&v_2&v_P\end{array}\right]$$ Where $v_P$ is vector along axis or rotation and $\{v_1,v_2\}$ is a basis for plane of rotation.
So what you have is some equations $Mw_1 = w_2$ where vectors in $w_2$ are coordinates for frame 2 and $w_1$ are same points in first frame. $w$ should be filled like this $w=[c_x,c_y,c_z,1]^T$ coordinate $x,y,z$ and don't forget the 1 at the end of vector.
The sin:s and cos:s make this slightly nonlinear, but there still exist quite okay methods to solve it. I think a few iterations of refining linear solver with some kind of correction should work.