I am given two sets of 3D points (actually 2D, see below) with corresponding pairs. I am seeking two 3D rotation matrices, such that (only) the X and Y components of the rotated points match best (they won't match exactly). Additionally, I am given the remaining Z distance for each pair of points.
All given points in both sets lie in a plane (w.l.o.g in the XY plane)
So for each pair of points $a_1$ and $b_1$:
(red variables are given)
I could also reformulate this as having only one rotation matrix but instead an unknown direction shift vector with known length:
(where length of the $v$ vector is 1)