I have a known 3d rotation matrix (orthogonal) for rotating an object, created from right, up, forward, and position vectors:
| $r_1$ $u_1$ $f_1$ $p_1$ |
| $r_2$ $u_2$ $f_2$ $p_2$ |
| $r_3$ $u_3$ $f_3$ $p_3$ |
I also have a new forward vector which I will call "$g$".
I was wondering if it were possible to use this information to create a rotated matrix where the new forward vector is used instead.
Ie: the expected result:
| ? ? g1 ? |
| ? ? g2 ? |
| ? ? g3 ? |
Where "?" is a calculated value based on the original rotation matrix.
I have looked through hundreds of tutorials and examples, but nothing really seems to match what I'm trying to do, so I'm wondering if it is actually possible to calculate given that these are the only values available.
I figured that was the case (I was missing other constraints).
I have found the solution now though, and thought I should post the solution in case someone else needs it. In this case both forward vectors (f & g), had their own world axis vector (w & k) which I hadn't mentioned in the original question, and the original matrix was NOT orthogonal in all cases.
1) I created a 3x3 rotation matrix for each coordinate system based off these values only.
2) I transformed the r, u, and p vectors using the inverse of matrix S.
3) I transformed the r1, u1, and p1 vectors using the matrix T.
This resulted in the new transformation matrix I was looking for: