A twist on the Hand to Sensor Calibration problem for Riemann metrics

Question

I have two sub-manifolds $M_1$ and $M_2$ of rank $k_1$ and $k_2$. Now given $k_1 \neq k_2$ there is no unique torsion free compatible connection between points in $M_1$ and $M_2$. However, I would like to construct a connection such that for two pushforward mappings $f_1:TM_1 \to TM_2$ and $f_2:TM_2 \to TM_1$, that $f_2 \circ f_1(x) = x$ for any $x \in M_1$. Following from this hypothesis, and presuming that these mappings can be written in coordinate form as matrices of shape $U_{f_1} \in \mathbb{R}^{m\times n}$ and $V_{f_2} = \mathbb{R}^{n \times m}$ then we have a simultaneous set of equation $$ UVx = x \text{ and } VUy = y $$ for $x \in M_2$, $y \in M_1$. This naturally leads to $V = U^\dagger$ however this leaves $U$ unrestricted. The alternative approach that I'm considering is: given the Riemann metrics $g_1$ and $g_2$ at $x \in M_1$ and $y \in M_2$ respectively, solve for the following set of equation $$ UV = g_2 \text{ and } VU = g_1. $$ Now I can break it down into an equivalence of the Hand to Sensor Calibration problem, where we have $$ Vg_1 = g_2V $$ however I'm missing a piece of the puzzle this has infinite solutions. Perhaps I'm on the wrong track, I'm very much in uncharted territories. I'm investigating holonomy groups in point-cloud manifolds and I'm looking for a band aid for the uneven rank deficiency of the metric across the manifold.