I have a system of equations (inspired by the DLT algorithm, see also section 2.1 here ) for which I want to precondition the inputs so that I get a well-conditioned data matrix. I know how to do this preconditioning for the basic DLT algorithm as follows:
$$\lambda Pn_i = H Pm_i$$
$Pm_i$ and $Pn_i$ are 3x1 homogeneous vectors, formed from pairs of corresponding 2D inhomogeneous points $\tilde Pm_i$ and $\tilde Pn_i$ (these are my input data). H is a 3x3 homogeneous matrix (this is the unknown I want to estimate from the data). $\lambda$ is a scalar. For each pair of points I get two constraints, so I need at least 4 point pairs to determine H up to a scale. I can find H by forming the homogeneous system $Ah=0$, with $h$ being $H$ vectorized into a column vector. The solution $h$ is the singular vector with smallest singular value of $A$. This is the basic DLT algorithm. The preconditioning in this case involves transforming the sets of 2D inhomogeneous points $\tilde Pm_i$ and $\tilde Pn_i$ so that each set is at a $\sqrt2$ average distance from the origin. I do that by applying a translation and an isotropic scaling to the point sets $Pm_i$ and $Pn_i$ so that I get the transformed $Pm'_i$ and $Pn'_i$ points. For this I can use the 3x3 similarity matrices $Tn$ and $Tm$ so that: $$Pn'_i = Tn Pn_i$$ $$Pm'_i = Tm Pm_i$$ then I solve the new system $\lambda Pn'_i = H' Pm'_i$ using DLT as before (notice that now the system will be better conditioned) and the solution to the original system will be $$H = Tn^{-1} H' Tm$$
However, the system in my application is a bit different, and I don't know how to precondition it. The system comes from wanting to solve this problem: $$\underset{H}\arg min \sum_{i} P_i(Pn_i-HPm_i)$$ where $P_i$ is a 3x3 matrix, and there is one of them for each pair of points $Pm_i$, $Pn_i$. $P_i$ has a structure like $\begin{pmatrix} a & b & 0 \\c & d & 0\\0 & 0 & 1\end{pmatrix}$ with the magnitudes of the vectors (a,b) and (c,d) going from 0 to 1. If I write it in the same way as the DLT algorithm above:
$$\lambda P_i Pn_i = P_iH Pm_i$$ I can solve this system forming a homogeneous system $Ah=0$ and proceeding as before. I have implemented this and it seems to work. However, the condition number of $A$ can be huge, which probably makes the solution very sensitive to noise in the data. I would like to improve the conditioning of this system by using some procedure as the one described above.
I have tried, naively, to apply the same similarity transformations $Tn$ and $Tm$ to the point pairs, solve, and then get the solution to the original problem with $H = Tn^{-1} H' Tm$. That didn't work well as it ignores the effect of the $P_i$ matrices. I have tried to manipulate my system of the equations to have something more similar to the original DLT but I can't find anything that seems useful.
How could I apply a preconditioning to my system?