Given two full-rank $3\times3$ integer matrices $M_1$ and $M_2$, I am trying to find integer matrices $N_1$ and $N_2$ such that $M_1N_1$=$M_2N_2$, such that $\left|\det(M_1N_1)\right|$ is minimal.
edit: The $N_i$ are only required to within a similarity transformation (i.e., $\tilde{N}_i\sim U_iN_iV_i$, where $U_i$ and $V_i$ are unimodular integer matrices).
This seems analogous to finding the "least common multiple" of two integer matrices, but I'm having trouble identifying an algorithm or matrix decomposition that helps me.
I'm aware of the relationship of this problem to ring theory, in that the columns of the "LCM" matrix that I'm looking for generate the ideals formed by the intersection of the ideals generated by $M_1$ and $M_2$. I'm not very familiar with ring theory, but I tried rewriting the problem in terms of the Smith normal forms of $M_1$ and $M_2$, which didn't get me much closer to a solution.
Just for context, my ultimate goal is to find the (crystallographic) lattice formed by the coincident points of two other lattices (described by the lattice vectors $L_1=LM_1$ and $L_2=LM_2$).
This is only a partial answer. (I assume you want $N_1$ and $N_2$ invertible, otherwise $N_1=0$ solves)
Given $M$ its matrix of cofactors $\mathrm{Cof}(M)$ (see for instance here) is such that $\mathrm{Cof}(M)=M^{-1}\det M$. As $\mathrm{Cof}(M)$ is made by minors of $M$ if $M$ is integer then $\mathrm{Cof}(M)$ is.
If $M_2N_2=M_1N_1$ we have $$N_2=\frac{\mathrm{Cof}(M_2)M_1}{\det M_2}N_1$$
This is computable. Reduce so that $\frac{\mathrm{Cof}(M_2)M_1}{\det M_2}=A/n$ where $A$ is integer. Then we have $$\det N_2=\frac{\det A}{n^3}\det(N_1)$$ If $\frac{\det A}{n^3}$ is reduced, then you must have $\det N_1\geq n^3$ and $nId$ solves.
In general this reduces the problem to solve $$\frac{A}{n}N_1$$ integer with $|\det N_1|$ minimal.