The 2-dimensional LCG is defined as below: $$ x_{i+2} = a\cdot x_{i} + b \cdot x_{x+1} + c \pmod{p} $$
where $a, b, c$ and the module $p$ are unknown.
We are given 10 consecutive outputs. The problem is to recover the modulus $p$ and $a,b,c$.
The solution to the 1-dimensional LCG is first construct $z_i \equiv 0 \pmod{p}$, and then calculate the gcd of $z_i$ to recover the modulus $p$.
However, in this 2-dimensional LCG case, it is hard to find something that equivlant to 0 (mod p).
We can first let $t_i = x_{i+1} - x_{i}$, where $t_{i+2} = a \cdot t_i + b \cdot t_{i+1}$.
Let $P_i = \begin{bmatrix} t_{i+2} & t_{i+1}\\ t_{i+1} & t_{i} \end{bmatrix}$, $A = \begin{bmatrix} b & a\\ 1 & 0 \end{bmatrix}$, then we have $P_{i+1} = A\cdot P_{i}$.
However, $P_{i+2} \cdot P_{i} - P_{i+1}^2 = A^2 \cdot P_i - A \cdot P_i \cdot A \cdot P_i \neq 0$ (mod p).
And I stuck here.
Is there any other methods that can solve this problem?