I have a system of equation on the $n \times n$ variable $X$ of the form:
$$A_1XB_1 = A_2XB_2$$ where $A_1, A_2 \in \mathbb{R}^{m \times n}$, with $m>n$ and $B_1, B_2 \in \mathbb{R}^{n \times p}$ with $n > p$.We can assume that $A_i, B_i$ are full column-rank.
I would like to solve the system of equation with the constraint on $X$ to be different from the trivial solution $X=0$, as well as det$(X)\neq 0$. Other constraints on $X$ might also be possible. How to proceed?
By defining $x= \operatorname{vec}(X)$ we can rewrite the system of equation as
$$\underbrace{(B_1^\top \otimes A_1 - B_2^\top \otimes A_2)}_{Q}x=0.$$
In order for the system to be solvable, matrix $Q$ needs to have a nullspace different from the $0$ vector alone. For instance, by adding the constraint $\|x\|=1$ I can solve (or at least approximate) the system by taking $x$ as the right singular vector of matrix $Q$ associated to the smallest singular value. But what if I would like to enforce the $X$ to have a determinant different from zero?
Similar to your previous question, we cannot achieve $\det X\neq 0$ in general.
For example, take $m=4$, $n=2$, $p=1$. $$ A_1= \begin{bmatrix} 1&0\\0&1\\0&0\\0&0\end{bmatrix}, B_1=\begin{bmatrix}1\\0\end{bmatrix} $$ and $$ A_2= \begin{bmatrix} 2&0\\0&2\\0&0\\0&0\end{bmatrix}, B_2=\begin{bmatrix}1\\0\end{bmatrix} $$ Then $$X=\begin{bmatrix} x_1&x_3\\ x_2&x_4\end{bmatrix}$$ in the equation $$ A_1XB_1=A_2XB_2 $$ satisfies $$ x_1=x_2=0. $$
This is because $$ A_1XB_1=A_1\begin{bmatrix}x_1\\ x_2\end{bmatrix} = \begin{bmatrix} x_1\\ x_2 \\ 0 \\ 0\end{bmatrix}, \ \ A_2XB_2=A_2\begin{bmatrix}x_1\\ x_2\end{bmatrix} = \begin{bmatrix} 2x_1\\ 2x_2 \\ 0 \\ 0\end{bmatrix} $$ Thus, any solution to the above equation satisfies $\det X=0$.