Expressing the solution to a quadratic matrix equation

Question

Let $A$, $B$ and $M$ be $3 \times 3$ matrices with $A$ and $B$ symmetric. I would like an expression for an $N$ satisfying $$NMN + NA +BN + M^{T} = 0$$ It doesn't have to work for the entire set of $(A,B,M)$ but it needs to work for at least a subset of non-zero measure (eg. you can assume $A$ or $M$ is positive-definite if it helps, but you can't assume $M=0$ .The space in which the latter property is true is of zero measure). Any insight into computing an $N$ that satisfies the equation would help. It looks like a spruced up quadratic equation.

Answer 1

It's an algebraic Riccati equation. If we choose generic symmetric real matrices $A,B$ and a generic real matrix $M$ (for example, randomly choose them), then there are $20$ complex solutions in $X=[x_{i,j}]\in M_3$; among these, there are (in general) $0$ or $4$ or $8$ real solutions for $X$.

Of course, there is no explicit formula giving $X$; in particular, $x[1,1]$ is a root of a polynomial of degree $20$ that is not solvable (by radicals).

Conclusion: use standard softwares to solve your equation.

I use the library Grobner of Maple; time of calculation: 1"5.

EDIT. Answer to the OP. Using the pseudo-Hamiltonian $H=\begin{pmatrix}-A&-M\\M^T&B\end{pmatrix}$ is a good method to obtain all the solutions. You calculate approximations of the $6$ eigenvalues (in general distinct) of $H$ and you seek the real invariant vector spaces of dimension $3$ that are $H$-invariant (if you want only the real solutions). To do that, you take, as a basis, approximations of $3$ eigenvectors of $H$ (one real and $2$ conjugates or $3$ real). Then if $H$ has

$6$ real eigenvalues (that is rare) you obtain $\binom{6}{3}=20$ solutions.

$4$ real eigenvalues $\rightarrow 4+4=8$ solutions

$2$ real eigenvalues $\rightarrow 2+2=4$ solutions

$0$ real eigenvalue $\rightarrow 0$ solution.