Prove that $\det(A^2+B^2+A+B+I_n) \ge 0$

Question

Let $A, B\in M_n(R)$ and $AB=O_n$ (where $O_n$ means the zero matrix). Prove that $$\det(A^2+B^2+A+B+I_n) \ge 0$$

I tried to write this as the product of two complex conjugates, and then the determinant would be $\ge 0$,but I can't manage to do this.

EDIT: the problem has two tasks, and the first was to prove $\det(A^2+A+I_n)\ge 0$, which I solved by writing $\det(A^2+A+I_n)=\det(A+\epsilon I_n) \cdot \det(A+\epsilon^2 I_n) \ge 0$,where $\epsilon$ is a complex third root of unity.

Answer 1

Here is a simple proof that I have just found.
We have proved that $\det(X^2+X+I_n) \ge 0$, $\forall X \in M_n(\mathbb R) $.
Also, $(A^2+A+I_n)(B^2+B+I_n)=A^2+B^2+A+B+I_n$ and by taking the determinant of this relation we are done.

Answer 2

Let $r$ be the rank of $B$. By a change of basis, we may assume without loss of generality that the image of $B$ (i.e. the column space of $B$) is precisely the linear span of $e_{n-r+1},e_{n-r+2},\ldots,e_n$, the last $r$ vectors in the standard basis of $\mathbb R^n$. Thus $$ A=\pmatrix{X_{(n-r)\times(n-r)}&0\\ Y_{r\times(n-r)}&0}, \quad B=\pmatrix{0&0\\ Z_{r\times(n-r)}&W_{r\times r}} $$ and $$ A^2+B^2+A+B+I=\pmatrix{X^2+X+I&0\\ \ast&W^2+W+I}. $$ Therefore, it suffices to prove that $\det(X^2+X+I)$ and $\det(W^2+W+I)$ are nonnegative. The rest should be easy.

Answer 3

More generally,

$\textbf{Proposition}.$ Let $P,Q\in\mathbb{R}[x]$ s.t. for every $x\in\mathbb{R}$, $P(x)+Q(0)\geq 0,P(0)+Q(x)\geq 0$. Let $A,B\in M_n(\mathbb{R})$ s.t. $AB=0$.

Then $\det(P(A)+Q(B))\geq 0$.

$\textbf{Proof}.$ Since $AB=0$, $A,B$ are simultaneously triangularizable over $\mathbb{C}$ and we may assume that $A,B$ are upper triangular.

There are orderings of $spectrum(A)=(a_i),spectrum(B)=(b_i)$ s.t. $a_ib_i=0$.

If $a_i\not= 0$, then we put $c_i=a_i$ and $R_i=P,S_i=Q$, otherwise $c_i=b_i$ and $R_i=Q,S_i=P$.

Let $I=\{i;im(c_i)>0\},J=\{j; c_j\in\mathbb{R}\}$.

Then $\det(P(A)+Q(B))=\Pi_{i\in I}|R_i(c_i)+S_i(0)|^2\Pi_{j\in J}(R_j(c_j)+S_j(0))\geq 0$. $\square$

Here $P(x)=Q(x)=x^2+x+1/2$ (for example).

EDIT. Answer to MathEnthusiast.

We can show the result, using the user1551's method.

We find $\det(P(A)+Q(B))=\det(P(X)+Q(0)I_{n-r})\det(P(0)I_r+Q(W))$. It remains to show that both determinants are $\geq 0$ (for example, the first one).

It suffices to see that the eigenvalues of $P(X)+Q(0)I_{n-r}$ are the $(P(x_i)+Q(0))_i$ where $spectrum(X)=(x_i)$ is composed of real numbers and pairwise conjugate numbers.