Let $A,B\in M_{2}(\mathbb{R})$ so that $A^2 = B^2 = I$. Which are eigenvalues of $AB$?
1) $1\pm \sqrt 3$
2)$3 \pm 2\sqrt2$
3)$\dfrac {1}{2},2$
4)$2 \pm 2\sqrt 3$
Let $A,B\in M_{2}(\mathbb{R})$ so that $A^2 = B^2 = I$. Which are eigenvalues of $AB$?
1) $1\pm \sqrt 3$
2)$3 \pm 2\sqrt2$
3)$\dfrac {1}{2},2$
4)$2 \pm 2\sqrt 3$
On
Presumably, this question should read, "Which of the following could possibly be the set of eigenvalues of $AB$?"
Hint: What are the possible determinants of $A$? Of $B$? Of $AB$? Can we eliminate some possibilities using these answers?
On
As $A^2=B^2=I$, the determinants of $A$ and $B$ can only be $\pm$. Hence $\det(AB)$ is also equal to $\pm1$. As determinant is equal to product of eigenvalues, you can readily rule out cases (1) and (4).
Are cases (2) and (3) possible? In order that they are possible, obviously neither $A$ nor $B$ can be equal to $\pm I$. It follows that both $A$ and $B$ are similar to $\operatorname{diag}(1,-1)$. So, we take $A=\operatorname{diag}(1,-1)$ and try to construct an appropriate $B$. As Eu Yu points out, involutory matrices come in handy. For simplicity, let us try $$ B=\pmatrix{x&1\\ 1-x^2&-x}. $$ We always have $B^2=I$ and $\det(B)=-1$ regardless of $x$, and $$ AB=\pmatrix{x&1\\ x^2-1&x}. $$ The eigenvalues of a $2\times2$ matrix are uniquely determined by the matrix's trace and determinant. In both cases (2) and (3), we have $\det(AB)=1$, which is already satisfied by our choices of $A$ and $B$. It remains to fit the traces. In case (2), $\operatorname{trace}(AB)=(3+\sqrt{2})+(3-\sqrt{2})=6$, so we may set $x=3$. In case (3), $\operatorname{trace}(AB)=2+\frac12=\frac52$ and we can set $x=\frac54$.
It's easy to show that possibilities (1) and (4) are impossible. Note that the magnitude of the determinants for both $A$ and $B$ must be $1$, indeed $$1 = \det(A^2) = \det(B^2) \implies |\det(A)| = |\det(B)| = 1$$ This necessarily implies that $|\det(AB)|=1$, i.e. the product of the eigenvalues must have magnitude $1$. This condition is satisfied by options (2) and (3), where the product of the eigenvalues is just $1$, but not by options (1) and (4).
Now let me show that both possibilities (2) and (3) are possible. For (3), let $$A = \begin{pmatrix}\sqrt{2} & 1 \\ -1 & -\sqrt{2}\end{pmatrix},\ \ \ \ \ \text{and}\ \ \ \ \ \ B=\begin{pmatrix}\frac{1}{\sqrt{2}} & -1 \\ -\frac{1}{2} & -\frac{1}{\sqrt{2}}\end{pmatrix}$$ You can easily verify that $A$ and $B$ satisfy $A^2 = B^2 = I$. Their product is given by $$AB = \begin{pmatrix}\frac{1}{2} & -\sqrt{2}-\frac{1}{\sqrt{2}} \\ 0 & 2\end{pmatrix}$$ which has eigenvalues $1/2$ and $2$, exactly as required. Let me now elaborate on how I found this example, and the same procedure will allow you to construct an example for case (2).
There is a rather well known parametrization for matrices satisfying $A^2 = I$ in the $2\times 2$ case. You can check that any $2\times 2$ matrix in the form of $$A=\begin{pmatrix}x & y \\ \frac{1-x^2}{y} & -x\end{pmatrix}$$ for any $x,y\in\mathbb{R}$ will satisfy $A^2 = I$. This condition can be found in the wikipedia article for involutory matrices for example. To simplify the expression above, let's set $y=1$ in general. Then we try a product of the form $$AB = \begin{pmatrix}a & 1 \\ 1-a^2 & -a\end{pmatrix}\begin{pmatrix}b & 1 \\ 1-b^2 & -b\end{pmatrix} = \begin{pmatrix}ab+1-b^2 & a-b\\ab^2 + (1-a^2)b - a & 1-a^2+ab\end{pmatrix}$$ where $a$ and $b$ are free to vary. The idea is to choose $a$ and $b$ so that we have our desired eigenvalues. However it is still rather difficult to find the eigenvalues we need. So let's try making the product matrix triangular so that we can read the eigenvalues right off the diagonal.
Making the product lower triangular requires $a=b$ which just gives the trivial solution $A=B$ and $AB=I$.
If we try making the product upper triangular, we find the condition $b = -\frac{1}{a}$ by solving the quadratic in $b$ found in the bottom left entry. The product then reduces to $$AB = \begin{pmatrix}-\frac{1}{a^2} & a+\frac{1}{a} \\ 0 & -a^2\end{pmatrix}$$ This is almost what we need, except that the eigenvalues here are negative. To fix this, we simply take the negative of $B$ used in the expressions above, i.e. $$B=-\begin{pmatrix}b & 1 \\ 1-b^2 & -b\end{pmatrix}$$ We've lucked out here since the solution now allows us to produce any pair of positive eigenvalues $\lambda_1$ and $\lambda_2$ such that $\lambda_1\lambda_2 = 1$.
Taking $a = \sqrt{2}$ gives a solution for case (3) which I've included above. Taking $a = \sqrt{3 + 2\sqrt{2}}$ will then give case (2) which I leave you to work out yourself.