Let $A,B\in M_{n}(\mathbb{R})$ be such that $B^2=I_n$ and $A^2=AB+I_n$. Prove that $$\det(A)\leq\left(\frac{1+\sqrt{5}}{2}\right)^n$$
I have been able to show that $AB=BA$, $B=A-A^{-1}$ and $A^4-3A^2+I=0$. Now from this how I can approach the problem.
If $A^4-3A^2+I=0$ than any eigenvalue of $A$ should satisfy $\lambda^4-3\lambda^2+1=0$. This is a biquadratic equation, solving which we get 4 roots: $\pm\frac{1+\sqrt{5}}{2}, \pm\frac{1-\sqrt{5}}{2}$. $det(A)$ is a product of its eigenvalues, so it will be biigest when all of them will be equal to $\frac{1+\sqrt{5}}{2}$.