What are the conditions on $a$ and $b$ so that the roots (real or complex) of the equation have magnitude $< 1$.
$$λ^2 − (a − b + 1)λ + a = 0$$
On a separate note, if you could explain (NOT Required for answer though):
Standard results from dynamic system theory say that the time behavior of the particle depends on the eigenvalues of the dynamic matrix $A$. The eigenvalues $λ_1$ and $λ_2$ (either real or complex) are the solutions of the equation: $$λ_2 − (a − b + 1)λ + a = 0$$ The necessary and sufficient condition for the equilibrium point given by Eq. (17) to be stable (an attractor) is that both eigenvalues of the matrix $A$ (whether real or complex) have magnitude less than 1.
If $\lambda_1$ and $\lambda_2$ are the roots of this equation:$$\lambda^2-(a-b+1)\lambda+a=0$$then we have:$$\lambda_1+\lambda_2=a-b+1\tag{1}$$$$\lambda_1\lambda_2=a\tag{2}$$We are also told that:$$|\lambda_1|\lt1$$$$|\lambda_2|\lt1$$$$\therefore |\lambda_1+\lambda_2|\lt2$$These condition applied to (2) yield:$$|a|\lt1$$and applied to (1) yield:$$|a-b+1|\lt2$$