I'm doing this exercise:
Consider the following differential equation system:
$$\begin{cases}\dot{x}=x(1-x-ay) \\\dot{y}=y(1-y-bx)\end{cases}$$
With $a,b>0$ and different to $1$. Sketch the phase portrait on the first quadrant and prove it has no periodic orbits.
What I've done so far is to find the equilibrium points. We make
$$\begin{cases}0=x(1-x-ay) \\0=y(1-y-bx)\end{cases}$$
and we find equilibrium points at $(0,0), (1,0), (0,1)$ and at the intersection point between $x=1-ay\ $ and $y=1-bx$, wich is
$$x=\frac{1-a}{1-ab},\ y=\frac{1-b}{1-ab}.$$
Once we have this, we have to check what's the local behaviour og the orbits near every equilibrium point:
We first find $$J=\begin{pmatrix} -x+(-1+x+ay) & -xa\\ -yb & -y+(-1+y+bx) \end{pmatrix},$$
but checking every equilibrium point $(J_{(0,0)},J_{(1,0)},J_{(0,1)},\dots )$ doesn't give me nothing.
Am I missing something? Once I have the local behavior on the equilibrium points, how can I prove it has no periodic orbits?
Thanks for your time.
Do you have any conditions on the values of $a,b$? Calculating the Jacobian I got, $$ J\left( x,y \right)=\left[ \begin{matrix} 1-2x-ay & -ax \\ -by & 1-bx-2y \\ \end{matrix} \right] $$ Then $ J\left( 0,0 \right)={{I}_{2\times 2}} $ so the origin is unstable. Let $\sigma(A)$ denote the set of eigenvalues of $A$. Then,
$$ \sigma \left( J\left( 0,1 \right) \right)=\left\{ -1,1-a \right\}\quad \text{and}\quad \sigma \left( J\left( 1,0 \right) \right)=\left\{ -1,1-b \right\} $$
so for $a,b >1$ these equilibria will be stable. Finally,
$$\sigma \left( J\left( \frac{a-1}{ab-1},\frac{b-1}{ab-1} \right) \right)=\left\{ \frac{(a-1)(b-1)}{ab-1},\frac{1-ab}{ab-1} \right\} $$
If $a>1, b>1$ then the first eigenvalue here is positive; so this one is unstable.
Here is a plot for $a=b=2$ with the equilibria marked in red.