I have been given the following system of equations: $$\begin{cases} x'=y-2x^2\\ y'=x(3-y-x^2)\end{cases}$$ The equilibrium points of this system are $(0,0), (1,2), (-1,2)$. The system has nullclines $y=2x^2, x=0, x=3-x^2$. After determining that the equilibrium solution $(-1,2)$ is stable and drawing the phase portrait. I've been given the following question:
Let $(x_0, y_0)\in \mathcal{S}_1$ ($\mathcal{S}_1$ denotes the area $0<x<\sqrt{3}$, $0<y<2x^2, 3-x^2$. In this area, $x'<0$ and $y'>0$.) and let $(x_o(t), y_0(t))$ be a solution for the system, which starts in $(x_0, y_0)$. Show that: $$\lim\limits_{t\to\infty}(x_0(t), y_0(t))=(1,2)$$
After drawing that phase portrait and determining the sign of $x'$ and $y'$, I understand that this is very likely. However, I do not know how to prove this explicitly. I hope someone can give me an idea as to how to prove this. Thanks is advance
Let's check how vector field defined by this system of ODEs behaves on the boundary of the rectangle $0 \leqslant x \leqslant \sqrt{3}, \; 0 \leqslant y \leqslant 4$. Let's denote it by $\mathcal{R}$. It's quite easy to see that along the boundary the vector field points to the inside of $\mathcal{R}$ (vector field is actuallytangent at a few points, but that doesn't spoil the picture). Due to this property, no trajectory that starts in the inside of a rectangle can escape it, hence $\omega$-limit set of any point (including points from $\mathcal{S}_1$) must be in $\mathcal{R}$.
We already have one stable equilibrium at $(1, 2)$ as a candidate for $\omega$-limit set of points from $\mathcal{R}$. Let's prove that there is nothing else. By Poincaré–Bendixson theorem, there are only three possibilities for a limit set of trajectory on plane: an equilibrium point, a limit cycle and an attracting homoclinic loop or heteroclinic contour. There is only one saddle in this system at $(0, 0)$, so no possibility for heteroclinic contour. If you check the eigenvector of this saddle that corresponds to negative eigenvalue, it points to the outside of the rectangle, hence stable manifold of saddle doesn't lie in $\mathcal{R}$ and there is no homoclinic loop in the $\mathcal{R}$. Due to Bendixson–Dulac theorem there is no limit cycles in $\mathcal{R}$: divergence of system is $\frac{d}{dx}\left(y-2x^2\right) + \frac{d}{dy}\left(x(3-y-x^2)\right) = -5x$ which is negative in the interior of $\mathcal{R}$ (and limit cycle has to lie in the interior of $\mathcal{R}$ if it exists). Thus, we ruled out all possible variants except the unique stable equilibrium.