Given the following system:
$\dot{x}=x-y-2x(x^2+y^2)$
$\dot{y}=x+y+xy-2y(x^2+y^2)$
Converting this to polar coordinates using $r\dot{r}=x\dot{x}+y\dot{y}$ and $\dot{\theta}=\frac{x\dot{y}-y\dot{x}}{r^2}$ I got:
$\dot{r}=r-2r^3+r^2\cos(\theta)\sin^2(\theta)$
and
$\dot{\theta}=1+r\cos^2(\theta)\sin(\theta)$
I'm trying to identify the regions where $r$ is expanding or shrinking by limiting the values of $\cos(\theta)\sin^2(\theta)$ I thought rewriting it to be $\cos(\theta)(1-\cos^2(\theta))=\cos(\theta)-\cos^3(\theta)$ would help seeing as how
$-0.4\leq \cos(\theta)-\cos^3(\theta)\leq 0.4$
But I don't know how to move forward from here. Most examples lead to a much simpler reduction of $\dot{r}$ am I doing something wrong here?
Indeed the maximum of $\cos(θ)\sin^2(θ)$ is $\frac2{3^{1.5}}=0.3849...<0.4$. Now you know that the radius dynamic obeys the inequalities $$ r(1-0.4r-2r^2)\le \dot r\le r(1+0.4r-2r^2) $$ The lower bound is positive and thus the radius always increasing in time on the interval $0<r<\sqrt{0.51}-0.1=0.6141...$. The upper bound is negative and thus the radius always decreasing in time for $r>\sqrt{0.51}+0.1=0.8141...$. Thus the annulus $0.5\le r\le 1$ is a trapping region in forward time. As $\dot θ\ge 1-0.4=0.6$ on this annulus, there are no other equilibrium points in this region, thus implying the existence of a limit cycle.