I have the following system
\begin{align*} \dot{x} &= -y+xy,\\ \dot{y} &= x+\frac{1}{2}(x^2-y^2)\\ \end{align*} I have to prove that this system has periodic solutions but no limit cycles.
My attempt: In order to prove that there exist a periodic solution I change my system into polar form. Thus,
$$\dot{r}=r^2\sin\theta (cos^2\theta+1/2cos(2\theta)),$$
but I cannot conclude anything from this. Another approach that I tried is guess the periodic solution and see if it satisfies the system. The most close thing that I got was a parametrization from a hyperbole but partially satisfies the system. Moreover, for the cycle limit I tried to apply the Bendixon criterion but again I cannot conclude anything.
Any help will be appreciated.
Your equations are equivalent to $$\dot{r}=r^2\sin\theta (\cos^2\theta+1/2\cos(2\theta)),\dot{\theta}=1$$ So $$ \frac{\dot{r}}{r^2}=\sin\theta (\cos^2\theta+1/2\cos(2\theta)),\theta=t+c. $$ For simplicity, choosing $c=0$ and integrating w.r.t $t$, one has $$ \frac{1}{r(t)}-\frac{1}{r_0}=-\int_0^{t} \sin t (\cos^2t+1/2\cos(2t))dt=-\frac1{3}\bigg[3+5\cos(t)+\cos(2t)\bigg]\sin^2(t/2) $$ and hence $$ r(t)=\frac{r_0}{1-\frac{r_0}{3}\bigg[3+5\cos(t)+\cos(2t)\bigg]\sin^2(t/2)} $$ which implies $r(t)$ is periodic for small $r_0>0$. Clearly if we choose $r_0=3$, and then $$ \lim_{t\to(\pi/2)^-}r(t)=\infty $$ and hence $r(t)$ don't have a limit cycle.