Limit sets and Poincare -Bendixson

Question

Does a) imply that the limit set is the entirety of the points $1/3<y^2+z^2<1/2$. If so what does the periodic solution in b) look like?

Answer 1

Does a) imply that the limit set is the entirety of the points

Not at all. (a) simply means the trajectories do not exit the annulus. It says nothing about what they do inside. They could be spiraling out toward the exterior circle. Or toward the interior circle. Or both, with circular separatrix in between. Or even do something like this picture (in principle; I'm not saying that your system does this):

Anyway, what you are expected to do in (a) is to study the behavior of expression $y^2+z^2$. Its time derivative is $$-2z^2 (2y^2+3z^2 -1)$$ You should check that this derivative is $\ge 0$ when $3(y^2+z^2)=1$ and it is $\le 0$ when $2(y^2+z^2)=1$. This will help with (a).

The idea of (b) is not to look for an equation of periodic orbit, but to apply the Poincaré-Bendixson theorem. Note that your system has no fixed points in the annulus (unlike my sketch above). Therefore, $\omega$-limit set of any orbit in the annulus has to be a periodic orbit: other possibilities are ruled out by the lack of fixed points.