Periodic solutions are Lyapunov Stable

Question

Consider the ODE $$ \left\{ \begin{align*} \dot{x}=&y\\ \dot{y}=&-x^2-bx-c. \end{align*}\right. $$

Under the assumption that $b^2-4c>0$, we find the equilibria $P_1=\left(\frac{-b+\sqrt{b^2-4c}}{2},0\right)$ and $P_2=\left(\frac{-b-\sqrt{b^2-4c}}{2},0\right)$. Around $P_1$, the linearized system allow us to conclude that $P_1$ is a center and orbits around $P_1$ are periodic.

Using the first integral $F(x,y)=\frac{y^2}{2}+\frac{x^3}{3}+\frac{bx^2}{2}+cx$ around $P_1$, we can use Morse Lemma to prove that these periodic solutions also occur on the non linear system as well, and therefore, stable.

I want to prove, however, that these periodic solutions (for the non linear system) are Lyapunov stable. Looking at the system on polar coordinates gave me nothing. Can someone give me a hint (not an answer)?

As reference, this is from exercise 5.4 from Verhulst's "Nonlinear Differential Equations and Dynamical Systems".

Thanks in advance!

Answer 1

It's a bit hard to give a hint without giving away the answer, but here's an attempt:

The periodic trajectories form a family of closed curves around $P_1$ (at least if $b>0$, which perhaps is understood). Like this, for example: WA plot. So if you're circling around on one of those curves and make a small perturbation, where do you end up?

Answer 2

I don't know if I answered your question. Let $\lambda_{1,2}=\frac{-b\pm\sqrt{b^2-4c}}{2}$ and $u=x-\lambda_1,v=y$ and then the equations become $$ u'=v,v'=-u^2-\sqrt{b^2-4c}u. $$ Let $$ V=\frac13u^3+\frac12\sqrt{b^2-4c}u^2+\frac12v^2 $$ and then $\dot{V}=0$. So $V\equiv C$ is a trajectory for any $C$. For small $C>0$, $V\equiv C$ is a closed trajectory. So $P_1$ is not Liapunov stable.