Finding a good Lyapunov function for $ \{ x'= y , y'= -4 x + 5x^3 - x^5 \}$

Question

For the system:

$$\begin{cases}x'= y \\y'= -4 x + 5x^3 - x^5 \end{cases} $$

I am trying to determine the stability of $(x,y)=(0,0)$ by means of a Lyapunov function. I am trying to find a good one, the regular $V(x,y)=ax^2 + by^2$ does not help me as I get odd-powered terms and products of $x$ and $y$ that do not cancel. Specifically: $$ \dot{V}(x,y) = 2axy + b xy(-1+5x^2-x^4)$$
Does someone have a better suggestion, what is the general approach in finding such a function for a given problem? I want to somehow use the fact that these odd powers of $x$ and $y$ appear in this system of equations, I haven't figured out how to do this in an effective manner.

Answer 1

Hint.

The dynamical system has an integral which is

$$ \frac 12 y^2+\frac{x^6}{6}-\frac{5 x^4}{4}+2 x^2=C $$

Studying the level curves we have the following graphics

and for $0 < C < 0.915$ we have closed level curves around the origin characterizing a center.

Answer 2

This is a conservative system, since it's equivalent to $$ x'' = -4x + 5x^3 - x^5 = -V'(x) , $$ with the potential $$ V(x) = 2x^2 - \tfrac54 x^4 + \tfrac16 x^6 . $$ So $H(x,y)=\tfrac12 y^2 + V(x)$ is a constant of motion, and the trajectories follow the level curves of $H$, which near the origin look like ellipses $2 x^2 + \tfrac12 y^2 = C$. This means that the origin is neutrally stable (i.e., stable but not asymptotically stable).

Compare the two plots (on Wolfram Alpha):

• Phase portrait
• Level curves