The system of ODEs is: $$ \dot{x} = -2x+yz \\ \dot{y} = x-xz \\ \dot{z} = xy $$
I found two lines of equilibria etc. but I now need to find the parameters for this "energy" or Lyapunov function, so as to construct a bounding ellipsoid I suppose. If it has finite size then I am thinking it contains an infinite number of orbits that either entering it, or were there, but never leave ($\frac{dL}{dt}=0$).
$L(x,y,z) = ax^2 + by^2 + cz^2$
I differentiated this and set to zero, to create a `shell' where the Lyapunov function is constant on (like energy), and came up with,
$$ 2xyz(a-b+c) + 2bxy-4ax^2 = 0, $$ which to get rid of the first term I set $a=b-c$, leaving,
$$ 2bxy=4(b+c)x^2. $$
Unfortunately, I still have a term $xy$, and do not see how to get it into the form of the ellipsoid. Is this the right method? I don't think completing any squares will help either.... In the end I am supposed to add a cubic term to the RHS $(\dot{x}_i = \cdots - x_i^3)$ and then show the origin is globally asymptotically stable, if that helps anyone understand the type of question.
In order for the derivative of your Lyapunov function to be always $\le 0$, you'd need $b=0$ as well (because $xy$ is sometimes positive and sometimes negative). And it's $a = b - c$, not $a = b + c$. You'll get hyperbolic cylinders, not ellipsoids. The origin is not asymptotically stable for this system (before you add the cubic term), because of those lines of equilibria.