Stability of zero solution of ODE system when largest real part of eigenvalues is $0$.

Question

Please help me to solve the following problem:

Determine stability of zero solution of ODE system : $$\dot x = -x + 3xy, \dot y = 2x - y^2.$$

Largest real part of eigenvalues is $0$, the most difficult case.

I tried the following:

1. Use second Lyapunov theorem to construct Lyapunov functions and prove it is stable. I tried Lyapunov functions $a x^{2n} + b y^{2m}, a > 0, b>0$.

2. Use Chetaev instability theorem to prove it is unstable. I tried functions $a x^{2n} + b y^{2m}, a , b$ both positive and negative.

I spend a lot of time trying to construct above mentioned functions, but I failed. Please advice how to handle this problem.

Thanks a lot for your hints and answers!

Answer 1

Consider the restriction to the line $x=0$. It turns out that this line is invariant, i.e. if the initital value is $x(0)=0$, $y(0)=y_0$, then $x(t)=0$ for all $t$. So we have the restricted equation $$\dot{y}=-y^2$$ This equation has explicit solution, but it suffices to see that for any $y(0)<0$, the derivative is negative, so $y(t)$ is decreasing. In other words, in any neighbourhood of zero, there are initial points such that trajectories escape the neighbourhood.

Therefore the system cannot be stable.

Answer 2

EDIT

The solution of the linearized ODE is correct, but my conclusion was wrong. In fact, for stability the solution must go to zero for arbitrary deviations in $x$ and $y$ independently, but this is not the case in the solution obtained (we have $\delta y = - 2 \delta x$).

Hence, as stated by others, the zero solution is not stable.

Observation: the other stationary solution $\left\{x\to \frac{1}{18},y\to \frac{1}{3}\right\}$ is also unstable.

Original solution

The stability of a solution $( \xi(t), \eta (t) ) = (0,0)$ is found by letting

$$x(t) = \delta x(t)$$ $$y(t) = \delta y(t)$$

and linearizing the ODEs, i.e. cancelling all terms with $\delta ^2$.

This gives

$$\delta \dot x(t) = -\delta x(t)$$ $$\delta \dot y(t) = 2 \delta x(t)$$

The solution of which is

$$\delta x(t) = a e^{-t}$$ $$\delta y(t) = - 2 a e^{-t} + b$$

Where $a$ and $b$ are constants of integration. Here $b=0$ because $\delta y(t) = 0$ for $t\to \infty$, and $a$ is arbitrary.

Hence the zero soluton is stable as a small deviation $a \ne 0$ is damped out in time.