I have the following system: $$\dot{x_1} = x_2(x_3-2) \\ \dot{x_2} = x_1(x_3-2) \\ \dot{x_3}=-x_3^3 $$ and I want to determine its equilibrium points together with their stability. To find the equilibrium points, I wrote $$x_2=0 \text{ or } x_3=2 \\ x_1=0 \text{ or } x_3=2 \\ x_3 =0$$ Thus, only equilibrium point $x^*$ for the system is the origin $0$. I found $\det{Df(0)} = 0$ so I expect that the origin is a non-hyperbolic equilibrium point, probably asymptotically stable. To determine its type, I want to find a Liapunov function. Now, I assumed I can take $V(x)=c_1x_1^2 +c_2x_2^2+c_3x_3^3$ for some positive constants $c_i$. We have $V(0)=0$ and $V(x)>0$ for any $x \neq 0$. Then, $$\dot{V}(x)=\nabla V\cdot f(x) = 2c_1x_1x_2(x_3-2)+2c_2x_1x_2(x_3-2)-2c_3x_3^4.$$
$(\star )$ Now, let $c_1=c_2=1$ and $c_3 =2$ so that we get $$\dot{V}(x)=4x_1x_2(x_3-2)-4x_3^4.$$
However, I do not know how to determine the sign of this expression to conclude the result. I want to ask
Is it enough to find a Lyapunov function which is always positive or negative to say that $x^*$ is stable/unstable/asymptotically stable?
Similar to the first question but am I allowed to set arbitary values for $c_i$'s? (The $\star$ part)
How can I deduce the result?
The equations in their current form, don't need a Lyapunov function for their stability analysis. After all, it seems the system is ill-defined. Here is why.
The third equation describing $x_3$ is totally independent from the other two, and can/should be eliminated: $$\dot{x}_3=-x_3^3\implies x_3(t)=\pm\frac1{\sqrt{2t-c_3}}$$ where $c_3$ is a constant that has to be determined according to the initial conditions: $$x_3(0)^2=\frac1{-c_3}\implies x_3(t)=\pm\frac1{\sqrt{2t+1/x_3^2(0)}}$$ Anyway, you still can't get rid of the $\pm$ sign. Because both answers satisfy the equation in its current form.
For the other two, you can simply divide them like this: $$\begin{matrix}\dot{x}_1=x_2(x_3-2)\\ \dot{x}_2=x_1(x_3-2)\end{matrix} \implies\frac{\dot{x}_1}{\dot{x}_2}=\frac{x_2}{x_1}$$ But $\frac{\dot{x}_1}{\dot{x}_2}=\frac{d x_1/dt}{d x_2/dt}=\frac{d x_1}{d x_2}$, which implies $$x_1 d x_1=x_2 d x_2\overset{\int}{\implies}x_1^2=x_2^2+c_2\quad\text{i.e.}\;x_2=\sqrt{x_1^2+c}$$ As you see, the equations are not independent. So the stability of the system cannot be determined without further information. If you put the above result into the first equation and also replace $x_3$ by the function we obtained for it, we will have a single ODE for $x_1$. I omit the rest.