3 equilibrium points are obtained in this system $$ \begin{align} \dot{x}&=y+1.6xy-2.7x\\ \dot{y}&=0.625x-xy+1.2y^2-0.7y \end{align} $$ as in this streamplot graph
(0,0) in blue
(0.3,0.5) is an unstable saddle point (in black)
(1.5,1.2) at the stable point (in red)
Calculating the Jacobian of the system gives \begin{equation} J = \begin{pmatrix} -2.7 + 1.6 y & 1 +1.6 x \\ 0.625 - y & - 0.7 - x +2.4 y \end{pmatrix} \end{equation} with 2 stable points: 1 at the origin and another at the red point (as in the graph attached)
The eigenvalues of the system yields $$ \lambda_{1,2}=\lambda+(\mbox{tr})\lambda+\mbox{det} $$ where $$\mbox{tr} = -3.4-x+4y \quad \mbox{and} \quad \mbox{det} = 1.265-4.9x+3.84x^2$$
I want to show that the system is locally stable at the red point, which I believe one of the method is by using Lyapunov function.
This is my first attempt so far (but this is when I assumed that the function is globally stable)
$$ \begin{align} V(x,y)&=x^2-1.6y^2 \\ \dot{V} &= 2x\dot{x} -3.2y\dot{y} \\ &=1.6x^2y-5.4x^2+3.6xy^2-3.84y^3+2.24y^2 \end{align} $$ which is not always positive definite (in $V$) and negative definite (in $\dot{V}$)
I’m still new with this approach, can anyone help to clarify the process? Thanks in advance!