I've been trying to find the Lyapunov function for the following system,
$\dot x= x^{2}-x-y$, $\dot y=x$.
Since Ive tried all typical ones, and some variations I havent been able to conclude that the derivative of the function is $<0$. The main question, I guess, is how to face the problem of finding a Lyapunov function when some term is even, which is the case of $\dot x$.
Probably there is a clever way to find this kind of functions and for sure Im missing it so, any help would be incredibly appreciated <3
The equilibrium of this system is not globally asymptotically stable:
so there is no "global" Lyapunov function. A "local" Lyapunov function can be found by linearizing the original system. The linearized system is $$\dot x= -x-y,$$ $$\dot y=x$$ or $$ \left(\begin{array}{c}\dot x\\\dot y\end{array}\right)= A \left(\begin{array}{c}x\\y\end{array}\right), $$ where $$ A=\left(\begin{array}{rr}-1&-1\\1&0 \end{array}\right). $$ Solving the Lyapunov equation $A^TP+PA=-Q$, where $P$ is an unknown matrix, $Q$ is any positive-definite matrix, one can obtain the solution $P$, which can be used to construct the Lyapunov function $$ V(x,y)=\left(\begin{array}{cc}x&y\end{array}\right) P \left(\begin{array}{r}x\\y \end{array}\right). $$ Update begin
$V$ obtained in this way is not only a Lyapunov function for the linearized system $$\tag{1}\dot x=Ax,$$ but for the original system $$\tag{2}\dot x= f(x)$$ too. Indeed, the original system can be written as $$\tag{2a}\dot x=Ax+\alpha(x)\|x\|,$$ where $\lim_{x\to 0}\alpha(x)=0$. The directional derivative $$ \dot V\Big|_{(2a)}= (Ax + \alpha (x)\|x\|)^T Px+x^T P(Ax + \alpha (x)\|x\|) $$ $$ =x^T(A^T P+ PA)x + 2x^T P\alpha (x)\|x\|= -x^T Qx + 2x^T P\alpha (x)\|x\|. $$ The inequality $$ x^T Qx \geq \lambda_{\min}(Q)\|x\|^2, $$ where $\lambda_{\min}(Q)$ is the minimum eigenvalue of $Q$, implies $$ \dot V\Big|_{(2a)}\leq -\lambda_{\min}(Q)\|x\|^2+ \Big|2x^T P\alpha (x)\|x\|\Big| \leq -\lambda_{\min}(Q)\|x\|^2+ 2\|\alpha (x)\|\cdot\|P\|\cdot\|x\|^2 $$ $$ =\|x\|^2 \left( -\lambda_{\min}(Q)+2\|\alpha (x)\|\cdot\|P\| \right). $$ Now notice that $\|\alpha (x)\|$ can be made arbitrarily small in some neighborhood of the origin because $\lim_{x\to 0}\alpha(x)=0$, hence, $\dot V\Big|_{(2a)}$ is negative definite.
This is actually a part of the proof of the indirect Lyapunov method.
Update end
For example, if we take $$ Q=\left(\begin{array}{rr}2&0\\0&2 \end{array}\right), $$ we obtain $V(x,y)=2x^2+2xy+3y^2$.