I have to find a proper Lyapunov function or use the linearization method, for the following system:
$\left\{ \begin{array}{ll} x'=-x-y+xy \\ y'=x-y+x^2+y^2 \\ \end{array} \right.$
So first of all I tried a linearization. I've found out that $(x,y)=(0,0)$ is a solution of our system. I build a Jacobian matrix, substitute with above point, and find the characteristic polynomial which is $\lambda^2+2\lambda+2=0$, hence $\lambda_1=-1-i$ and $\lambda_2=-1+i$. We have that $Re \lambda <0$, so the zero solution is asymptotically stable. Am I right?
I've also tried traditionally the Lyapunov function $V(x,y)=x^2+y^2$ ... but isn't the above enough?
Phase portrait:
Please check my solution, and if anything is wrong - correct me. I would also need help with Lyapunov function in this case
A complete solution follows as :
For the equilibria :
$$\begin{cases} -x-y+xy &= 0 \\ x-y + x^2+y^2 &= 0 \end{cases} \Rightarrow \begin{cases} x=y/(y-1) \\ y/(y-1) - y + y^2/(y-1)^2 + y^2 = 0 \end{cases}$$
$$\implies $$
$$y(y-1) - y(y-1)^2 + y^2 + y^2(y-1)^2 = 0$$
$$\Rightarrow $$
$$y\big[y^3-3y^2+5y-2\big] = 0$$
This yields the following two stationary points :
$$\begin{cases} x = 0 \\ y = 0 \end{cases} \quad \text{and} \quad \begin{cases} x= -1.20557 \\ y = 0.546602\end{cases} $$
Thus, the origin $O(0,0)$ and the point $A(-1.20557, 0.546602)$ are stationary points for the given system of ODEs.
The linearization matrix is the Jacobian of the system :
$$J(x,y) = \begin{bmatrix} -1 + y & -1+x \\ 1+2x & -1+2y \end{bmatrix}$$
The linearization matrix around the stationary point, namely the origion $O(0,0)$, is :
$$J(0,0) = \begin{bmatrix} -1 & -1 \\ 1 & -1 \end{bmatrix}$$
with $\det(J(0,0)) \neq 0$, thus the origin $O(0,0)$ is a non-hyperbolic stationary point for the given system of ODEs.
The eigenvalues of the linearization matrix around the origin are :
$$\det(J(0,0) - \lambda I) = 0 \Rightarrow \begin{vmatrix} -1 - \lambda & -1 \\ 1 & -1-\lambda \end{vmatrix} = 0 \Rightarrow (\lambda +1)^2+1 = 0 \Leftrightarrow \lambda = -1 \pm i$$
Noting that $\Re\{\lambda\} = -1 < 0$, this tells us that the origin $O(0,0)$ is an asymptotically stable focus/spiral point. This is a strong and sufficient conclusion and no further testing is needed.
The same approach shall be carried out for the point $A$. Again, you will have a strong and sufficient conclusion which means no further testing is needed.
Forming a stream plot (phase portrait) for our system of ODEs :
$\qquad \qquad \qquad \qquad \qquad \qquad $
From the stream plot, we see the asymptotic stability of the origin, forming a spiral (focus). Also, the point $A$ seems to be an unstable saddle point, judging from its nature.
Personally, I think that the exercise should want you to study the origin only, as finding the solution to the other point is not easy and also computing all the stuff is also messy (its exact form isn't a "nice" fraction).
Alternative : An alternate approach would be via a Lyapunov test function. This problem though would need a kind of weird one, since none of the "classic" ones seem to work.