Find all of the critical points for the following nonlinear system. $$\begin{pmatrix}\dot{y}_1 \\ \dot{y}_2\end{pmatrix}=\begin{pmatrix}-y_1+ y_2 - 2\\ y_1 -y_1y_2^2\end{pmatrix}$$ and then use linearisation to find the type and stability of the critical points which lie on an axis.
Critical points occur at $$\begin{pmatrix}-y_1+ y_2 - 2\\ y_1 -y_1y_2^2\end{pmatrix}=\begin{pmatrix}0\\0\end{pmatrix}$$
$$y_2=y_1+2$$$$y_1-y_1(y_1+2)^2=0$$ $$\implies y_1-y_1(y_1^2+4y_1+4)=0$$ $$y_1-y_1^3+4y_1^ 2+4y_1=0=y_1^2-4y_1-5=(y_1-5)(y_1+1)$$
and hence $(y_1,y_2)=(5,7)\text{&}(-1,1)$
then using linearisation we get $F=\dot y$
$\dot{F}=\begin{pmatrix}-1&1\\1-y_2^2&2y_1y_2\end{pmatrix}$
$\dot{F}(5,7)=\begin{pmatrix}-1&1\\-48&70\end{pmatrix}$,$(-1-\lambda)(70-\lambda)+48=0=\lambda^2-69\lambda-70,\lambda=-70,1$ so this is an unstable saddle point.
$\dot{F}(-1,1)=\begin{pmatrix}-1&1 \\ 0&-2\end{pmatrix},\lambda=-1,2$ so this is also an unstable saddle point.
Is this the correct procedure, thank you!
You have a couple issues in your work that I will point out and then you can finish the problem.
The first issue is here:
$$y_1-y_1^3+4y_1^ 2+4y_1=0=y_1^2-4y_1-5=(y_1-5)(y_1+1)$$
You should have:
$$y_1 - y_1 (y_1+2)^2 = -y_1^3 - 4 y_1^2 - y_1 = -y_1(y_1+1) (y_1+3) = 0$$
This yields three critical points as:
$$(y_1, y_2) = (-3, -1), (-1, 1), (0, 2)$$
The second issue is that there is a sign problem in your Jacobian Matrix, it should be:
$$J(x,y) =\begin{pmatrix}-1&1\\1-y_2^2&-2y_1y_2\end{pmatrix}$$
Now, you can reevaluate each of the critical points using the Jacobian and see what each of the eigenvalues are and then draw the phase portrait and figure out stability.
The phase portrait should be:
$~~~~~~~~~~~~~$