Given the planar dynamical system
$$\dot x = x - y - x^3, \\ \dot y = x + y - y^3$$
show this is positively invariant in the annulus $1 \leq x^2 + y^2 \leq 3$. Hints:
$$x^4 + y^4 = (x^2 + y^2)^2 − 2x^2y^2$$
$$x^4 + y^4 = \frac 12 (x^2 + y^2)^2 + \frac 12 (x^2 - y^2)^2$$
My attempt:
I am really unsure whether what I am doing is correct or not, but I seem to get the right answer.
$$V:= x^2 + y^2$$
Take the derivative,
$$\frac{dV}{dt} = \frac{dV}{dx} \frac{dx}{dt}$$
which gives
$$\frac{dV}{dt} = \pmatrix{2x \\ 2y} \cdot \pmatrix{x-y-x^3 \\ x+y-y^3} = 2x^2 - 2xy - 2x^4 + 2xy + 2y^2 - 2y^4 \\ = 2(x^2 + y^2 -x^4 - y^4)$$
By using the 2 hints given I get that
$$2(x^2 + y^2 -x^4 - y^4) = 2(x^2 + y^2) - 2(x^2 + y^2)^2 + 4x^2y^2$$
and
$$2(x^2 + y^2 -x^4 - y^4) = 2(x^2+y^2) - (x^2+y^2)^2 - (x^2-y^2)$$
Converting these into polar coordinates will give
$$2r^2(1-r^2) + 4x^2y^2 \geq 0$$
when $r^2=1$, and
$$2r^2 \left( 1-\frac{r^2}{2} \right) - (x^2-y^2)^2 < 0$$
when $r^2=3$. Therefore can conclude from this that they are both pointing inwards.
Even though I am getting the right answer, I don't know if my methodology is correct. The place where I am not confident is when I call $V := x^2+y^2$ and then take like the orbital derivative.
You found that $$ 2V(1-V)\le \dot V\le V(2-V) $$ Treating these differential inequalities in the Bernoulli fashion now divide by $V^2$ to get for $U=1/V$ $$ 2U-2\le -\dot U\le 2U-1\\ 1\le\dot U+2U\le 2\\ U_0e^{-2t}+\frac12(1-e^{-2t})\le U(t)\le U_0e^{-2t}+(1-e^{-2t})\\ \frac{1}{1+(1/V_0-1)e^{-2t}}\le V(t)\le \frac2{1+(2/V_0-1)e^{-2t}} $$ which shows that from any position (except the origin) the solution curve will move to the annulus $1\le V\le 2$.