First I tried to convert the system to polar coordinates. This only made things worse (unless I made some idiotic mistake).
Can I plug in the given ellipse (rectangular coordinates) into the system and solve from there? If so, we get x' = -4y and y' = x. The periodic orbits of this system are ellipses that can be parameterized as gamma(t) given.
Thank you for your help.
On
Not a formal proof by any means, but I set up the system and ran it using Runge-Kutta , step size , .001 . The drawing shows $t$ from $t = 0 ... 40\pi$ , with initial position (0,1) . A second run using a variety of initial positions shows that the ellipse is a limit cycle. Points interior to the ellipse spiral out to the ellipse, points exterior to it spiral into it.
It is easy to see that
$\gamma(t) = (2\cos 2t, \sin 2t)^T \tag{1}$
solves
$\dot x = -4y + x(1 - \dfrac{x^2}{4} - y^2), \tag{2}$
$\dot y = x + y(1 - \dfrac{x^2}{4} - y^2); \tag{3}$
we merely need substitute
$x(t) = 2\cos 2t \tag{4}$
and
$y(t) = \sin 2t \tag{5}$
into (2) and (3); we observe that
$1- \dfrac{(x(t))^2}{4} - (y(t))^2 = 1- \dfrac{4\cos^2(t)}{4} - \sin^2 t$ $= 1 - (\cos^2 2t + \sin^2 2t) = 1 - 1 = 0, \tag{6}$
which shows that along $\gamma(t)$ the nonlinearities of (2), (3) vanish; we are left with
$\dot x = -4y \tag{7}$
and
$\dot y = x; \tag{8}$
but it is easy to see that (1), or (4) and (5), satisfy (7)-(8):
$\dot x(t) = \dfrac{d (2\cos 2t)}{dt} = -4\sin 2t = -4y(t), \tag{9}$
$\dot y(t) = \dfrac{d \sin 2t}{dt} = 2\cos 2t = x(t). \tag{10}$
We have thus shown that $\gamma(t) = (2\cos 2t, \sin 2t)^T$ obeys (2)-(3); $\gamma(t)$ is clearly periodic of period $T$ such that $2T = 2\pi$, or $T = \pi$; and since
$\dfrac{(x(t))^2}{4} + (y(t))^2 = \cos^2 2t + \sin^2 2t = 1, \tag{11}$
$\gamma(t)$ clearly lies in the ellipse
$\dfrac{x^2}{4} + y^2 = 1. \tag{12}$
The above remarks address the OP's concerns, but we can with little effort say a few things more: we see that
$\gamma'(t) = (-4\sin 2t, 2\cos 2t)^T; \tag{13}$
thus
$\Vert \gamma'(t) \Vert = \sqrt{16\sin^2 2t + 4\cos^2 2t}$ $= \sqrt{4(\sin^2 2t + \cos^2 2t) + 12 \sin^2 2t} = \sqrt{4 + 12 \sin^2 2t} \ge 2; \tag{14}$
from (14), we see that $\Vert \gamma'(t) \Vert \ne 0$, thus $\gamma(t)$ cannot reverse direction as $0 \to t \to \pi$, and since
$\gamma'(0) = (0, 2)^T, \tag{15}$
we see that $\gamma(t)$ traverses the ellipse in a counter-clockwise direction; $\gamma(t)$ is a smooth, closed curve enclosing the origin as $0 \to t \to \pi$.
The system (2)-(3) has a zero at $(0, 0)^T$; so much is clear. Does it have any other equilibria? These are points where $\dot x = \dot y = 0$; at such $(x, y)^T$ we have
$-4y + x(1 - \dfrac{x^2}{4} - y^2) = 0, \tag{16}$
$x + y(1 - \dfrac{x^2}{4} - y^2) = 0. \tag{17}$
If $(x, y)^T \ne (0, 0)^T$, at least one of $x, y$ does not vanish; suppose then $y \ne 0$. Then from (17),
$1 - \dfrac{x^2}{4} - y^2 = -\dfrac{x}{y}, \tag{18}$
and if (18) is substituted into (16) we find
$-4y - \dfrac{x^2}{y} = 0 \tag{19}$
or
$4y^2 + x^2 = 0. \tag{20}$
(20) has no real solutions $x$, so there is no equilibrium point with $y \ne 0$; likewise if $x \ne 0$, (16) yields
$1 - \dfrac{x^2}{4} - y^2 = \dfrac{4y}{x}, \tag{18}$
and (17) becomes (20) once again; there are no equilibria when $x \ne 0$ either. Thus the only critical point of (2), (3) is $(0, 0)^T$; we may examine the linearization of the system at this point; the Jacobean matrix of partial derivatives at any point $(x, y)^T$ is
$J(x, y) = \begin{bmatrix} \dfrac{\partial \dot x}{\partial x} & \dfrac{\partial \dot x}{\partial y} \\ \dfrac{\partial \dot y}{\partial x} & \dfrac{\partial \dot y}{\partial y} \end{bmatrix} = \begin{bmatrix} 1 - \dfrac{3x^2}{4} - y^2 & -4 - 2xy \\ 1 - \dfrac{xy}{2} & 1 - \dfrac{x^2}{4} - 3y^2 \end{bmatrix}, \tag{19}$
and thus
$J(0, 0) = \begin{bmatrix} 1 & -4 \\ 1 & 1 \end{bmatrix}; \tag{20}$
the eigenvalues of $J(0, 0)$ are given by
$\lambda^2 - \text{Trace}(J)\lambda + \det J = \lambda^2 - 2\lambda + 5 = 0; \tag{21}$
we see from the quadratic formula that
$\lambda = \dfrac{1}{2}(2 \pm \sqrt{-16}) = 1 \pm 2i. \tag{22}$
Since the eigenvalues of $J(0, 0)$ are a complex conjugate pair with positive real part, it follows from (22) that the origin $(0, 0)^T$ is an unstable spiral point; thus no trajectory initialized at a point other than $(0, 0)^T$ may approach the origin as $t \to \infty$; by the Poincare-Bendixson theoram, since the origin is the only critical point, such an integral curve must approach a periodic orbit; if there are indeed no periodic solutions other than $\gamma(t) = (2\cos 2t, \sin 2t)^T$, then every trajectory initialized at a non-origin point within the ellipse $x^2/4 + y^2 = 1$ must approach this ellipse. These considerations, though heuristic and lacking in complete rigor, lend some degree of corroboration to the results obtained by Alan in his numerical analysis.
Hope this helps. Cheers,
and as always,
Fiat Lux!!!