I want to convert this system into Polar coordinates
$$\dfrac{dx}{dt} = y,\\ \dfrac{dy}{dt} = -\mu(x^2 + y^2 - 1)y - x$$
I know in order to convert this ODE system in polar coordinates, you would use these formulas, $$x \,dx/dt+y\,dy/dt=r\,dr/dt,\\ \,ydx/dt-x\,dy/dt=-r^2\,dθ/dt.$$ Here is what I have so far, $$xy+y(-μ(r^2-1)y-x)=r\,dr/dt,\\ y^2-x(-μ(r^2-1))*y-x.$$ Can someone please tell me how I convert the system of $dy/dt$ and $dx/dt$ into polar coordinates?
On
You can get a polar coordinates relationship linked to stability by adding
$$ \cases{ x x'= x y\\ y y'=-x y - \mu y^2(x^2+y^2-1) }\Rightarrow \frac 12(x^2+y^2)'=-\mu y^2(x^2+y^2-1) $$
and now using the polar coordinates
$$ \cases{ x=r\cos\theta\\ y=r\sin\theta } $$
we get
$$ \frac 12(r^2)' = -\mu r^2\sin^2\theta(r^2-1) $$
NOTE
$$ \left(\begin{array}{c}\dot x\\ \dot y \end{array}\right) = \left(\begin{array}{cc}\cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta \end{array}\right)\left(\begin{array}{c}\dot r\\ \dot \theta \end{array}\right) = \left(\begin{array}{c}r\sin\theta\\ -\mu(r^2-1)r\sin\theta-r\cos\theta\end{array}\right) $$
and finally
$$ \cases{ \dot r = -\mu r \left(r^2-1\right) \sin ^2\theta\\ \dot\theta = -\mu \left(r^2-1\right) \sin\theta\cos\theta-1 } $$
You just need to remember that $x=r.cos(\theta)$ and $y=r.sin(\theta)$ so $dx=cos(\theta).dr-r.sin(\theta).d\theta$ by the chain rule and the same applies for $y$. Once you have them solved it is only needed to substitute in the original equation and you are done.