Take the dynamical system: $$x' = 0.5(1-x)xy$$ $$y' = -y(1-x)^3-y^2(2x^2-1.5x+0.5)-2x(1-x)^4+x(1-x)^3.$$ I want to find the central manifold and deduce its dynamics (stable or unstable). The above system is already in the following required form: $$ x' = Ax + f(x,y)$$ $$y' = By + g(x,y)$$ where necessarily $A=0$ and $B=-1$. Given this, we can parameterise the centre manifold by: $$h(x) = ax^2+bx^3+cx^4 +O(x^5).$$ First, we compute $y' = \frac{dh}{dx}x'$ which is: $$ y' = a^2x^4 + O(x^5)$$ and we compare it with the $y'$ from the above dynamical system, which is: $$y' = -x+(5-a)x^2+(3a-b-9)x^3+(-\frac{a^2}{2}-3a+3b-c+7)x^4 + O(x^5).$$ Comparing coefficients between the two $y'$'s gives $a=5$, $b=6$ and $c=-27.5$. This means that the centre manifold should be parameterised by: $$h(x) = 5x^2+6x^3-27.5x^4 +O(x^5).$$
Question: I do not believe the stated $h(x)$ to be the correct approximation to the manifold. You can see the correct centre manifold in the figure of the phase plane for the system I have attached. If you plot $h(x)$ on something like Desmos, you can clearly see that it is not a good approximation. Can you spot an error in my working or have I not included something I should have? Thanks


Your linearized system is wrong. It should be
$$\left\{ \begin{array}{rcl} x' & = & 0x + 0y + f(x,y) \\ y' &=& -x-y+g(x,y) \end{array} \right.$$
where bith $f$ and $g$ are second order terms. In particular, the center manifold is tangent to $\{x+y=0\}$, i.e. to the vector $(1,-1)$, which is what you observe on the plot. The expression for $h$ has to be changed accordingly.