I have three maps $F:\mathbb R\rightarrow\mathbb R$
$F(x)=-x, F(x)=x+x^2, F(x)=x-x^3$
I want to check whether the non-hyperbolic fixed point at the origin is Liapunov stable, asymptotically stable or not.
Well in all three cases the non-hyperbolic fixed point is $0$, because we know that for being non-hyperbolic the eigenvalues of the linearized system need to have a non-zero real part.
The definition for Liapunov stable I use is: $x$ is Liapunov stable iff for all $\epsilon>0$ there exists $\delta >0$ s.t if $|x-y|<\delta$ then $|\phi(x,t)-\phi(y,t)|<\epsilon$
Quasi-asyptotically stable iff there $\exists \delta>0$ such that if $|x-y|<\delta$ then $|\phi(x,t)-\phi(y,t)|\rightarrow 0$ as $t\rightarrow \infty$
Asymptotically stable if both defintions above hold.
Question: How can find out now explicitly in my three cases whether there is some kind of stability? Simply by solving the diff equations?
If you look at $F=x+x^2$, i.e. the ODE $$ x' = x+x^2 $$ you can see that if you start out with $x(0)=-1$, you get a constant solution, since then $x'(0)=0$. The same holds for $x(0)=0$.
Now watch what happens if $x(0)$ is slightly smaller than $-1$. You then get $x'(0) > 0$, i.e. $x$ will get larger as time progresses. Due to the uniqueness theorem, the solution can't get larger than $-1$, though, so it'll converge to something between $x(0)$ and $-1$.
Now let $x(0)$ be slightly larger than $-1$ (but less than $0$). You then get $x'(0) < 0$, so the solution will fall, and again due to uniqueness converge to something between $-1$ and $x(0)$.
Finally, look at $x(0) > 0$. Then, $x'(0) > 0$, so the solution will grow, which will increase $x'$ even further, leading to faster growth, and so on.
Can the system thus be lyapunov-stable? No - an arbitrary small change in the initial condition, i.e. $x(0)=0$ vs. $x(0)=0+\delta$ leads to arbitrary large differences after enough time has passed.
A more formal method of showing (but not disproving) lyapunov stability around the equilibrium point $0$ is to find a lyapunov function, i.e. a function $V(x) \geq 0$ with $V(x)=0$ only for $x=0$ and whose time derivative $\frac{df}{dt}V(x) \leq 0$, again taking zero only for $x=0$. For $F(x)=-x$ that's particularly easy - just take $$ V(x)=x \text{.} $$ Since $\frac{df}{dt}V(x) = x' = -x$, that function is indeed a lyapunov function, and thus the system is lyapuniv-stable around $0$. It's also asymptotically stable, because the lyapunov function's property don't only hold locally around $0$ (as would be sufficient for lyapunov stability), but globally, and because whenever $|x|\to\infty$ then $V(x) \to \infty$.