I am using one from Wikipedia (https://en.wikipedia.org/wiki/Lyapunov_stability#Definition_for_continuous-time_systems).
That is,
An equilibrium, $x_{e}$, is said to be Lyapunov stable, if, for every $\epsilon >0$, there exists a $\delta >0$ such that, if $\|x(0)-x_{e}\|<\delta $, then for every $t\geq 0$ we have $\|x(t)-x_{e}\|<\epsilon$.
One confusing part for me is that this seems to be vacuously true when $\|x(0)-x_{e}\|$ is positive. Proof is obtained by simply choosing $\delta$ to be $0.5\|x(0)-x_{e}\|$.
Also, intuitively, the definition states that a point, $x_e$, is Lyapunov stable, if distance between the trajectory and $x_e$ (i.e. $\|x(t)-x_e \|$) is bounded for all $t$.
But I am not sure how the definition above captures this idea. To me, the following definition makes more sense:
An equilibrium, $x_{e}$, is said to be Lyapunov stable, if, there exists $\epsilon>0$ such that we have $\|x(t)-x_{e}\|<\epsilon$ for all $t\geq 0$.
Please help me to identify my misunderstanding.
Loosely it says that if you start nearby at $t=0$ you need to stay nearby for all $t \ge 0$. Just choosing $\delta$ as in the question does not guarantee that $x(t)$ remains close to $x_e$.
Take the system $\dot{x} = x$. Note that $x_e = 0$ is an equilibrium. The solution is just $x(t) = x(0) e^t$.
However, for any $\epsilon>0$ there is no $\delta>0$ such that for any initial condition $x(0)$ satisfying $|x(0)| < \delta$ then $|x(t)| < \epsilon$ for all $t \ge 0$.
It should be clear that for any $\delta>0$, if we take $x(0) = {1 \over 2} \delta$, then $x(t) \to \infty$, so $x_e$ is not stable in the sense of Lyapunov.
To reiterate, if $x_e$ is stable in the sense of Lyapunov then for any $\epsilon>0$ there must exist some $\delta>0$ such that if I choose any initial condition satisfying $\|x(0)-x_e\| < \delta$ then the solution $x(t)$ remains $\epsilon$ close to $x_e$ for all $t \ge 0$.
We must have $\delta \le \epsilon$ of course, perhaps that is what is confusing you?
The following is crude, but may illustrate the idea. All curves start inside the '$\delta$ ball'. The green curves remain within the '$\epsilon$ ball', the red curve strays outside.