Is equilibrium point $x = 0$ stable for $\ddot{x}+\dot{x}=-x\sin x$?
It is a case of a bigger problem, where we need to check the stability of critical points $x \in [-\frac \pi 2, \frac \pi 2]$ for $\ddot{x}+\dot{x} = x(a-1)(\sin x-a)$. I've done with all other situations, but can not do anything with this case. Obviously, we need to say $\dot{x}=y$, $\dot{y} =-y-x\sin x$. Linear approximation doesn't help here. So, it seems we need to find Lyapunov function $V$ in order to use Lyapunov theorem or Chetaev theorem. I've tried some, but I didn't manage to find any. I think that we shoud in some way modify $V = \frac{y^2}2 - x\cos x + \sin x$, because $\dot{V} = -y^2$.
If $\mathbf{a = 0}$
The nonlinear system
$$ \left[\matrix{\dot{x} \cr \dot{y}}\right] = \left[\matrix{y \cr - x \sin(x) - y}\right] $$
has multiple equilibrium points at $x = \pi n$, where $n \in \mathbb{Z}$.
Perform a linearization on the nonlinear system and obtain
$$ \left[\matrix{\dot{x} \cr \dot{y}}\right] = \left[\matrix{0 & 1 \cr - x \cos(x) - \sin(x) & -1}\right] \left[\matrix{x \cr y}\right] .$$
If the system is linearized at $x = 0$, then we get
$$ \left[\matrix{\dot{x} \cr \dot{y}}\right] = \left[\matrix{0 & 1 \cr 0 & -1}\right] \left[\matrix{x \cr y}\right] $$
$$ \mathbf{\dot{x}} = \mathbf{A} \mathbf{x} $$
and the eigenvalues of $\mathbf{A}$ are obtained by solving for the roots of the characteristic equation
$$ |\lambda \mathbf{I} - \mathbf{A}| = 0 $$
$$ \lambda_{1} = -1, \quad \lambda_{2} = 0 .$$
Since one eigenvalue is at the origin, the response will increase with time for any $\dot{x}(0) \neq 0$, and we conclude that the equilibrium $(x,\dot{x}) = (0,0)$ is unstable. Thus, it is not surprising that your Lyapunov function doesn't work when it is applied to an unstable point.
On the other hand, if the system is linearized at $x = -\pi$, then we get
$$ \left[\matrix{\dot{x} \cr \dot{y}}\right] = \left[\matrix{0 & 1 \cr -\pi & -1}\right] \left[\matrix{x \cr y}\right] $$
and the eigenvalues are $\lambda_{1,2} = - 0.5 \pm 1.70047i$. Since the eigenvalues have negative real parts, we conclude that the point $(x,\dot{x}) = (-\pi,0)$ is a stable equilibrium (as shown in the phase portrait by Hans).
If $\mathbf{a > 1}$
Moving on to the general problem, the system can be written as
$$ \left[\matrix{\dot{x} \cr \dot{y}}\right] = \left[\matrix{y \cr - f(a,x) - y}\right] $$
where
$$ f(a,x) = x \left(a - 1\right) \left(a - \sin(x)\right) ,$$
we can perform analysis and find that it satisfies $f(0) = 0$ and $x f(a,x) > 0$ for all $x \neq 0$ if $a > 1$ is selected. Although multiple equilibrium points exist in the trigonometric function, only one real equilibrium $(x,y) = (0,0)$ exists.
Consider the Lyapunov function candidate
$$ V(x, y) = \frac{1}{2} \left[\matrix{x & y}\right] \left[\matrix{1 & 1 \cr 1 & 2}\right] \left[\matrix{x \cr y}\right] + 2 \int_{0}^{x} f(\chi) \; d\chi > 0 \quad \text{for} \quad x \neq 0, y \neq 0 .$$
The derivative of $V$ along the trajectories of the system is given by
$$ \dot{V}(x,y) = - x f(a,x) - y^{2} $$
and it satisfies
$$ \dot{V}(x,y) < 0 \quad \text{for} \quad x \neq 0, y \neq 0 .$$
Since there is only one real equilibrium $(x,y) = (0,0)$ for $a > 1$, then it can be concluded that the origin is asymptotically stable.
The streamplot for $a = 1.8$ is shown below.