If I have this nonlinear state space model: $$ \begin{matrix} \dot{x}_1 = x_2\\ \dot{x}_2 = -\frac{g}{l}\sin(x_1) -\frac{k}{m}x_2 \end{matrix} $$
According to this Wikipedia article: https://en.wikipedia.org/wiki/LaSalle%27s_invariance_principle#Example:_the_pendulum_with_friction
I get this Lyapunov function:
$$ V(x_1, x_2) = \frac{g}{l}\left[1-\cos(x_1)\right] + \frac{1}{2}x_2^2$$
Question:
Why do I get that Lyapunov function and not this one:
$$ V(x_1, x_2) = -\frac{g}{l}\sin(x_1) -\frac{k}{m}x_2$$
The same here: https://en.wikipedia.org/wiki/Lyapunov_stability#Example
How do I determine that I should end up with this Lyapunov function:
$$ V = \frac{1}{2}(x_1^2 + x_2^2) $$
Those Lyapunov function doesn't make sense.
My goal is to check stability in the state vector $x_e$.
What's the method to find that Lyapunov function of that nonlinear state space model? I'm not going to linearize it.
A candidate Lyapunov function should always at least be semi-positive definite, so $V(\vec{x})\geq 0\ \forall\ \vec{x} \in \mathbb{R}^n$. In order to say something about the stability of the system the time derivative of this Lyapunov function should then at least be semi-negative definite, so $\dot{V}(\vec{x})\leq 0\ \forall\ \vec{x} \in \mathbb{R}^n$. One can view the Lyapunov function as a sort of virtual energy of the system, with lowest energy state set as zero. If the Lyapunov function is always greater or equal to zero and its derivative always smaller of equal to zero, then the value of the Lyapunov function should always decrease or stay constant.
Your candidate Lyapunov function is not semi-positive definite, so from that one can already conclude that it will not be a suitable Lyapunov function. However the first candidate Lyapunov function is semi-positive definite
$$ V(x_1,x_1) = \frac{g}{l} (1 - \cos x_1) + \frac12 x_2^2. $$
The time derivative of this candidate Lyapunov function can be found with
$$ \dot{V} = \frac{\partial V}{\partial x_1} \dot{x}_1 + \frac{\partial V}{\partial x_2} \dot{x}_2 = \frac{g}{l} \sin(x_1)\, x_2 + x_2 \left(-\frac{g}{l} \sin x_1 - \frac{k}{m} x_2\right) = -\frac{k}{m} x_2^2. $$
The time derivative of the candidate Lyapunov function is thus semi-negative definite (assuming $k,m>0$).
It can be noted that the system in your question has infinitely many equilibriums and that the candidate Lyapunov function is only equal to zero at its stable equilibriums.