I have been studying the following problem of gradient systems. Given a system:
\begin{equation} \overset{\cdot}{x}=f(x), \ \ x=x(t) \in \mathbb{R}^3, \end{equation} where \begin{equation} f=-\nabla g, \ \ g\in C^1(\mathbb{R}^3), \end{equation} let's suppose that $x_0=0$ is an equilibrium point of this system. I have found references stating that when for example $x_0=0$ is also a local minimum of $g$, then this means that $0$ is a stable point of equilibrium. This is shown by means of the Lyapunov function $V=g(x)-g(0)$.
I have found no reference whatsoever about maxima. If we had to study the same system, but $x_0=0$ is now a local (or global) maximum of $g$, then is there a conclusion about the stability of $0$? I have tried defining the function $V=g(0)-g(x)$, but for this function we have that $\overset{\cdot}{V} \geqslant 0$ and not $\overset{\cdot}{V} \leqslant 0$. Is there even a way to define a Lyapunov function?
Thanks in advance.
If $g$ has a maximum, the equilibrium is unstable. (Think about the intuition: the system is pushing the point in the direction where $g$ decreases fastest!)
To show this rigorously, you can make the change of variables $s=-t$, which is equivalent to changing the sign of $g$ (so that the maximum becomes a minimum). Then apply the first result, to show that the system is stable when you use the backwards time variable $s$, hence unstable when the time $t$ runs in the usual direction.