i'm following a course in Hamiltonian systems and regarding the part of linear systems I found this exercise from a book and need to solve it. My ideas are just after the test of the exercise.
Consider the system $(*)$ $Mq'' + V q = 0$, where M and V are n × n symmetric matrices and $M$ is positive definite. From matrix theory there is a nonsingular matrix $P$ such that $P^T MP = I$ and an orthogonal matrix $R$ such that $R^T (P^T V P)R = Λ = diag(λ1, . . . , λn).$ Assume now that the above equation can be reduced to $p''+Λp = 0$.
a) Show that if V has one negative eigenvalue, then some solutions of $(*)$ tend to infinity as $t→±∞$.
b) Consider the system $(**)$ $Mq'' + ∇U(q) = 0$, where $M$ is positive definite and $U : R^n → R$ is smooth. Let $q_0$ be a critical point of $U$ such that the Hessian matrix of $U$ at $q_0$ has one negative eigenvalue (so $q_0$ is not a local minimum of $U$). Show that $q_0$ is an unstable critical point for the system $(**)$
My idea for point a) is to use the undamped pendulum to visualize this: i know that in the phase plane when you have a negative eigenvalue if i am right solutions are from the origin to infinity .