Classify the critical points associated with the Hamiltonian: $$H(p,q)=\frac{p^2}{2}+p \sin q.$$
The Hamiltonian equations are: $$\dot{p}=-\frac{\partial H}{\partial q}=-p \cos q,~~\dot{q}=\frac{\partial H}{\partial p}=p+\sin q.$$ Critical points are given by: $$\dot{p}=0 \implies p=0, q=\left( n+\frac{1}{2} \right) \pi$$ and $$\dot{q}=0 \implies p+\sin q=0.$$ When $p=0,$ $q=n \pi$ ; when $q=\left( n+\frac{1}{2} \right) \pi,$ $p=\mp 1.$
I need help in continuing here on ward and classify the critical points and sketching them. Appreciate your help. Thank you.
You can sketch your phase portrait quite nicely with the 'pplane' command in MATLAB. Another way is this nice online tool: http://comp.uark.edu/~aeb019/pplane.html
In your case you get this nice plot
You can see that your stable fixed points are not asymptotically stable. This is quite typical for Hamiltonian systems since these orbits around a stable critical point behave periodic like 'circles'. This can be seen as the Hamiltonian gives you a Lyapunov function that gives you stability but NOT asymptotic stability.
For example let's take the fixed point $(1,-\pi/2)$ as seen in the plot. It is stable but not asymptotically stable as the other orbits in the neighborhood don't move towards this fixed point.
We have $$ Df(p,q) = \begin{pmatrix} -\cos(q) & p \sin(q) \\ 1 & \cos(q) \end{pmatrix} $$
and at the equilibrium $(1,-\pi/2)$ we get
$$ Df\left(1,-\frac{\pi}{2}\right) = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} $$ with eigenvalues $\pm i$. These are non-hyperbolic fixed points since their real part is zero and stability can't be seen immediately. For this we introduce the concept of Lyapunov functions. But before this have a look at some unstable fixed point. For example as seen in the plot $(0,0)$. You have $$ Df\left(0,0\right) = \begin{pmatrix} -1 & 0 \\ 1 & 1 \end{pmatrix} $$ and therefore the eigenvalues $-1$ and $1$. Since one eigenvalue has positive real part we get that $(0,0)$ is unstable. You see stability of hyperbolic equilibria are quite easy to compute.
Now, back to our non-hyperbolic fixed point.
So let's take $H(x,y)=\frac{p^2}{2}+p\sin(q)+C$ and choose $C$ such that $H(1,-\pi/2)=\frac{1}{2}-1+C=0$ i.e. $C=\frac{1}{2}$.
Since $$\nabla H\left(1,-\frac{\pi}{2}\right)=\binom{p+\sin(q)}{p\cos(q)}\bigg|_{(1,-\frac{\pi}{2})}=\binom{0}{0} \hspace{0.1cm}$$ $$\text{D}^2 H\left(1,-\frac{\pi}{2}\right)=\begin{pmatrix} 1 & \cos(q) \\ \cos(q) & -p\sin(q) \end{pmatrix}\bigg|_{(1,-\frac{\pi}{2})}=\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$$
and therefore your Hessian is positiv definite and $(1,-\pi/2)$ is strict local minimum. By our theorem above $(1,-\pi/2)$ is a stable (but not asymptotically stable) equilibrium of your system.
Since I mentioned this is typical for Hamiltonian systems I also want to mention the famous Duffing-oscillator. It is described by the system
$$\binom{\dot x}{\dot y}=\binom{ y}{x-x^3} $$
with the Hamiltonian $H(x,y)=\frac{1}{2} y^2 - \frac{1}{2} x^2 + \frac{1}{4} x^4$. You will get $(\pm 1,0)$ are stable (but not asymp.) fixed points and $(0,0)$ is unstable (instability follows immediately by looking at the eigenvalues of the Jacobian.)