Let $f$ be a Morse function. Define the gradient vector field on a Riemannian manifold by $df_p(w)=\langle \text{grad } f|_p, w\rangle \; \text{ for all } p\in M, w\in T_pM.$ A zero $p$ of $\mathrm{grad}$ is then a critical point of $f$ ($d_pf = 0$) and for a local chart $(U,h)$ with $h(p) = 0$ the Hessian $\left( \frac{\partial^2(f \circ h^{-1})}{\partial x_i \partial x_j}(0)\right)$ is nonsingular due to the Morse property of $f$.
$p$ is a non-degenerate zero of $\mathrm{grad}$ if the differential of the pushforward of $\mathrm{grad}$ at $0$ is an isomorphism. I suppose the latter is just given by the Hessian matrix above but how can I prove it ?
The Hessian of a function can be defined as $\text{Hess}(f) = \nabla \text{grad} f$, in other words the Hessian of $f$ at a point $p$ is the linear map $T_p M \to T_p M$ given by $$ \text{Hess}(f)_p : v \mapsto \nabla_v(\text{grad}f)|_p. $$ The fact that the function $f$ is Morse means the Hessian is nondegenerate and hence at each tangent space this map is an isomorphism. That "the differential of the pushforward of grad at 0 is an isomorphism" should follow by writing the above map in coordinate charts.