Let $M$ be a smooth manifold and $f$ a smooth function $M\to\mathbb{R}$. Let $p$ be a critical point of $f$. We define the Hessian of $f$ at $p$ to be the symmetric bilinear functional $f_{**}$ on $T_p M$ given by $f_{**}(v,w)=\widetilde{v}_p(\widetilde{w}(f))$ where $\widetilde{v}$ and $\widetilde{w}$ are extensions of $v$ and $w$ to vector fields on $M$. It must be checked that $f_{**}$ is symmetric and well defined.
Milnor states that $f_{**}$ is symmetric because $\widetilde{v}_p(\widetilde{w}(f)) - \widetilde{w}_p(\widetilde{v}(f)) = [\widetilde{v},\widetilde{w}]_p(f)$ where $[\widetilde{v},\widetilde{w}]$ denotes the Poisson bracket, which he says satisfies $[\widetilde{v},\widetilde{w}]_p(f)=0$ because $f$ has a critical point at $p$. Thus since $\widetilde{v}_p(\widetilde{w}(f))$ is independent of the extension of $v$ and $\widetilde{w}_p(\widetilde{v}(f))$ is independent of the extension of $w$ we have that $f_{**}$ is well defined and symmetric by the preceding.
My confusion is why $[\widetilde{v},\widetilde{w}]_p(f)=0$. I'm not familiar with Poisson bracket but this looks like the Lie bracket to me, not the Poisson bracket. I'm also not sure why it is zero when evaluated at $p,f$.
Also one more (non essential) question out of curiosity, can we only define the Hessian in a coordinate free way at points where a function has a critical point? This would seem strange to me.
You're correct that $[\tilde{v}, \tilde{w}]$ is the Lie bracket of vector fields. $[\tilde{v},\tilde{w}]_p(f) = 0$ since $$[\tilde{v},\tilde{w}]_p(f) = df_p([\tilde{v},\tilde{w}]) = 0,$$ as $p$ is a critical point of $f$.
This definition of the Hessian only makes sense at critical points. One way to get a coordinate-free definition of the Hessian everywhere on the manifold is to choose a Riemannian metric (so that we get the Levi-Civita connection $\nabla^{LC}$) and define $$\mathrm{Hess}(f)(X,Y) = (\nabla^{LC} df)(X,Y).$$ This definition depends on the choice of Riemannian metric, however. Nevertheless, one can show that at a critical point, it is equivalent to the definition from Milnor's book.