In Milnor's Morse Theory, the Hessian of a smooth function $f : M \to \mathbb R$ defined on a manifold $M$ at a critical point $p$ is the bilinear functional on $T_p M$ defined as follows:
$$f_{**}(v, w) = \tilde v_p(\tilde w(f))$$
where $\tilde v, \tilde w$ are vector fields that extend $v$ and $w$. It is written that $\tilde w(f)$ denotes the directional derivative of $f$ in the direction $\tilde w$ and that $\tilde v_p$ is, of course, $v$. I am assuming that this means $\tilde w(f) = T_p(f)(w)$, but
- I do not know what it means to write $v(T_p(f)(w))$
- it is claimed that $f_{**}$ is symmetric because $$\tilde v_p(\tilde w(f)) - \tilde w_p(\tilde v(f)) = [v, w]_p(f) = 0$$ since $p$ is critical. If I am not mistaken, the argument used here is that since $p$ is critical, any directional derivative of $f$ at $p$ (e.g. $[v, w]_p(f)$) is zero, but wouldn't that also mean that $\tilde w(f) = \tilde v(f) = 0$?
Hopefully, I am only mixing up notations. I would appreciate any comment.
You don't need anything as sophisticated as Lie derivatives or connections to make sense of this. There are two main points to keep in mind:
First, if $w\in T_pM$ is a tangent vector at $p$, then $w$ acts as a linear map from $C^\infty(M)$ to $\mathbb R$; for any smooth function $f$, the number $w(f)$ is interpreted as the directional derivative of $f$ in the direction $w$. It can be written in a number of equivalent ways: $$ w(f) = df_p(w) = T_p(f)(w). $$
Second, if $\bar w$ is a smooth vector field and $f$ is a smooth function, then $\bar w(f)$ is another smooth function, whose value at $p\in M$ is $$ \bar w(f)(p) = \bar w_p(f). $$ So, given another vector field $\bar v$, we can apply $\bar v_p$ to this function and get a real number $\bar v_p(\bar w (f))$. Note that in general this value depends on knowing the vector field $\bar w$ in a neighborhood of $p$, not just on the value $\bar w_p$. Moreover, $\bar v_p(\bar w (f))$ and $\bar w_p(\bar v (f))$ might be different numbers, because $$ \bar v_p(\bar w (f)) - \bar w_p(\bar v (f)) = [\bar v,\bar w]_p (f).\tag{$*$} $$ (This is essentially the definition of the vector $[\bar v,\bar w]_p$.)
However, if $p$ is a critical point of $f$, then $[\bar v,\bar w]_p (f) = df_p([\bar v,\bar w]_p) = 0$, so $(*)$ shows that $\bar v_p(\bar w (f)) = \bar w_p(\bar v (f))$. It also shows that this number is independent of the vector fields $\bar v$ and $\bar w$ chosen to extend the vectors $v$ and $w$, because the first expression depends only on $\bar v_p = v$ and the second depends only on $\bar w_p = w$. Thus at a critical point $p$, the Hessian of $f$ is a well-defined symmetric bilinear form on $T_pM$. That's why Milnor treats the Hessian only at critical points.