My textbook defines Hessian functions as follows:
Suppose that $f: U \subset \mathbb{R}^n \to \mathbb{R}$ has second-order continuous derivatives $\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0)$, for $i, j = 1, \dots, n$, at a points $\mathbf{x}_0 \in U$. The Hessian of $f$ at $\mathbf{x}_0$ is the quadratic function defined by
\begin{align} Hf(\mathbf{x}_0)(\mathbf{h}) &= \dfrac{1}{2} \sum_{i, j = 1}^n \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) h_i h_j \\ &= \dfrac{1}{2} [h_1, \dots, h_n] \left[\begin{matrix}\frac{\partial^2 f}{\partial x_1^2} & \ldots & \frac{\partial^2 f}{\partial x_1\partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n\partial x_1}& \ldots & \frac{\partial^2 f}{\partial x_n^2}\end{matrix}\right] \left[\begin{matrix} h_1 \\ \vdots \\ h_n \end{matrix}\right] \end{align}
Notice that, by equality of mixed partials, the second-derivative matrix is symmetric.
This function is usually used at critical points $\mathbf{x}_0 \in U$. In this case, $\mathbf{D}f(\mathbf{x}_0) = \mathbf{0}$, so the Taylor formula (see Theorem 2, section 3.2) may be written in the form
$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + Hf(\mathbf{x}_0)(\mathbf{h}) + R_2(\mathbf{x}_0, \mathbf{h})$$
Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $f$.
Theorem 2, section 3.2, is as follows:
Theorem 2 First-Order Taylor Formula
Let $f: U \subset \mathbb{R}^n \to \mathbb{R}$ be differentiable at $\mathbf{x}_0 \in U$. Then
$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + R_1(\mathbf{x}_0, \mathbf{h}),$$
where $\dfrac{R_1(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||} \to 0$ as $\mathbf{h} \to \mathbf{0}$ in $\mathbb{R}^n$.
My question is, what is meant by this:
Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $f$.
I would greatly appreciate it if people could please take the time to clarify this.
EDIT:
I think the authors actually meant to refer to theorem 3, section 3.2:
Theorem 3 Second-Order Taylor Formula
Let $f: U \subset \mathbb{R}^n \to \mathbb{R}$ have continuous partial derivatives of third order. Then we may write
$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + \dfrac{1}{2} \sum_{i, j = 1}^n h_i h_j \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) + R_2(\mathbf{x}_0, \mathbf{h}),$$
where $\dfrac{R_2(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||^2} \to 0$ as $\mathbf{h} \to \mathbf{0}$ and the second sum is over all $i$'s and $j$'s between $1$ and $n$ (so there are $n^2$ terms).
Well the first term in the Taylor series is $f(x_0)$. This doesn't depend on $h$ and so we are calling it `constant'. Usually the next term contains $\nabla f(x_0)$, but at a critical point, this is zero. Therefore the first term in the Taylor series that depends on $h$ is the term involving the Hessian.