I'm trying to understand how the divergence formula in curvilinear coordinates is derived, but unfortunately my textbook doesn't go into much detail. Here is what they show:
I think I understand how they get the LHS of (8.9) but I don't know how they get to the RHS. Can someone please show me how the LHS equals the RHS?

The expression you are considering is the definition of differential for a function obscured by the notation the author is (forced) to use. You can see it in one variable:
$$f(x+dx)=f(x)+f'(x)dx +[\text{terms of second and greater degree}]$$
so is, $f(x+dx)\approx f(x)+f'(x)dx$ or $f(x+dx)-f(x)\approx f'(x)dx$
Now, simply "translate" it considering that partial derivatives are the same as as derivatives for a single variable maintaining constant the variables not involved in the derivative:
$A_1h_2h_3|_{x^1x^2x^3}$ is some function of the three variables, say $g(x^1,x^2,x^3)$, then, we, following more strictly than the author the notation he chose, can write: $\left.\dfrac{\partial(A_1h_2h_3)}{\partial x^1}\right|_{(x^1,x^2,x^3)}=\dfrac{\partial g(x^1,x^2,x^3)}{\partial x^1}$, so is, the derivative wrt $x^1$ evaluated at the point $(x^1,x^2,x^3)$
You surely can complete the "translation" (remember that $x^2$ and $x^3$ act as simple constants here).