I am confused with how the integral in (6.1.1) is even defined. We have dxi as the second term for each integral but I don't understand how that makes sense. Particularly as the second entry in the p function is an element of the sigma field but we are integrating over an element of the sigma field.
Thanks for the help
Sometimes people write the integral of a function $f$ over the set $B_0$ with respect to a measure $\mu$ as $\int_{B_0} f(x_0) \, \mu(dx_0)$. In your case, $f(x_0) = \int_{B_1} p(x_0, dx_1) \cdots \int_{B_n} p(x_{n-1}, dx_n)$.
For a fixed $x_0$, the map $A \mapsto p(x_0, A)$ is a measure, so one can similarly write $\int_{B_1} g(x_1) p(x_0, dx_1)$ for the integral of $g$ over the set $B_1$ with respect to this measure. Here, $g(x_1) = \int_{B_2} p(x_1, dx_2) \cdots \int_{B_n} p(x_{n-1}, dx_n)$. The other integrals are defined similarly.