Calculate the height of the tetrahedron OABC with respect to O, given the points O(0,0,0) , A(2,4,0), B(0,2,4), C(6,0,2) .
Volume is $$V = \frac{1}{3}S_{base}h = \frac{|(A \cdot (B \times C) )|}{6} = \frac{88}{6}$$
And the area of triangle ABC is
$$S_{base} = \frac{|\vec{AB} \times \vec{BC}|}{2} = 10\sqrt2$$
And according to my calculations this should result in $h = \frac{11\sqrt2}{5}$.
$$\frac{10\sqrt2}{3}h = \frac{88}{6} \rightarrow h = \frac{11\sqrt2}{5} $$
But my textbook says this should result in $h = \frac{13\sqrt2}{5}$. So who's correct?
Your book is correct.
The volume of tetrahedron is $\frac{104}{6}$, not $\frac{88}{6}$. This is where you're wrong.
(Always be careful while calculating the value of determinant, sometimes we forget to multiply by negative sign, while opening through first row, in the case of center element, I.e. $a_{12}$.)