I have problems understanding which determinant rules where used to get this equation here. $E$ is $n$ x $n$ and $H$ is $m$ x $m$ matrix, I hope you understand other dimensions from that
$$\begin{align} \det D(G)=&\det \begin{pmatrix} 0&\overline a & \overline f \\ \overline b & E-\left(b_i+a_j\right) & 0 \\ \overline g & 0 & H-\left(g_i+f_j\right) \end{pmatrix} \\=&\det \begin{pmatrix} 0 & \overline a \\ \overline b & E - \left(b_i + a_j\right) \end{pmatrix} \det\left(H-\left(g_i+f_j\right)\right)\\ &+\det \begin{pmatrix} 0 & \overline a \\ \overline g & H-\left(g_i+f_j\right) \end{pmatrix} \det\left(E-\left(b_i+a_j\right)\right) \end{align}$$
Whole text is here http://www.math.ucsd.edu/~ronspubs/77_01_distance_matrix.pdf .
They've divided up a matrix of the form $$\begin{bmatrix} 0 & A & B\\ C& D&0\\ E&0&F \end{bmatrix}$$ and they are telling you that its determinant is equal to a combination of determinants of smaller matrices. Specifically, $$\det \begin{bmatrix} 0 & A \\ C& D \end{bmatrix} \times \det \begin{bmatrix} F \end{bmatrix} + \det \begin{bmatrix} 0 & B\\ E&F \end{bmatrix} \times \det \begin{bmatrix} D \end{bmatrix} .$$