In Selberg's PDE lecture notes, to prove Theorem 1 (which is equivalent to proving uniqueness of homogeneous wave equation), he defined the energy function as $$ E(t) := \frac{1}{2}\int_{B_t} |\partial u|^2\,dx $$ He then proceeded to differentiate this energy function to get the following equation: $$ E'(t) = \frac{1}{2}\int_{B_t}(u_tu_{tt} + \nabla u\cdot\nabla u_t)\,dx - \frac{1}{2}\int_{\partial B_t}|\partial u|^2\,d\sigma(x) \tag{*}\label{*} $$
I understand that the second term is present because, with the way $B_t$ was defined, it changes with time so need to account for it when we differentiate. However, I can't figure out how did he get the negative sign in the second term.
In his succeeding exercise, the result established is $$ \phi(r):= \int_{B_r(x)}f(r,y)\,dy \quad\Rightarrow\quad \phi'(r) = \int_{B_r(x)}\partial_rf(r,y)\,dy + \int_{\partial B_r(x)}f(r,y)\,d\sigma(y) $$ which has the second term being positive as opposed to the negative in Eqn (*). This negative term was crucial in closing his proof for Theorem 1 so I don't believe it is a typo. Has this just got to do with how $\partial B_t$ was oriented or is there something deeper that I'm missing?
I think this resource (Theorem 5.12) shows how you get the first equation. To be honest, I am not sure how the hint to the first exercise should help here (it confused me as well).
So the idea here is to observe that these are equivalent: $$ \int_{B_t} \phi(x, t) \mathrm d x = \int_0^{t_0 - t} \int_{\partial B_t} \phi(x, t) \mathrm ds \mathrm d \tau$$ Trying to put it in words, you cover the same domain of integration if you integrate over the whole ball with radius $t_0 - t$ or if you integrate only over the surface of that ball, $\partial B_t$, and then "inflate it" by integration over the radius $t_0 - t$. It helped me to think of this as in sperical coordinates (as you probably can already infer from my arguing).
Now with this in mind, rewrite \begin{align} E(t) = \frac{1}{2} \int_{B_t} | \partial u |^2 \mathrm d x = \frac{1}{2} \int_0^{t_0 - t} \int_{\partial B_t} | \partial u |^2 \mathrm ds \mathrm d \tau \end{align} Note that the term $\int_{\partial B_t} | \partial u |^2 \mathrm ds $ is basically only a scalar function $f(\tau, t)$ and the energy is its one-dimensional integral over $\tau$. Thus, you can apply the Leibniz Integral Rule in 1D (the exercise is essentially about the multi-dimensional case) to obtain \begin{align} E'(t) &= \frac{d}{dt} \frac12 \int_0^{t_0 - t} f(\tau, t) \mathrm d \tau \\ &= \frac12 \bigg( f(t, t - t_0) \frac{d}{dt} (t_0 - t) - f(t, 0) \frac{d}{dt} (0) + \int_0^{t_0 - t} \partial_t f(\tau, t) \mathrm d \tau \bigg) \\ &= \frac12 \bigg( f(t, t - t_0) (-1) - 0 + \int_0^{t_0 - t} \partial_t f(\tau, t) \mathrm d \tau \bigg) \\ &= \frac12 \bigg( -\int_{\partial B_t} | \partial u |^2 \mathrm ds + \int_0^{t_0 - t} \partial_t \int_{\partial B_t} | \partial u |^2 \mathrm ds \mathrm d \tau \bigg) \\ &= \frac12 \int_0^{t_0 - t} \int_{\partial B_t} \partial_t | \partial u |^2 \mathrm ds \mathrm d \tau - \frac12 \int_{\partial B_t} | \partial u |^2 \mathrm ds \\ &= \frac12 \int_{ B_t} \partial_t | \partial u |^2 \mathrm dx- \frac12 \int_{\partial B_t} | \partial u |^2 \mathrm ds \\ &= \frac12 \int_{ B_t} \partial_t (\nabla u \cdot \nabla u + u_t^2) \mathrm dx- \frac12 \int_{\partial B_t} | \partial u |^2 \mathrm ds \\ &= \frac12 \int_{ B_t} 2 (\nabla u \cdot \nabla u_t + u_t u_{tt}) \mathrm dx- \frac12 \int_{\partial B_t} | \partial u |^2 \mathrm ds \\ &= \int_{ B_t} (\nabla u \cdot \nabla u_t + u_t u_{tt}) \mathrm dx- \frac12 \int_{\partial B_t} | \partial u |^2 \mathrm ds \end{align}