I am having trouble understanding why line 3 to line 4 works, specifically when the derivative of u with respect to time is taken.
u is an integral of a function of tau with respect to tau, but taking the derivative with respect to time results in f(t).
How is this possible?

For any continuous function $f$, we have that
$$\frac{d}{dx} \left(\int_a^x f(y)dy \right) = f$$
this is called the Fundamental theorem of calculus.
In your case, you have $t$ in the upper bound of the function. for every $t$, $\int_0^t f(\tau)d\tau$ is some number, so you have a function of $t$ (not of $\tau$.