I found this text online:
"In general, integrating the $\delta$ function or one of its integrals makes it smoother. Differentiating it increases the discontinuities. For example $\int\delta $ is discontinuous itself. $\int \int \delta $ is continuous but with a discontinuous first derivative. $\int \int \int \delta$ is continuous, but with a discontinuous second derivative, etc..."
I agree that $\int\delta $ is certainly discontinuous since it equates to the Heaviside function. However I do not agree that $\int \int \delta $ is continuous since wouldn't this simply be the Ramp function ($\int H(x)$) And isn't the Ramp function also discontinuous at $x = 0$?
The first integral is the Heaviside function. It's discontinuous at $0$ because the limits from the left and from the right are different ($0$ and $1$). One might explain the discontinuity by saying that one can't trace the graph without lifting the pencil.
The second integral is continuous: you can trace this graph without lifting your pencil. The limit as $x\to 0$ is $0$, and it agrees with the value of the function there.
The fact that there is a corner at $0$ means something different: this function is not differentiable at $0$. (The slopes from the left and from the right don't match.)
And the third integral is differentiable: