I know that the following holds for Step function, but not sure if it holds for the distribution too.
Does the following hold for the Dirac delta distribution too?
$\delta$ is a linear functional from a space of test functions. The space is here taken from Schwartz space $S$ or the space of all smooth functions of compact support $D$. The Dirac distribution is differentiable everywhere $(-\infty, \infty)$ except at the point $x = 0$ where the function has a nontrivial jump discontinuity. This can be solved by removing the discontinuity of $\delta'$ by setting \begin{equation} \delta' = 0 \end{equation} which is continuous now on the entire line even though $\delta$ is not differentiable on the real line.
The text you quoted messes with different notions of differentiability and I can't make any sense of it.
Since the delta distribution is a linear functional on a space of test functions it is not a function on the real line and hence, it does not make sense to say something like "$\delta$ is differentiable on $(-\infty,\infty)$ except at $x=0$" since $\delta$ is not defined on that set.
When viewing $\delta:\mathcal{S}\to\mathbb{R}$ (linear and continuous with respect to the usual semi-norms on the Schwartz-space – or similar on the space of test functions), it makes sense to say that $\delta$ is continuous. To say that it is differentiable, one has to define the notion of differentiability for such objects (which is done by duality).