Let $$f(x)=\sum_{p=0}^\infty J_p(x)$$ We have that $f(0)=1$ and a Bessel function like curve. Can $f(x)$ be expressed in terms of known functions? $$f(x)=1-\sum_{n=0}^\infty \sum_{p=0}^{n-1}\frac{(-1)^n(x/2)^{n+p}}{n!\ p!}$$ Is the closest I could get to a definite form by hand. The terms are just partial sums of the exponential series, which can be represented in terms of Partial gamma functions however they themselves cannot be summed.
I tried using the fact that $$ J_p(x)=\frac1{2\pi}\int_{-\pi}^\pi e^{i(pt-x\sin(t))}dt$$ However I think I run into convergence issues when trying to exploit the geometric series of $e^{ipt}$.
I have found the 'closed form' of the sum here. The question still stands on the derivation of $$\sum_{p=0}^\infty J_p(x)=\frac12+\frac{x+1}2 J_0(x)+\frac\pi4x(J_1(x) \mathbf{H}_0(x)-J_0(x) \mathbf{H}_1(x))$$ Where $\mathbf{H}_p(x)$ is the Struve function.
The derivation of the stated result is not that hard. Define
$$g(x) = J_0(x) + 2 \sum_{p=1}^{\infty} J_p(x) $$
Note that $g(0)=1$. Take the derivative:
$$g'(x) = J_0'(x) + 2 \sum_{p=1}^{\infty} J_p'(x) $$
Use the recurrence relation
$$J_p'(x) = \frac12 \left [J_{p-1}(x) - J_{p+1}(x) \right ] $$
and the fact that $J_0'(x) = -J_1(x)$. Then
$$\begin{align}g'(x) &= -J_1(x) + \sum_{p=1}^{\infty} J_{p-1}(x) - \sum_{p=1}^{\infty} J_{p+1}(x) \\ &= -J_1(x) + J_0(x) + \sum_{p=1}^{\infty} J_p(x) + J_1(x) - \sum_{p=1}^{\infty} J_p(x) \\ &= J_0(x) \end{align}$$
Therefore
$$g(x) = J_0(x) + 2 \sum_{p=1}^{\infty} J_p(x) = 1 + \int_0^x dt \, J_0(t)$$
Use the fact that
$$\int_0^x dt \, J_0(t) = x J_0(x) + \frac{\pi}{2} x \left [J_1(x) \mathbf{H}_0(x) - J_0(x) \mathbf{H}_1(x) \right ] $$
and therefore, adding $J_0(x)$ to both sides, we get
$$2 \sum_{p=0}^{\infty} J_p(x) = 1+(1+x) J_0(x) + \frac{\pi}{2} x \left [J_1(x) \mathbf{H}_0(x) - J_0(x) \mathbf{H}_1(x) \right ] $$
The stated result follows.
ADDENDUM
The issue of convergence of the sum in question has not been addressed. The OP mentioned a concern about being able to reverse order of summation and integration in an attempted derivation due to convergence concerns.
The sum easily converges because the Bessel functions exhibit a rapid decrease for fixed $x$ as $p \to \infty$:
$$J_p(x) \sim \frac1{\sqrt{2 \pi p}} \left (\frac{e x}{2 p} \right )^p \quad (p \to \infty)$$
i.e., $J_p(x) = O \left (p^{-p+1/2} \right ) $. This asymptotic behavior may be seen by observing that the first term in the Taylor series of $J_p(x)$ dominates in this limit.