Preferably using elementary mathematics.
for example , for $p = 2$
$-1 + 3 \times 2^2 - 3 \times 3^2 + 4^2 = 0$
EDIT:
changed the identity to be proven from $\sum_{j=1}^{p+3} \, (-1)^{j} \binom{p+2}{j-1} {j}^p = 0$ to $\sum_{j=1}^{p+2} \, (-1)^{j} \binom{p+1}{j-1} {j}^p = 0$, from which the other follows
Given a polynomial $p(x)$, define $\Delta p$ to be the polynomial defined by $$\Delta p(x)=p(x)-p(x+1).$$ Such an operator is usually called a finite difference operator. Notice that if $p$ has degree $d$, then $\Delta p$ has degree at most $d-1$, and continuing in this manner $\Delta^{d+1} p$ must be the zero polynomial.
Now let $f(x)=x^p$ and calculate $\Delta^{p+1} f$. This gives (you should check this) $$\Delta^{p+1} f(x)=\sum_{k=0}^{p+1} (-1)^k\binom{p+1}{k} f(x+k)=0,$$ and now plug in $x=1$ to obtain the result.