Consider the quantity:
$$\frac{(N+p-1)!}{p!(p-1)!N!}$$ where $N$ and $p$ are positive integers. How can we show that this is always an integer (which I believe has to be the case since it represents the number of ways $N$ identical particles can be put into $p$ identical boxes, I think).
You've miscounted.
I see that you are essentially taking the number of solutions to $x_1+x_2+\cdots+x_p=N$ and dividing by $p!$. But if $x_i=x_j$ then not all permutations give a different solution.
The number of solutions to $x_1+x_2+\cdots +x_p=N$ with $x_1\leq x_2\leq \cdots \leq x_p$ is a much more complicated value.
These are called "integer partitions" of $N$ into at most $p$ parts, and there id no simple formula for counting these. See Wikipedia.