Can someone provide an alternative explanation to why $e$ can be thought of as an infinite summation of the likelihood of picking any one arrangement out of $n$ arrangements?
$$\frac{1}{0!} + \frac{1}{1!} + \frac{1}{2!} + ... + \frac{1}{n!}$$
Though I am not averse to axiomatic arguments, I am seeking a "wordier" explanation or analogy; why should $e$ somehow be related to the idea of "the (summed) probability of choosing a particular arrangement of $n$ arrangements, as $n$ approaches infinity."
To answer the question I suggested in the comments(!), here are a few probabilistic interpretations for $e$ and $e^{-1}$.
The two canonical probabilistic interpretations for $e^{-1}$ that I tend to see are the question of 'meeting expectations' and the derangements problem. The meeting expectations question is simple:
Since the probability of success is $1/n$, the probability of failure is $(1-1/n)$, and since the tries are independent the probability they all fail is $(1-1/n)^n$; as $n$ gets larger and larger, then, this approaches $e^{-1}$.
The derangements problem is similar but subtly different:
Here it can be shown by various means (for instance, inclusion-exclusion) that this probability is $1-1/(1!)+1/(2!)-1/(3!)+1/(4!)+\ldots+(-1)^{n}/(n!)$, and so once again the limit as we approach infinitely many hats is $e^{-1}$.
While $e$ itself can't be interpreted as a probability per se (since it's larger than 1), it can be interpreted as a stopping time or expected value for a process. For instance, suppose that we follow this process:
Then we can ask about how many cards the deck will have in it when we stop, on average. Now, the probability that we stop after $n$ steps is the probability that we've succeeded on all the previous steps ($1\cdot\frac12\cdot\frac13\ldots\cdot\frac1{n-1} = \frac1{(n-1)!}$) times the probability that we fail this time ($\frac{n-1}n$), or in other words $\frac1{n\cdot(n-2)!}$. (This requires $n\geq 2$, but it's clear to see that the probability we stop after the first shuffle is zero.) Since the number of cards in the deck then is $n$, we can compute the expectation of the number by multiplying that count by the stopping probability, and adding over all of the possible choices: $\sum_{n\geq 2}n\cdot\frac1{n\cdot(n-2)!}$ $=\sum_{n\geq 2}\frac1{(n-2)!}$ $=\sum_{m\geq 0}\frac1{m!}$ $=e$.