Why does the representation theory of $S_n$ fit perfectly with young tableaux?

Question

This is a "soft question" of sorts. I've been studying the representation theory of the symmetric group, and I continue to be bewildered.

Why does the representation theory of $S_n$ relate to partitions and young tableaux?

It seems completely unmotivated to me that all of this works.

1. Partitions feels like a "number theoretic" object to me, and second only a combinatorial one.

2. The conditions imposed for a partition to be a tableaux (rows weakly increasing, columns strictly increasing) feel arbitrary.

3. The actual point of contact, that of conjugacy class ~ cycle type ~ partition feels reasonable. The part with permutation ~ pair of yonng tableaux (via the Robinson Schensted correspondence) feels unbelievable.

4. The formula:

$$ \sum_{\lambda \in \texttt{partitions}(n)} (\dim\lambda)^2 = n! $$

is one such amazing "coincidence" that works from both the bijection point of view (Robinson Schensted) and the representation theoretic point of view.

Is there some sort of "deep reason" of why it all works out so well? I'd love to understand, because I am currently simply mystified.