We have the following result: Every infinite subset $E$ of a compact set $K$ contains a limit point in $K$. I understand the proof given in Rudin.
Now consider the following statement: All limit points of a subset $E$ (not necessarily infinite) of a compact set $K$ are contained in $K$. Clearly this has to be false, since it is much stronger than the first proposition, but I don't understand what's wrong with the following reasoning: Since $K$ is compact, it is closed, so it contains all of it's limit points. Since $E$ is contained in $K$, every limit point of $E$ is a limit point of $K$, and is therefore contained in $K$.
There’s nothing wrong with your reasoning: the statement is true, and you don’t in fact need $K$ to be compact: it’s true for any closed set. What isn’t true for arbitrary closed sets is the really significant part of the original theorem, which is not that $K$ contains the limit point in question: it’s that every infinite subset of $K$ actually has a limit point.
Consider the closed set $C=[0,1]\cup\Bbb Z$ in $\Bbb R$: $C$ is closed, so it contains all limit points of all of its subsets, but $\Bbb Z$ is an infinite subset of $C$ that has no limit point at all.