In the wikipedia article about the inverse limit it is stated that for a prime number $p$ $$\varprojlim_{n \in \mathbb{N}} \mathbb{Z}/p^n\mathbb{Z} = \mathbb{R}/\mathbb{Z},$$ where the arrow between the groups $\mathbb{Z}/p^n\mathbb{Z} \longrightarrow \mathbb{Z}/p^{n+1}\mathbb{Z}$ is given by multiplication by $p$.
I don't quite understand that: I interpret this "multiplication by $p$" as $[x]_{p^n} \mapsto [px]_{p^{n+1}}.$ But shouldn't the arrows point in the other direction from $\mathbb{Z}/p^{n+1}\mathbb{Z}$ to $\mathbb{Z}/p^n\mathbb{Z}$ for the inverse limit to make sense? On the other hand if one interprets it as a direct limit, then the arrows become inclusions and the limit is simply the group of all roots of unity of order $p^n$ for every $n \in \mathbb{N}$. I also find it a bit odd that one can get a "continous" object from "discrete" objects without invoking any topological notions at all.
How can this be reconciled, is this direct limit correct?
This is false (I'll correct it on Wikipedia). The correct statement is that the direct limit / filtered colimit of $\mathbb{Z}/p^n \mathbb{Z}$ with these maps is the Prufer $p$-group $\mathbb{Z} \left[ \frac{1}{p} \right]/\mathbb{Z}$, which is the subgroup $\mu_{p^{\infty}}$ of $\mathbb{R}/\mathbb{Z}$ consisting of $p$-power roots of unity. The inverse limit / cofiltered limit (with a different set of maps going the other way, namely the quotient maps $\mathbb{Z}/p^{n+1}\mathbb{Z} \to \mathbb{Z}/p^n\mathbb{Z}$) is the $p$-adic integers $\mathbb{Z}_p$.
There's a reason people don't take inverse limits over diagrams that are "increasing to the right" like this diagram is:
$$\mathbb{Z}/p^0\mathbb{Z} \xrightarrow{p} \mathbb{Z}/p\mathbb{Z} \xrightarrow{p} \mathbb{Z}/p^2\mathbb{Z} \xrightarrow{p} \dots$$
and it's that the inverse limit over any such diagram is its first term.
The solenoid example is also wrong (and I will also correct it). $\mathbb{R}/p^n \mathbb{Z}$ is not a ring because $p^n \mathbb{Z}$ is not an ideal. You can in fact prove that every compact Hausdorff ring must be totally disconnected, or equivalently must be profinite, so rings like the $p$-adic integers are all you get.