Let $\mathcal{M} : = \{ \mu | \mu \text{ is a probability measure} \}$, and let $f:\mathbb{R} \rightarrow \mathbb{R}$ be a measurable function.
$\mu \in \mathcal{M}$ is ergodic if for all $A \in \mathcal{B} $ (the Borel set on $\mathbb{R}$) such that $f^{-1}(A) = A$ either $\mu(A) = 0$ or $\mu(A) = 1$.
Let $f$ have a periodic orbit of period $n \in \mathbb{N} $ with points $\{p_1, \dots, p_n \}$ then $$\mu = \frac{1}{n} ( \delta_{p_1} + \dots + \delta_{p_n} )$$ is easily seen to be ergodic according to the definition given. Even a weighted average of the $ \delta_{p_1} + \dots + \delta_{p_n}$ would be.
Under the article on Ergodicity on wikipedia I find that "A random process is ergodic if its time average is the same as its average over the probability space", if I Imagine the orbit of $f$ as a random process this statement does not seem to agree with the definition since the time average would be
$$\frac{1}{n} ( p_1 + \dots + p_n )$$ and the average over the probability space is $$\mu = \frac{1}{n} ( \delta_{p_1} + \dots + \delta_{p_n} )$$ (?).
Also the statement "a Markov chain is ergodic if there is a positive probability to pass from any state to any other state in one step" seems in contrast with the example give with $f$ having a periodic orbit of period $n$.
In short my question is, how does the given definition of ergodicity reconcile itself with the statements quoted on the Wikipedia page? Simple examples are welcome (or utilizing mine would be great).
For your first question, let us place ourselves in a more general setting. Let $(X, \mathcal{A}, \mu)$ be a probability measure space, $S:X \to X$ be a measure-preserving transformation, i.e., it holds that $$ \mu(S^{-1}(A))=\mu(A), \quad \forall A \in \mathcal{A}. $$ We say that a set $A \in \mathcal{A}$ is invariant under $S$ if $S^{-1}(A)=A$. The map $S$ is said to be ergodic if every invariant set is such that either $\mu(A)=0$ or $\mu(X \backslash A)=0$, i.e., if all the invariant subsets of $S$ are trivial.
Given an ergodic measure-preserving map, the Birkhoff individual ergodic theorem tells us that if $f \in L^1(X, \mathcal{A},\mu)$, there exists an $f^\ast \in L^1(X, \mathcal{A},\mu)$ such that $$ f^\ast(x)= \lim_{n \to \infty}\frac{1}{n}\sum_{k=0}^{n-1}f(S^k(x)) \quad a.e. x \in X \, . $$ Setting $x=S(x)$ in the above expression we obtain $$f^\ast(x)=f^\ast(S(x)), \quad a.e. x \in X$$ Also since $S$ is measure preserving we have that $$ \int_X f(x) {\rm d}\mu(x) = \int_X f(S(x)) {\rm d}\mu(x)=\int_X f(S^2(x)) {\rm d}\mu(x) =\dots $$ Some careful analysis will then give you that $$ \int_X f^\ast(x){\rm d}\mu(x) =\int_X f{\rm d}\mu(x) $$ Also, since $f^\ast(x)=f^\ast(S(x))$ a.e., and $S$ is ergodic, it follows that $f^\ast$ must be constant a.e.(assume it is not and you will be able to derive a contradiction)
Putting all of this together we have that $$ \lim_{n \to \infty}\frac{1}{n}\sum_{k=0}^{n-1}f(S^k(x)) =f^\ast(x)=\int_Xf(x) {\rm d}\mu(x) \quad a.e. x \in X \, , $$ which tells you that $\mu$-a.e. averages along the trajectory for any observable are the same as averages over the measure space. Going back to your setting just let $X=\mathbb{R}$. (I just realised there is conflict between your $f$ and mine. Your function $f$ is what I have called $S$)