Is there an example of $f: [0,1] \to [0,1]$ s.t. $|f(x)-f(y)|<|x-y|$
but a sequence $x_0,f(x_0),f^2(x_0)...$ doesn't converge to its fixed point?
where $f^n$ denotes repeated application.
Also, if possible, can you also explain the situation in a general compact metric space?
Partial work: I know there is a unique fixed point, and that the distance to it strictly decreases, and that there will be a convergent subsequence. But I don't think this precludes a counterexample.
Let the fixed point be $P$ and let $L$ be some point such that $L \not= f(L)$. Choose $\epsilon < (|L-P| - |f(L)-P|)/2$. For any $x$ such that $|x-L| < \epsilon$ and any $y$ such that $|y - f(L)| < \epsilon$, we have
\begin{align} |y - P| & = |y - f(L) + f(L) - P| \\ & \leq |y - f(L)| + |f(L) - P| \\ & < \epsilon + |f(L) - P| \\ & < |L - P| - \epsilon \\ & < |L - P| - |L - x| \\ & \leq |x - P| \end{align}
In particular, since $|f^n(x_0) - P|$ is decreasing, $|f^n(x_0) - L|$ is eventually always greater than $\epsilon$. So the only possible limit points of $f^n(x_0)$ are fixed points of $f$, and there is only one of those. In general, a sequence in a compact space with only one limit point has a limit.