I have the following implicit equation
$$ x= f(x) $$
which I solve by starting with some value for $x$, then setting $x$ to the new value $f(x)$ and so forth until convergence.
How is that method called? What are its convergence properties, what are caveats one should be aware of, especially wrt the convexity of $f$? In my case $f$ is infinitely derivable.
It is called the fixed point method. There is a whole theory, called Fixed Point Theory, part of an nonlinear analysis, but it's results are applied in many other fields such as numerical analysis (your question is about that I suppose).
The most known result about this theory is called Banach's contraction principle, and states: In a complete space, for a contractive mapping, there exist a unique fixed point.
So your method is convergent if the space you're working in is complete and if the function is contractive.