There are a lot of well-known examples for "continuous at point but not continuous on a neighbourhood", "differentiable at a point but not continous on a neighbourhood" (of course then not differentiable or not continuously differentiable).
Is it correct that for a function having a continuous derivative at a point, it must have a continuous derivative on a neighborhood around that point? I can't think of any counter-examples.
No. Considering $n=1$. The set of points of continuity of the derivative of a differentiable function must be nonempty, but the set of points of discontinuity can be dense. So if you take any function that is differentiable and whose derivative is discontinuous on a dense set, any of the points of continuity of the derivative will provide a counterexample. For much more info and references, see this answer of Dave Renfro: https://math.stackexchange.com/a/112133/
For a concrete example
$$\sum\limits_{n=1}^\infty \frac1{n^2} \left(x-\frac1n\right)^2\sin\left(\frac1{x-\frac1n}\right)$$ should be a counterexample at $0$. See also Show that function is differentiable but its derivative is discontinuous.