I noticed that when defining gradient descent as taking only the last iteration, the convergence proofs assume that $f$ is convex and $\nabla f$ is Lipschitz continuous, while when defining gradient descent as taking the average of the iterations, only $f$ being convex, differentiable and Lipschitz continuous is assumed.
Is there some Lipschitz, convex, differentiable function $f$ whose gradient is not Lipschitz, for which the first version of gradient descent does not converge? I haven't been able to think of a counter example, or even of a Lipschitz continuous function whose gradient is not Lipschitz.