We want to minimize the mean square error in a regression model that uses just one paremeter $w_0$. For convinience we take the mean square error to be $MSE(w_0) = \frac{1}{2N} \sum_{i=1}^{N} (y_n - w_0)^2$ so that gradient descent method takes the form $w^{t+1} = (1-\gamma) w^{t} + \frac{\gamma}{N} \sum_{n = 1}^{N} y_n$.
In the case $\gamma \in ]0,1]$ this sequence clearly converges using the fact that it is a convex combination of points and that therefore we get a monotone and bounded sequence.
What happens if $\gamma \ni ]0,1]$?