All the definitions of differentiability I found (Wolfram Mathworld for instance) only require this limit to exist, but say nothing about the domain in which that has to happen.
So what if that limit is $\pm\infty$? Wouldn't (in the "normal" interpretation of a function $\mathbb R \rightarrow \mathbb R$, for instance) the slope of the tangent of $f$ in $x$ be undefined?
Edit: a link clearing it up.
This is an unfortunate case of terminology getting the better of us. The definition of a limit is
So the limit, if it exists is a number. Infinity is not a number, so technically, if $\lim_{x\to a}f(x)=\infty$, the limit doesn't exist. And yet it is infinity. It's confusing at first, but once you get used to it, it's not too bad.