According to the epsilon-delta definition of a limit, don’t we have to be able to say that if the value of x is within delta of the approaching value, then the value of of the expression must be within some epsilon of the limit?
How does this work for limits that approach infinity since there would be no finite value of delta?
The definition at infinity is different, notably we say that
$$\lim_{x\to x_0 }f(x)=\infty$$
when
$$\forall M\in \mathbb{R} \quad \exists \delta>0 \quad \forall x \quad |x-x_0|<\delta \quad f(x)>M$$
and similar for $-\infty$.
Note that if $x\to \infty$ the definition is
$$\forall M\in \mathbb{R} \quad \exists \delta>0 \quad \forall x >\delta \quad f(x)>M$$
and similar for $x\to -\infty$.