According to the epsilon-delta definition of a limit, L is limit of f(x) at c if For every arbitrary ε>0 we can find δ>0 such that for "all" x satisfying c-δ<x<c+δ (except c possibly), f(x) lies in (L-ε,L+ε)
does this definition imply that f(x) has to be defined at "all" points in the interval (c-δ,c+δ)?
Yes and no. Any function $f:A\to B$ of course has to be defined on all of $A$, otherwise it is not a function. However, you may defined $f:[0,1] \to \mathbb{R}$ and still pick $c=0$ and, say, $\delta = 2$. This does not mean that $f$ must be defined on the interval $(-2,0)$, however: The definition says more precisely that for $x\in A$ satisfying $c-\delta<x<c+\delta$, $f(x)$ has to be $\epsilon$-close to $f(c)$. With the quantifier $x\in A$, we do not run into any problems.