$\varepsilon$-$\delta$ definition of a limit:
For every $\varepsilon$ $>$ $0$, there exists $\delta$ $>$ $0$ such that whenever $0$ $<$ $|x-c|$ $<$ $\delta$, we have $|f(x)-L|$ $<$ $\varepsilon$.
My doubt arised from the following source:
Prove, using delta and epsilon, that $\lim_{x \to 5} ( 3x^{2} - 1 ) = 74$
Feel free to get the bigger picture: refer to "Examples using a Non-Linear Function", highlighted in blue. Note that the picture above it's only part of the proof, which is defining $\varepsilon$
How To Construct a Delta-Epsilon Proof. (n.d.). Milefoot. Retrieved April 27, 2021, from http://www.milefoot.com/math/calculus/limits/DeltaEpsilonProofs03.htm
According to what I'ved learned, we must prove that $0 < |x - c| < \delta$ $\implies$ $0 \leq |f(x) - L| < \varepsilon $, not $|x - c| < \delta$ $\implies$ $0 \leq|f(x) - L| < \varepsilon $

The given solution makes the assumption that $f$ is continuous at $c$, meaning $L = \lim_{x \to c} f(x) = f(c)$. With this assumption, for $x = c$ we still have $\lvert f(x) - L\rvert < \epsilon$ because $L = f(c)$.
If $f$ is not continuous at $c$, meaning $L \ne f(c)$, then we could have $f(x)$ is close to $f(c)$ for all $x$ close to but not equal $c$, but at exactly $f(c)$ we have a discontinuity.
Here's a picture from Brilliant.org I've modified (where $x_0 = c$) to include a discontinuity at $f(x_0)$, making $f(x_0)$ totally different from its neighboring values and showing the $\epsilon-\delta$ definition clearly doesn't hold.