For the Euler Lagrange equation formulation, they only use a "small" variation $\varepsilon$ for \begin{equation} y(x,\varepsilon) = y(x) + \varepsilon\eta(x) \end{equation} where $y(x)$ is our extremum curve.
My question is: why only "small" variation?
My guess is that they consider "small" variation because they don't want to run the risk of confusion of what extremum we are considering(like having 2 extremum for our function). Like we choose a small enough $\varepsilon$ such that our extremum is a local extremum.
OR, that somewhere in the formulation uses a theorem that requires us to only work with a neighbourhood for some reason.
Both! Euler-Lagrange is merely a generalization of the simple idea that if $f \in C^1(\mathbb{R};\mathbb{R})$ has a local extremum at some $y^* \in \mathbb{R}$, then $f'(y^*) = 0$. When you prove this simple that, you consider small variations $y_\varepsilon = y^* + \varepsilon \eta$ with say $0 < \varepsilon \ll 1$ and $\eta \in [-1,1]$. You then write, that, if $y^*$ is a local maximum (say), then $$ f(y_\varepsilon) \leq f(y^*). $$ So here you use that $\varepsilon$ is small to use the local assumption on $f$. And then you use the smallness of $\varepsilon$ a second time to write $$ f(y_\varepsilon) = f(y^*) + \varepsilon \eta f'(y^*) + O(\varepsilon^2) $$ using Taylor's formula. So you use the smallness of $\varepsilon$ in the two possible ways that you just described.