Corollary 2.3 of these notes on Morse theory seem to suggest the "easy" part of the Morse lemma is a corollary of Ehresmann's fibration theorem.
That is, if $f:X\to \mathbb R$ is a proper smooth map without critical values in $[a,a+\varepsilon]$ then there's a diffeomorphism $f^{-1}(-\infty,a]\cong f^{-1}(-\infty,a+\varepsilon]$.
I don't understand how to deduce this from Ehresmann's fibration theorem.
The proof I know constructs a normalized gradient field on $X$ which is supported on a compact neighborhood of $f^{-1}[a,a+\varepsilon]$ and flows along it.
As you say I am not sure the other answer covers the detail you want. If $f$ is proper, then the set of critical values is closed (exercise) so if there are no critical values in $[a, a+\epsilon]$, then nor are there in $[a-\delta, a+\epsilon]$ for $\delta$ small. Now you know from Ehressman that there is a diffeomorphism
$$g: X_{[a-\delta, a+\epsilon]} \cong [a-\delta, a+\epsilon] \times X_a,$$ so that under this diffeomorphism $f(g^{-1}(t,x)) = t$; that is, $f$ is taken to projection onto the first factor. Here I write $X_S = f^{-1}(S)$ as notation I prefer.
We will use this to construct the desired diffeomorphism. To do so, pick a diffeomorphism $$h: [a-\delta, a] \to [a-\delta, a+\epsilon]$$ which is the identity near $a-\delta$. Of course, this induces a diffeomorphism $$H: X_{[a-\delta, a] \to [a-\delta, a+\epsilon]}$$ which is the identity near $X_{a-\delta}$; this induced diffeo comes from the diffeo $g$ above (and its restriction to $X_{[a-\delta, a]}$ as well).
Then the desired diffeomorphism $$F: X_{(-\infty, a]} \to X_{(-\infty, a+\epsilon]}$$ by saying that $F$ is the identity on $X_{(-\infty, a-\delta]}$, and on $X_{[a-\delta, a]}$, $F = H$, the function defined above.
The idea is that we are flowing backwards from $a+\epsilon$ to $a$, but we use $h$ to "slow down the flow" past $a$ so that by $a-\delta$ we have stopped flowing completely.