Let $f$ and $g$ be continuous an differentiable on $[a,x]$, and let $f(a)=g(a)=0$, and assume that $g'$ and $g$ are $\neq 0$ on $(a,x]$.
Apply the Cauchy-Schwarz Mean Value Theorem to $f$ and $g$ on $[a,x]$. Then, there is a number $\alpha_x$ in $(a,x)$ such that
$$[f(x)-f(a)]g'(\alpha_x)=[g(x)-g(a)]f'(\alpha_x)$$
$$\frac{f(x)}{g(x)}=\frac{f'(\alpha_x)}{g'(\alpha_x)}\tag{1}$$
$\alpha_x$ approaches $a$ as $x$ approaches $a$, ie $\alpha_x-a|<|x-a|$ so $0<|x-a|<\delta \implies 0<|\alpha_x-a|<\delta$.
Now assume that $$\lim_{x\to a} \frac{f'(x)}{g'(x)}$$ exists.
If we take the limit of $(1)$ as $x\to a$, apparently we can write the following
$$\lim\limits_{x\to a}\frac{f(x)}{g(x)}=\lim\limits_{x\to a}\frac{f'(\alpha_x)}{g'(\alpha_x)}=\lim\limits_{x\to a} \frac{f'(x)}{g'(x)}\tag{2}$$
I've thought about this way too many hours and I can't convince myself in terms of the $\epsilon$ and $\delta$ definition of limit that we can justify the second equality in $(2)$.
After all, $\lim\limits_{x\to a}\frac{f'(\alpha_x)}{g'(\alpha_x)}$ means
$$\forall \epsilon>0\ \exists \delta>0\ \forall x, 0<|x-a|<\delta \implies \left | \frac{f'(\alpha_x)}{g'(\alpha_x)}-l \right| <\epsilon$$
Now, this also means that
$$\forall \epsilon>0\ \exists \delta>0\ \forall x, 0<|\alpha_x-a|<|x-a|<\delta \implies \left | \frac{f'(\alpha_x)}{g'(\alpha_x)}-l \right| <\epsilon$$
But even if we write
$$\forall \epsilon>0\ \exists \delta>0\ \forall x, 0<|\alpha_x-a|<\delta \implies \left | \frac{f'(\alpha_x)}{g'(\alpha_x)}-l \right| <\epsilon$$
this still doesn't fit into the cookie-cutter definition of limit. I would expect to be able to write $\forall \alpha_x$, but it wouldn't be true I believe.
I would really like to understand how and why we can say
$$\lim\limits_{x\to a}\frac{f'(\alpha_x)}{g'(\alpha_x)}=\lim\limits_{x\to a} \frac{f'(x)}{g'(x)}$$
By the way, for context on where this comes from, it is part of the proof of L'Hôpital's Rule (as present in Spivak's Calculus)
The second equality in your equation $(2)$ is a special case of the following fact:
In your case is $h(x) = f'(x)/g'(x)$ and $\alpha(x) = \alpha_x$.
Proof: Let $\epsilon > 0$. It is given that the limit $l = \lim_{x\to a} h(x)$ exists, so there is a $\delta > 0 $ such that $$ \tag{$*$} a < x < a + \delta \implies |h(x) - l | < \epsilon \, . $$ The crucial point is that $a < \alpha(x) < x$ for all $x$, so if $(*)$ can be applied to $x$ then it can be applied to $\alpha(x)$ as well. Formally: $$ a < x < a + \delta \implies a < \alpha(x) < a + \delta \underset{(*)}{\implies} |h(\alpha(x)) - l | < \epsilon \, . $$ We have shown that for all $\epsilon > 0$ there is a $\delta > 0$ such that $$ a < x < a + \delta\implies |h(\alpha(x)) - l | < \epsilon $$ and this proves that $\lim_{x\to a} h(\alpha(x)) = l$.
Remark: Slightly more general, we have this “chain rule” for limits: