$\textbf{Theorem:}$ If $f(x)$ is continuous at $L$ and $\lim_{x\to c} g(x)=L$, then $\lim_{x\to c} f(g(x))=f(L)$.
$\textbf{Proof:}$ Assume $f(x)$ is continuous at $L$ and $\lim_{x\to c} g(x)=L$.
As $f(x)$ is continuous at $L$, we know $\lim_{x\to L} f(x)=f(L)$.
(1) Let $\epsilon >0.$ We know $0<|x-L|<\delta_1\implies|f(x)-f(L)|<\epsilon$ for some $\delta_1>0$.
(2) As $\delta_1>0$, we also know $0<|x-c|<\delta_2\implies|g(x)-L|<\delta_1$ for some $\delta_2 >0$ by definition of a limit for $g(x)$.
Note the conclusion for $g(x)$ is restricted to the same values as $x$ is in the hypothesis for $(1)$. This means we can replace $x$ with $g(x)$ for the conclusion of $(1)$. The only exception is when $g(x)=L$ whenever $0<|x-c|<\delta_2$ where $(1)$ holds true vacuously.
I noticed a lot of proofs leave off the scenario where $g(x)=L$. Am I correct in saying it is a vacuous argument here? I see a lot of proofs out there that ignore this part which is why I feel I am doing something wrong because it seems to me like a major part of the proof.
The fact of the continuity of $f$ is given, so to write the "$0<$" part of $0 < |x-L|<\delta_1\dots$ in (1) is an unneeded restriction. You should drop the "$0<$" from (1), which should remove your uneasy feeling. The reason you can drop it is because $f$ is a well-defined function, defined at $L$, and $0<\epsilon$ always.
But in (2) you should leave the "$0<$", since we have no reason to think $c$ is in the domain of $g$.