What is going on when we "add constant symbols" to extend a language L.
A new constant is not in the "alphabet" of L. C is, for example, just { 0, 1 } in many cases, but C could be an infinite set. Constants are distinguished elements of the universe. But the universe is associated with the models, $\mathcal{M}$.
So, frequently we add new elements to C, or rather think of the signature as $\sigma = (C \cup C_0, F, R, \sigma')$ where $C_0$ is a set of new constants.
Often we add a whole bunch of constants corresponding to a set A in some particular L-structure $\mathcal{M}$. So $A \subseteq M$ (M is the universe of $\mathcal{M}$) So, the elements in A correspond to elements in the universe.
$L(\underline{M})$ is L with extra constants for every element in the universe of $\mathcal{M}$. So, a bigger language.
Why are we doing this? We are "considering formulas as sentences in an extended language." How will the constants help?
One reason is to show that there are models of $L$ with some property. If we add constant symbols to $L$, we can also add axioms involving those symbols. Any model of the expanded structure is still a model of $L$, so if we can show that the new symbols and axioms do not cause a contradiction we have shown that there exist models of $L$ with the desired property.
A classic example is to show Peano Arithmetic has models with nonstandard elements. We know (or assume) the usual PA axioms are consistent. We now add a distinguished element $c$ along with a countably infinite set of axioms $c \gt 0, c \gt S(0), c \gt SS(0), \dots$ For any finite set of the axioms, the usual $\Bbb N$ is still a model, because there are only finitely many of the $c \gt $ and we can just take $c$ to be a large enough standard number. The compactness theorem now tells us that the whole set has to have a model, but we cannot have $c$ being any of the standard numbers, so it must be nonstandard.