I don't understand when we're allowed to use what label when we're writing a proof in first order logic.
During $\forall$-intro we introduce a new variable $c$ and then close out with something in terms of $x$:
Given arbitrary c from the domain: [potentially problematic line]
...
P(c)
∀x P(x) [potentially problematic line]
And during $\exists$-elim we introduce a new variable $c$:
∃x P(x)
Given c from the domain such that P(c): [potentially problematic line]
...
Q [some line that has no dependency on c so we can pull it out]
Q
But with $\forall$-elim we're also introducing a new label $c$:
∀x P(x)
P(c) [potentially problematic line]
And with $\exists$-intro we're potentially introducing a label $x$:
P(c)
∃x P(x) [potentially problematic line]
What I don't understand are when we're allowed to use what label. In each case, what must I "watch out for"? I can't tell when something needs to be a completely new label, versus when I can re-use a label again. Do I have to check for conflicts with things inside the current and any super-contexts? Or just the headers? Both? I just don't understand at all.
Four rules: we will start from the simple ones.
Universal instantiation (or : elimination) rule formalizes the intuitive principle :
In other terms, to say that $\forall x P(x)$ holds, means to say that every element in the "universe" of the interpretation will have property $P$.
Thus, let $t$ denotes an object whatever: we may safely assert that $t$ has property $P$, i.e. that $P(t)$ holds as well.
Thus :
Note : as usual, the language of FOL is made of : predicate symbols: $P_i$; function symbols: $f_i$; constant symbols: $c_i$; individual variables: $x_0, x_1,\ldots$ (countably many); connectives and quantifiers.
A term is : either a constant $c_i$ or a variable $x_i$, or an expression $f_i (t_1,\ldots, t_n)$, where $t_1,\ldots, t_n$ are terms.
Regarding Universal generalization (or : introduction) :
we have to consider that the rule holds with a proviso : "provided that $x$ does not occur free in $\Gamma$".
(Obviously, when $\Gamma = \emptyset$, there is no $x$ free in it, and thus the rule becomes : $\text { if } \vdash \varphi, \text { then } \vdash \forall x \varphi$.)
What is the meaning of the proviso ?
Its role is to formalize the intuitive rule that if we can prove that "property" $P$ holds for an arbitrary element of the "universe", without using any information about it, then $P(x)$ must be true regardless of what element $x$ is, and this means that $∀x P(x)$ must be true.
Compare now with Existential introduction :
The rule has no proviso on $t$.
Obviously, it is enough that $P$ is true of some (at least one) element $t$ in the universe, to license us to assert that $\exists x P(x)$ is true.
And we are left with Existential elimination:
Also this rule has the proviso that the term $t$ to be used must be "new", in the sense of having no other occurrences in the proof, except for $\varphi[x/t]$.
Again, the intuition is not very difficult. How can we deduce something from the assumption that "There is a $P$" ? We proceed by writing "Let $t$ be a $P$" (a new temporary assumption) and then we use it in order to derive further statements.
If in the proof we have used no additional properties about $t$, except for the fact that $t$ is a $P$, and the conclusion that we have reached does not contain $t$, we could safely discharge the temporary assumption and assert that the conclusion depends only on the fact that there is a $P$.
Note : rules can be expressed either using free variables and terms (see van Dalen, page 91 and Chiswell & Hodges, page 177) or, following Gentzen's original presentation, with individual parameters : $a,b,c$ (see Natural Deduction : Rules for Quantifiers), that are always used free, while symbols $x,y,z$ are used for bound variables.