When you say "$S$ is a positive sentence (of first order logic)," it means $S$ contains only the symbols { ∧ ∨ ∃ ∀ }, but not ¬. My understanding is that the set of relation symbols in question are { ¬ ∧ ∨ ∃ ∀ }, and "positive" means "can't use ¬".
But what does it mean for a set Γ of formulas to be positive in some arbitrary set $Q$? I found this wording here (pdf) stating:
PROPOSITION 1. If a set Γ of formulas is positive in all the relation symbols in a set Q, then Γ is increasing in Q.
My best understanding is that it means
- Every formula in Γ is positive
- with "positive" meaning it contains only symbols in $Q$ except ¬.
I'm probably misunderstanding here, but what does "positive" mean if $Q$ doesn't include ¬ in the first place? For example, when $Q$ is { = } for some language about equality -- this $Q$ is a set of relation symbols from which ¬ is absent.
The symbols $\forall, \exists, \land, \lor, \lnot$ are not relation symbols: they are logical connectives. Relation symbols are the things like $\leq$ or $P$ that come in a language / signature and are used to form atomic formulas.
So your guess doesn't make any sense: $Q$ is a set of relation symbols, and $\lnot$ is not a relation symbol, so it could not be in $Q$.
Instead, we say that a formula is positive in a relation $R$ if every instance of $R$ appears within the scope of an even number of negations. For example, $R(y,y)\lor \lnot \exists x\, (P(x)\land \lnot R(x,y))$ is positive in $R$ but not positive in $P$. Equivalently, if you "push all the negations inside", then no instance of $R$ appears negated. For example, the formula above is equivalent to $R(y,y)\lor \forall x\,(\lnot P(x)\lor R(x,y))$.
Now we say a set of formulas $\Gamma$ is positive in a set of relation symbols $Q$ if every formula in $\Gamma$ is positive in every relation symbol in $Q$.