Consider an alphabet $\Sigma$ from which I can generate words, which are just concatenations of symbols from the set $\Sigma$. I then define some "rule" that indicates whether or not a word belongs to another set called a formal language.
Does that inclusion/exclusion rule always constitute a formal grammar? If not, is there a formal name for such a rule?
Motivation: Most definitions of formal grammars are explicitly dependent on production rules, which are statements about how to "rewrite" words in a language into other words in the language. This seems far more restrictive than a simple inclusion/exclusion rule—perhaps both of these are equivalent?
Definition by a production grammar and definition by a total Turing machine are equivalent definitions of recursive languages.
There exist nonrecursive languages. The language accepted by a partial Turing machine is such a language. ("Total" and "partial" are used in their usual function theoretic senses. A total Turing machine halts on all finite inputs from the alphabet of the language, so implements a total function. A partial Turing machine need not halt on all such inputs, so implements a partial function.) Any undecidable problem gives an example. The existence of non-recursive languages (that are still recursively enumerable) has been addressed at CS.SE and the answers there are based on undecidable problems.
So, a way to give a rule that is not equivalent to a production grammar is to say that your language is positive (xor negative) problem instances of an undecidable problem. For instance, your language is the collection of all "sets of Diophantine equations having a solution".