I recently had a logic written exam and there was a question like this:
"What sets are represented by the following formulas?"
First formula: $Theor(Theor(x))$ I gave the correct answer to this one: the set of the Gödel numbers of the theorems in PA. Reasonment: $\vdash Theor(Theor(x))$ iff $\vdash Theor(x)$ iff $\vdash x$ so $\vdash Theor(Theor(x))$ iff $\vdash x$.
Second formula: $\neg Theor(x)$. My answer was: The complement of the set of the Gödel numbers of the theorems in PA. That's because in my understanding, at least in the standard model of the natural numbers, that formula is true for all the Gödel numbers of non-theorems. But the professor said that this is the wrong answer. Then he hinted at the fact that, as a corollary of Gödel's second theorem, $\neg Theor(x)$ is never a theorem of PA.
So I tried to come up with another possible answer, and in doing this I noticed that I'm probably missing something about what "a formula representing a set" means. $\neg Theor(x)$, in the non standard models of PA, is not satisfacted by the same numbers of the standard model. If the definition of "a formula representing a set" is that the formula must be satisfied by all the numbers of that set in every model of PA, and the set must be the maximum one with this property, then $\neg Theor(x)$ probably represents the empty set. Is this definition right? Is my reasonment right?
You are on the right track: PA doesn't prove "$\varphi(n)$ isn't provable" for any $n$ at all. Why is this? Note the following:
If a theory $T$ is inconsistent, then it proves everything.
PA proves the above bulletpoint.
In particular, PA proves "If PA is inconsistent then PA proves everything."
By Godel's second incompleteness theorem, PA doesn't prove Con(PA) ...
... so PA+$\neg$Con(PA) is consistent.
Alright, now put the two bulletpoints (and the completeness theorem) together, and think about what a model of PA+$\neg$Con(PA) thinks about $\neg Theor(n)$ for a given $n\in\mathbb{N}$ ...
As a coda, here are a couple quick and hopefully useful comments:
First, the use of the completeness theorem above is only needed since you've defined representation semantically. If we define "$\varphi(x)$ represents $A$" as "$A$ is the set of natural numbers which PA proves satisfy $\varphi$," then there's no need to talk about models at all. And this more formalist perspective is often quite useful, if only for building correct intuition.
Second, this exercise demonstrates the fundamental difference between provability in a theory and truth in a model. Given a formula $\varphi(x)$ and a structure $\mathcal{A}$, we can talk about the set $$\varphi^\mathcal{A}:=\{a\in\mathcal{A}: \mathcal{A}\models\varphi(a)\}.$$ Of course we always have $$\varphi^\mathcal{A}=\mathcal{A}\setminus\neg\varphi^\mathcal{A},$$ and in particular in the standard model $\mathcal{N}$ of PA we have $\neg Theor(x)^\mathcal{N}$ is the set of Godel numbers of non-theorems of PA. The discrepancy between $\neg Theor(x)^\mathcal{N}$ and the set represented by the formula $\neg Theor(x)$ is explained by noting that the first set is model-specific while the second only gets to "see" the theory PA.
Third, I actually made a small-but-important mistake above when I wrote "$\neg Theor(n)$" where $n$ is a natural number. That's a category error: sentences have to use symbols, and the literal natural number $n$ isn't a symbol of the language of PA. Instead, I should have written "$\neg Theor(\ulcorner n\urcorner)$," where "$\ulcorner n\urcorner$" is the standard numeral corresponding to $n$ for $n\in\mathbb{N}$ (e.g. $\ulcorner 2\urcorner=S(S(0))$).
Finally, a notational issue. It rapidly becomes important to talk about different theories in the same sentence, and so to distinguish between their different notions of provability/representability/etc. So a better notation would be "$Theor_{PA}(x)$," and a better terminology would be "PA-represents," etc. This may seem like a minor point here, but down the road explicitly mentioning the theory in question will be important.