We all know that Godel first incompleteness theorem states that any recursive sufficiently strong theory to express arithmetic is incomplete. in particular, arithmetic is not complete.
It's common for authores to comment that adding a Godelian sentence will not solve the problem since we cal carry the construction again in this expanded theory to get a new Godelian sentence.
What if we started with the "standared" arithmetic $A_1$ and get a godelean sentence $\phi_1$ and let $A_2 = A_1 \cup \{\phi_1\}$ and got a new godelean sentence $\phi_2$ and expanded $A_2$ by adjoining $\phi_2$ to it to get $A_3$ etc ...
The point is that we will not make this process only for finite numbers. But we will use a transfinite sequence of $A$'s. For example $A_{\omega}=\bigcup_{i\in \mathbb{N}} A_i$
In general, for any ordinal $\alpha$,
$A_{\alpha+1}=A_{\alpha}\cup{\phi_{\alpha}}$ if $\alpha$ is a successor ordinal and $A_\alpha= \bigcup_{\beta < \alpha} A_{\beta}$ if $\alpha $ is a limit ordinal
let $A=\bigcup_{\beta \text{ is ordinal}} A_{\beta}$, Will A not contain every possible godelian sentence? if not, we take this $A$ to be $A_1$ and start the process all over again!
Eventually, every true sentence of arithmetic will be included in one item of this sequence and so we've completed arithmetic!
I know that an objection may raise that expanding the theory in such a way could yield a non-recursive theory. But adding a formula per time will not turn a recursive set into a non-recursive one. Only taking a infinite union could yield problems for recursivness , So can we show that such a union does in fact yield a problem for recursivness?
Can we understand Godel first incompleteness theorem as the assertion that such transfinite union does in fact yield problems?
We do indeed run into problems.
$\varphi_\alpha$ only makes sense if $\alpha$ is a computable ordinal (actually, even this isn't quite true - we need to talk about notations for ordinals instead of ordinals themselves - but this is enough detail to see the problem). There are only countably many such ordinals, so there is an upper bound to the theories we can access this way. This upper bound is a non-recursive (in fact, non-definable - the set of Godel numbers of axioms is not first-order definable in the language of arithmetic) theory, so we cannot take its Godel sentence.
This type of progression was introduced by Turing in his paper on ordinal logics, and further studied by Shoenfield and others. See also https://mathoverflow.net/questions/171553/transfinitely-extending-sf-pa-can-we-get-stronger-than-sf-zfc and https://mathoverflow.net/questions/67214/pi1-sentence-independent-of-zf-zfconzf-zfconzfconzfconzf-etc?lq=1 (and maybe more).
Note that as mentioned in my answer to the first linked question, each theory we can obtain in this manner is a subtheory of the starting theory + all true $\Pi^0_1$ sentences, and this upper bound is achievable. So in fact we can conclude that the "ultimate" theory, $A_{\omega_1^{CK}}$, is still incomplete, since there are sentences in arithmetic not decidable by any extension of $A$ by true $\Pi^0_1$ sentences (this is true for computability-theoretic reasons: the set of true $\Pi^0_n$-sentences has complexity $0^{(n)}$, while the set of sentences provable from $A$ + true $\Pi^0_1$ sentences has complexity $0^{(2)}$). So even this trick won't get you a complete theory, even if you're willing to go to a non-recursive theory.
As to my comment on notations: if you look at it carefully, you'll notice that even such a simple theory as $A_{\omega+1}$ isn't well-defined! What should it be? More specifically, what should the sentence "$\varphi_\omega$" be?
The ambiguity resides in the fact that there are multiple ways to describe the progression $1, 2, 3, ...$. For instance, $\forall n\varphi_{2n}$ is a perfectly reasonable choice for $\varphi_\omega$ (except for the part where, as written, it's gibberish :P).
To make this precise, we talk about notations for ordinals. The precise definition is a little complex - see https://en.wikipedia.org/wiki/Kleene%27s_O - but basically, we assign to each computable ordinal a bunch of natural numbers which "name" it in a reasonably effective way; these names are the ordinal notations. Now, there's lots of non-recursiveness here - for instance, the set of numbers which are notations for some ordinal is not only non-recursive, it's $\Pi^1_1$ complete! - but for "local" considerations (e.g. talking about one specific $A_\alpha$) it's a very nice system. Unfortunately, notations are extremely variable: given any true $\Pi^0_1$ sentence $\psi$, $\psi$ is provable in one of the versions of $A_{\omega+1}$ (assuming $A$ is something reasonable like $PA$).