In short, according to Gödel's incompleteness theorems, if a formal mathematical system is consistent, then
- it will contain true statements that can neither be proven nor disproven
- it will be unable able to prove its own consistency
However, can Bayesian reasoning be used to address the challenges posed by Gödel's incompleteness theorems in such systems?
For instance...
As more and more time goes by and more and more mathematicians fail to discover a contradiction in a given system $S$, can we use Bayes' theorem to update our posterior distribution of belief and increase our confidence that $S$ is indeed consistent? Intuitively, I suspect yes, but the space of possible theorems in $S$ may be infinite and the set of all possible theorems proven up to any point in time will always be finite. Hence, the ratio of "non-contradictory theorems" to "all possible theorems" will be $0$, which does little to inspire my confidence.
Or, as more and more mathematicians fail to prove some conjecture $P$, can we use Bayes' theorem to update our posterior distribution of belief and increase our confidence that $P$ is a Gödel sentence? Again, intuitively I suspect yes, but it seems impossible to distinguish between a Gödel sentence and a provable theorem that is really, really, really hard to prove.