I couldn't decide whether this question is more appropriate to post here or at the philosophy SE, but I thought I'd give people with a mathematical perspective the opportunity to help me decide. I'm happy to migrate the question if people think it necessary.
Anyway, the question is about a famous quote in von Neumann's The Computer and the Brain (published posthumously), that goes:
When we talk mathematics, we may be discussing a secondary language, built on the primary language truly used by the central nervous system.
Also
Thus logics and mathematics in the central nervous system, when viewed as languages, must structurally be essentially different from those languages to which our common experience refers.
I've read the relevant passages a few times, and far as I can see he is trying to convey a statement along the lines of: "The mathematical or logical langauge of the nervous system is not the same as that we use when we do/talk about mathematics."
I can't decide whether the quote is vacuous or profound. From a modern perspective, we may say that the nervous system is a biological neural network, with (as von Neumann himself suggested in this book) low precision but high reliability. And then, presumably, this neural net is Turing-complete, and the actual way in which computations are performed is not important - they are still computations, and they could just as well have been run on some sort of a cellular automaton.
So my question is: are there any concrete consequences of von Neumann's observation within the realm of metamathematics? For instance, is there some theorem that becomes relevant? If the answer is 'no, this is mathematically irrelevant' then I suppose this is a philosophical question, in which case I'd be interested in implications for the philosophy of mathematics.
Level of my question: I am somewhat familiar with various notions that may be relevant, like first and second order logic, the Entscheidung problem, Gödel's theorems etc, but I was trained as a physicist, not a mathematician.
TLDR: was von Neumann onto something fundamental here about the nature of mathematics as a language , or is it merely a subtlety around the physical instantiation of systems that compute, and their architecture?
I would say it's somewhere in between what you say at the end:
Presumably von Neumann viewed the 'language of the brain', i.e. the way that the brain represents information, and the processes defined over those representations in order to 'infer' or 'compute', as significantly different than the 'language of humans', i.e. the languages that mathematicians, physicists and, for that matter, all of us using natural language are using. Think, for example, of the way artifical neural networks represent and process information versus more traditional symbolic languages and processes.
As such, characterizing the difference as 'mere' differences in physical implementation I think falls a bit short of the point that Von Neumann was trying to make: in Von Neumann's eyes we would find nothing like the kinds of algorithms that mathematicians typically employ in the brain, even if we do go beyond implementational detail: the brain's computations presumably are really quite different in nature. When we catch a ball, for example, we shouldn't think that our brains are solving that problem by describing it using certain kinds of Newtonian mechanical/dynamical equations and solving those using the typical methods we as mathematicians employ.
But yes, at the same time, Von Neumann would probably agree that whatever the brain is doing is still within the realm of Turing-computability. It's just that this realm is so vast, that trying to draw any parallels between the 'language of the brain' and the 'language of mathematicians' is pretty futile.