What is the relationship between information in the sense of Shannon entropy and information for the human brain?

Question

In an informatic theoretic sense, complete randomness maximizes information. For instance, an image of randomly distributed black and white pixels has a very high entropy/information. For a human brain, such an image does not contain any valuable information. However, if we remove some randomness and align some pixels to represent a specific shape, the human brain can recognize this shape and process/gather information. Thus, for the human brain, maximum information in the sense of Shannon entropy is not valueable.

My question is: What is the relationship between Shannon entropy and information, understandable by the human brain? Why can we not make sense of a channel with maximum information? Why do we need less than maximum information to gather valuable information?

Thanks!

Answer 1

For convenience, I'll call your concept of an "image of randomly distributed black and white pixels" a "white noise picture".

While there is as yet no satisfactory mathematical characterization of the human brain, we can be confident that the brain's functions arose from evolutionary pressures. It is also widely accepted in academic research psychology that the human brain is a computational device. From this point of view, then, there is no difference between Shannon information and the information processed by the brain.

From this point of view, there is no evolutionary value in recognizing white noise pictures or distinguishing one such picture from another, so the brain doesn't have this functionality. This addresses your second question and much of your third question.

There is plainly evolutionary value in recognizing what we all would call familiar shapes. Since these require less information to describe that a white-noise picture, we get closer to your third question.

Answer 2

Let me sidestep the question of how exactly entropy and the human brain are related, and instead say something about the connection between entropy and knowledge, which I think ultimately gets at your question.

An intuition you should have about Shannon entropy is that "channels" (since this is the term information theorists use) which are higher entropy require more knowledge to remember, in principle. For example, if you know that the top half of an image is white and the bottom half is black, you can write down a short formula that can reproduce the picture for you - much shorter, in fact, than the naive way to memorize an image, which is to just write down a giant table that lists the hue of every pixel individually. Consequently the "minimal amount of knowledge" required to know every pixel value is at least as short as the short description we just gave. (Another way to say this is that the image can be compressed in a very efficient way.)

However, for a specific image which was produced by a random noise process, we expect that there will typically not be a short description of the image - we can try all sorts of different "compression schemes" (algorithms which systematically look for shorter descriptions) and they will all be around the same length as just a giant matrix of pixel values. But memorizing an entire image pixel-by-pixel requires storing tens of thousands of pieces of information somewhere! This is way worse than storing a short formula or description or something. Consequently, the knowledge associated with actually knowing a specific random noise image is expected to be quite high.

In sum, entropy <-> minimum amount of knowledge required to memorize a channel (normalized against the length of the channel).

This does not directly address your intuition that indeed, a random noise image does not look like any specific piece of information to most of us. But this is the story almost every information theorist has in their head about Shannon entropy, and so is at least a decent attempt at an answer to your question.

Answer 3

For the human being, a stable random picture of white/black pixels is just a background, quite the same as if it is all white or all black. And quite the same if it is a forest, a sand desert, a meadow, etc.
Except for those recognized as dangerous, and specially if coincidentally with the other senses.

When the image is moving, some attention is called upon, and then relaxed if recognized "familiar": forest trees with wind, sky with clouds, and .. tv with advertising.

It looks that our attention concentrates on the variation in space and time of the entropy: either a black&white flag in the snow, or a white dish on a black&white towel.

That means, we are to distinguish a sudden (relatively long) burst of zeros amidst a casual binary string, or a burst of (high frequency) noise amidst a relatively "stable" signal.