Everyday entropy: information systems


Information systems only work if the possible configurations are known by both the transmitter and receiver.  If you don’t speak English, it isn’t an information system for you.  If your computer doesn’t speak java or C, it might as well be a brick.  So the Shannon measure of information isn’t a measure of uncertainty or ignorance in the abstract, it is a measure of the pre-determined set of valid configurations that have not yet been determined.  A system that is infinitely variable is not an information system.  In a physical system, all possible configurations are valid and the uncertainty is infinitesimal.

Any physical system can be transformed into an information system.  The first step is to determine which configurations are not valid.  Only by reducing the number of valid configurations to a set that is grossly distinguishable and small enough to be remembered can you hope to have an open communication channel.  Then you have to reduce the entropy within some part of the system to the point where there is a gross or macroscopic differential whose behaviour is predictable so that the transmitter can choose one channel or the other and the receiver can observe the difference.  This is the moment where Shannon entropy meets physical (Gibbs) entropy, but they stare at one another from across a divide between different spacetimes.

The physical (Gibbs) entropy measures the uncertainty of a system’s configuration, considering all possible configurations, while information (Shannon) entropy measures the undetermined configuration of a system, considering only the pre-determined set of valid configurations.  Note that Shannon information applies physically to photons transmitted between bodies, where the valid configurations are limited by Planck’s formula for quantum blackbody radiation.  When you look at the atomic transmission and absorption spectra, you see why some bodies cannot be configured as information systems because they do not resonate with the same wavelengths.

It is wrong to say that we live in a post-factual world.  What we know is that facts are limited to the available information.  The information is ultimately limited physically by Heisenberg’s quantum uncertainty and conceptually by Goedel’s incompleteness theorem.  This is because the smallest unit of information is the oscillation of a single particle, whose position and momentum are subject to uncertainty, and no information system can completely define itself.  Ultimately, an information systems is bounded by the uncertainty of its physical state and the incompleteness of its definitions.  To the best of our knowledge, facts are real, but our information about them is uncertain and incomplete.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s