Home Page Overview Site Map Index Appendix Illustration About Contact Update FAQ


Information, SMI, and Entropy


Contents

Maxwell's Demon
Different Kinds of Information
An Example of Specific Information
Mathematical Formulation of Information and SMI
Order/Disorder
Complexity
SMI and Entropy

Maxwell's Demon

Maxwell's Demon Information is a concept first used to resolve the paradox of Maxwell's demon. About 150 years ago, the physicist J. C. Maxwell came up with an intriguing idea. He conceived a thought experiment, in which a little demon who operates a friction-free trap door to separate molecules of one type from the other (see Figure 01a), and finally arrives at a system with lower entropy. Such organizing entity of Maxwell's seems to violate the second law of thermodynamics as the demon only selects molecules but does no work. This paradox kept physicists in suspense for half a century until Leo Szilard showed that the demon's stunt really isn't free of charge. By selecting a molecule out of the alternative of 2 types, he creates something called information, which produces an amount of entropy (through mental processing in the brain, see Integrated Information Theory of Consciousness) exactly offsetting the decrement in the re-arrangement. The unit of this commodity is the bit, and each time the demon chooses a molecule to shuffle, he shells out one bit of information for this cognitive act, precisely

Figure 01a Maxwell's Demon

balances the thermodynamics accounts. The new concept has since shown its usefulness in communication and computer, but perhaps its greatest power lies in biology, for the organizing entities in living beings - the proteins and certain RNAs - are but Maxwell's demons.

Information *** Information is actually a rather vague concept (Figure 01b) according to Arieh Ben-Naim in "Entropy and the Second Law, Interpretation and Miss-Interpretationsss, 2012" which stresses the special relationship between entropy and Shannon's Measure of Information (SMI). This webpage has been revised to clarify the confusion on the various examples about SMI and information, which is just the negative of SMI. In addition, a section on "SMI and Entropy" has been added to provide further explanation.

Figure 01b Information [view large image]

Go to Next Section
 or to Top of Page to Select
 or to Main Menu

.