Information, SMI, and Entropy

Different Kinds of Information

Specific information is defined by the content in a given sequence of units. It could be words in a sentence, or nucleotides in DNA, ... etc. The amount depends on the minimum number of instructions needed to specify or describe the structure. Many instructions are needed to specify a complex, information-bearing structure such as DNA. Only a few instructions are need to specify an orderly structure such as a crystal. In this case we have a description of the initial sequence or unit arrangement which is then repeated ad infinitum according to the packing instructions (see Figure 02). Such system contains no information.

Figure 02 Crystal [view large image]

A system would be most probably in an equilibrium state when leaves to its own devices as shown in Figure 03. Such state would also contain no information. In static configuration, it is the un-organized arrangement of books in library, for example. In thermodynamics, it is the distribution of particles in the most probable state. In DNA, it is the random arrangement of the nucleotides. In atom/molecule, it is the ground state (lowest energy level) for the electrons/atoms.

Figure 03 Equilibrium [view large image]

Pre-requisites for Specific Information:


• There should be rules or processes to arrange the system in a non-equilibrium state. The rules themselves are a form of stored information. They are similar to mathematical formulae but not as compact, more like a set of algorithm (a logical sequence of steps). Just like mathematical formulae, a small number of fixed rules can generate an un-predictable amount of complexity - resembling the endless varieties of movement described by the Newton's equation of motion.
  • In static configuration, it is the rules of cataloging in the library. External energy is required for the arrangement.
  • In thermodynamics, it is the external energy that drives the system to a non-equilibrium state. The rule is prescribed by thermodynamics.
  • In DNA, it requires energy to arrange the codons (sequence of three nucleotides) for the production of a particular protein. The rule is provided by the genetic code.
  • In atom/molecule, it is again the external energy that excites the electron(s)/atom(s) to higher energy level. Quantum theory is the rule in the atomic/molecular domain.
• There should be a number of available states.
  • In static configuration, it is the space on the book shelf.
  • In thermodynamics, it is the distribution of particles in phase space (composed of position and velocity).
  • In DNA, it is the position in the helix.
  • In atom/molecule, it is the higher energy levels for the electrons or atoms.
• There is a mean (a way) to execute the rule/process.
  • In static configuration, a librarian is required to complete the task.
  • In thermodynamics, an electrical motor will usually do the trick.
  • In DNA, it relies on natural selection to acquire the viable codons.
  • In atom/molecule, it requires photon energy to excite the atom/molecule to higher energy level.

  The resulting state is considered to be informational / useful, if it provides a way to locate the books in the static configuration example. it can produce refrigeration in a box, e.g, in the thermodynamics example. the gene benefits the living organism. the atoms can be excited to metastable state for the production of laser. Figure 04 shows the non-equilibrium states for the same examples as illustrated in Figure 03. The "specific

  Figure 04 Non-equilibrium [view large image]

  information" defined in the above context is not universally accepted because of its subjective connotations - many situations can be met only by an intelligent being or a living organism; and "usefulness" to some may be garbage to others.

  
  

  There are other kinds of information which do not follow the rules prescribed above. Figure 05 shows an example which is one of the various definitions of "information". It takes into account only the variations that is possible in the message. Thus the grammatical rules are more or less taken out of the equation. The image on the left portrays Bob as a dummy who can only utter Ba Ba Ba Ba to Alice. There is no variation in the message resulting in no "information content". On the other hand, the

  Figure 05 Information Content [view large image]

  picture on the right shows Bob to possess a wonderful command of vocabulary. Alice is surprise to hear the actual message out of so many possibilities. Therefore, this definition uses the "average amount of surprise" as its criterion. It doesn't take into account the "meaning" of the message.

  
  The series of lottery winning numbers provides one more variety, according to which the odds of winning in Lotto 6/49 is one in 13,983,816 or in term of one definition of information, it is equal to -log₂(13983816) -24 bits (see formula in "Mathematical Definition of Information").
  
  See one more definition of information as S-matrix elements in "Black Hole Information Paradox".
  

  Go to Next Section
   or to Top of Page to Select
   or to Main Menu

  .