quotes | Chris Aldrich

Information Theory is the New Central Discipline

Replied to Information Theory is the new central discipline. by Nassim Nicholas Taleb (facebook.com)

INFORMATION THEORY is the new central discipline. This graph was from 20y ago in the seminal book Cover and Thomas, as the field was starting to be defined. Now Information Theory has been expanded to swallow even more fields.

Born in, of all disciplines, Electrical Engineering, the field has progressively infiltrating probability theory, computer science, statistical physics, data science, gambling theory, ruin problems, complexity, even how one deals with knowledge, epistemology. It defines noise/signal, order/disorder, etc. It studies cellular automata. You can use it in theology (FREE WILL & algorithmic complexity). As I said, it is the MOTHER discipline.

I am certain much of Medicine will naturally grow to be a subset of it, both operationally, and in studying how the human body works: the latter is an information machine. Same with linguistics. Same with political “science”, same with… everything.

I am saying this because I figured out what the long 5th volume of the INCERTO will be. Cannot say now with any precision but it has to do with a variant of entropy as the core natural generator of Antifragility.

[Revised to explain that it is not *replacing* other disciplines, just infiltrating them as the point was initially misunderstood…]

Nassim Nicholas Taleb via Facebook

[My comments posted to the original Facebook post follow below.]

I’m coming to this post a bit late as I’m playing a bit of catch up, but agree with it wholeheartedly.

In particular, applications to molecular biology and medicine are really beginning to come to a heavy boil in just the past five years. This particular year is the progenitor of what appears to be the biggest renaissance for the application of information theory to the area of biology since Hubert Yockey, Henry Quastler, and Robert L. Platzman’s “Symposium on Information Theory in Biology at Gatlinburg, Tennessee” in 1956.

Upcoming/recent conferences/workshops on information theory in biology include:

BIRS Workshop: Biological and Bio-Inspired Information Theory
Entropy and Information in Biological Systems at NIMBios
CECAM Workshop: Entropy in Biomolecular Systems
ALife breakout session on Information Theoretic Incentives for Artificial Life (which will also spawn off a special issue of the journal Entropy):

At the beginning of September, Christoph Adami posted an awesome and very sound paper on arXiv entitled “Information-theoretic considerations concerning the origin of life” which truly portends to turn the science of the origin of life on its head.

I’ll note in passing, for those interested, that Claude Shannon’s infamous master’s thesis at MIT (in which he applied Boolean Algebra to electric circuits allowing the digital revolution to occur) and his subsequent “The Theory of Mathematical Communication” were so revolutionary, nearly everyone forgets his MIT Ph.D. Thesis “An Algebra for Theoretical Genetics” which presaged the areas of cybernetics and the current applications of information theory to microbiology and are probably as seminal as Sir R.A Fisher’s applications of statistics to science in general and biology in particular.

For those commenting on the post who were interested in a layman’s introduction to information theory, I recommend John Robinson Pierce’s An Introduction to Information Theory: Symbols, Signals and Noise (Dover has a very inexpensive edition.) After this, one should take a look at Claude Shannon’s original paper. (The MIT Press printing includes some excellent overview by Warren Weaver along with the paper itself.) The mathematics in the paper really aren’t too technical, and most of it should be comprehensible by most advanced high school students.

For those that don’t understand the concept of entropy, I HIGHLY recommend Arieh Ben-Naim’s book Entropy Demystified The Second Law Reduced to Plain Common Sense with Seven Simulated Games. He really does tear the concept down into its most basic form in a way I haven’t seen others come remotely close to and which even my mother can comprehend (with no mathematics at all). (I recommend this presentation to even those with Ph.D.’s in physics because it is so truly fundamental.)

For the more advanced mathematicians, physicists, and engineers Arieh Ben-Naim does a truly spectacular job of extending ET Jaynes’ work on information theory and statistical mechanics and comes up with a more coherent mathematical theory to conjoin the entropy of physics/statistical mechanics with that of Shannon’s information theory in A Farewell to Entropy: Statistical Thermodynamics Based on Information.

For the advanced readers/researchers interested in more at the intersection of information theory and biology, I’ll also mention that I maintain a list of references, books, and journal articles in a Mendeley group entitled “ITBio: Information Theory, Microbiology, Evolution, and Complexity.”

Venn Diagram of how information theory relates to other fields. — Figure 1.1 [page 2] from
Thomas M. Cover and Joy Thomas’s textbook *Elements of Information Theory*, Second Edition
(John Wiley & Sons, Inc., 2006) [First Edition, 1991]

A Mathematical Symphony

…I hope that in addition there will be some readers who will simply take pleasure in a mathematical journey toward a high level of sophistication. There are many who would enjoy this trip, just as there are many who might enjoy listening to a symphony with a clear melodic line.

Robert B. Ash, mathematician
in A Primer of Abstract Mathematics (The Mathematical Association of America, 1998)

A Primer of Abstract Mathematics by Robert B. Ash

The Single Biggest Problem in Communication

The single biggest problem in communication is the illusion that it has taken place.

apocryphally attributed to George Bernard Shaw,
but more likely William H. Whyte in Fortune, “Is Anybody Listening?” Start Page 77, Quote Page 174, Published by Time, Inc., New York (September 1950)

“We didn’t cover much, but we sure did learn.”

We had to prove the theorems ourselves—with hints, of course—and as a result didn’t get very far, covering perhaps a fourth of what might be done in a conventional lecture. But so what? The most persistent myth of mathematics education is that what is covered is the same as what is learned. We didn’t cover much, but we sure did learn.

Steve Mitchell, algebraic topologist
on learning mathematics in Steve Mitchell short biography

To Understand God’s Thought…

To understand God’s thought, we must study statistics, for these are the measure of His purpose.

Florence Nightingale, OM, RRC (1820-1910), English social reformer and statistician, founder of modern nursing, renaissance woman
in Florence Nightingale’s Wisdom, New York Times, 3/4/14

Florence Nightingale developed the polar pie chart to depict mortality causes in the Crimean War.

Information Theory is Something Like the Logarithm of Probability Theory

To put it saucily: information theory is something like the logarithm of probability theory. In early modern times the logarithm simplified multiplication into addition which was more accessible to calculation. Today, information theory transforms many quantities of probability theory into quantities which allow simpler bookkeeping.

More seriously, information theory is one of the most universal concepts with applications in computer science, mathematics, physics, biology, chemistry and other fields. It allows a lucid and transparent analysis of many systems and provides a framework to study and compare seemingly different systems using the same language and notions.

Dr. Daniel Polani, reader in Artificial Life, University of Hertfordshire
in “Research Questions”

Not only a great quote, but an interesting way to view the subjects.

Tool Review: Zojirushi Stainless Steel Mug

Designer/Artist William Morris once said, “Have nothing in your house that you do not know to be useful, or believe to be beautiful.” My Zojirushi stainless steel mug is one of the few things I’ve ever owned that I feel truly meets both of these criteria.

The design, materials, manufacturing and workmanship of the mug are nothing short of outstanding; the aesthetics and heft in the hand are truly fantastic. I really could not want for more out of such a product. I love looking at it, I love holding it, and I love using it.

I hope one day to come back and write a review worthy of how truly great this travel mug is, but for now, suffice it to say that I’m in love. I spent a LOT of time reading reviews on Amazon and elsewhere, and searching stores and vendors to find the best thermos/mug on the planet and settled on this one. Not only is it easy and intuitive to take completely apart and wash thoroughly (too many I’ve come across are impossible to take apart and clean properly, if at all), but it seals completely and doesn’t spill.

Even better it keeps my beverages piping hot or cold for far longer than I wish it would. There have been days that I’ve filled it with hot coffee or tea and come back several times to drink it hoping that it had cooled a bit only to find it still too hot to consume. After several rounds with this over an eight hour span, I finally opened it up and put in some ice so I could finally drink my coffee. Now I often just leave the cap open (or off) to let it cool a bit more quickly, although even this is a fairly slow process. Now I try to put my beverages in at the temperature I want to drink them knowing that that’s generally the temperature they’ll be when I get around to drinking them.

I love the fact that the cap is designed with a two stage opening mechanism (which probably won’t be noticed by most users because it’s so subtle). One pushes the button and the top opens just a few millimeters. Then letting go of the button allows the top to spring back and click neatly into place so that it doesn’t fall forward and bonk one on the nose when attempting to take a drink.

When I first came across it, I will admit I was a bit reticent at it’s relatively high price (particularly in comparison with cheaper mugs on the market, many of which I’ve tried and been highly disappointed with), but the Zojirushi is certainly worth ever penny; I would not hesitate for a moment to buy more of these.

As a small aside, I will mention that due to physics and the design of the mug that it can occasionally leak a bit when filled with carbonated beverages and then shaken. Doing this creates additional interior pressure that pushes up the internal seal mechanism on the cap that allows a small amount of liquid to escape. Beyond this small category of fluids, which I infrequently use with the mug (and I’m sure others probably won’t either), it has been absolutely airtight and worry-free.

Rating 5 out of 5 stars.

Review by Chris Aldrich

Lecture Series Review: “Augustine: Philosopher and Saint” by Phillip Cary

Augustine: Philosopher and Saint (Great Courses, #611)

Augustine: Philosopher and Saint byProfessor Phillip Cary, Ph.D., Eastern University (The Learning Company, 1997)

My rating: 4 of 5 stars

This series of 12 audio lectures is an excellent little overview of Augustine, his life, times, and philosophy. Most of the series focuses on his writings and philosophy as well as their evolution over time, often with discussion of the historical context in which they were created as well as some useful comparing/contrasting to extant philosophies of the day (and particularly Platonism.)

Early in the series there were some interesting and important re-definitions of some contemporary words. Cary pushes them back to an earlier time with slightly different meanings compared to their modern ones which certainly helps to frame the overarching philosophy presented. Without a close study of this vocabulary, many modern readers will become lost or certainly misdirected when reading modern translations. As examples, words like perverse, righteousness, and justice (or more specifically their Latin counterparts) have subtly different meanings in the late Roman empire than they do today, even in modern day religious settings.

My favorite part, however, has to have been the examples discussing mathematics as an extended metaphor for God and divinity to help to clarify some of Augustine’s thought. These were not only very useful, but very entertaining to me.

As an aside for those interested in mnemotechnic tradition, I’ll also mention that I’ve (re)discovered (see the reference to the Tell paper below) an excellent reference to the modern day “memory palace” (referenced most recently in the book Moonwalking with Einstein: The Art and Science of Remembering Everything) squirreled away in Book X of Confessions where Augustine discusses memory as:

“fields and spacious palaces” “…where are the treasures of innumerable images, brought into it from things of all sorts perceived by the senses. There is stored up, whatsoever besides we think, either by enlarging or diminishing, or any other way varying those things which the sense hath come to; and whatever else hath been committed and laid up, which forgetfulness hath not yet swallowed up and buried.”

Those interested in memes and the history of “memoria ex locis” (of which I don’t even find a reference explicitly written in the original Rhetorica ad Herrenium) would appreciate an additional reference I subsequently found in the opening (and somewhat poetic) paragraph of a paper written by David Tell on JSTOR. The earliest specific reference to a “memory palace” I’m aware of is Matteo Ricci’s in the 16th century, but certainly other references to the construct may have come earlier. Given that Ricci was a Jesuit priest, it’s nearly certain that he would have been familiar with Augustine’s writings at the time, and it’s possible that his modification of Augustine’s mention brought the concept into its current use. Many will know memory as one of the major underpinnings of rhetoric (of which Augustine was a diligent student) as part of the original trivium.

Some may shy away from Augustine because of the religious overtones which go along with his work, but though there were occasional “preachy sounding” sections in the material, they were present only to clarify the philosophy.

I’d certainly recommend this series of lectures to anyone not closely familiar with Augustine’s work as it has had a profound and continuing affect on Western philosophy, thought, and politics.

View all my reviews

The Two Cultures

A good many times I have been present at gatherings of people who, by the standards of the traditional culture, are thought highly educated and who have with considerable gusto been expressing their incredulity of scientists. Once or twice I have been provoked and have asked the company how many of them could describe the Second Law of Thermodynamics. The response was cold: it was also negative. Yet I was asking something which is the scientific equivalent of: Have you read a work of Shakespeare’s?

I now believe that if I had asked an even simpler question — such as, What do you mean by mass, or acceleration, which is the scientific equivalent of saying, Can you read? — not more than one in ten of the highly educated would have felt that I was speaking the same language. So the great edifice of modern physics goes up, and the majority of the cleverest people in the western world have about as much insight into it as their neolithic ancestors would have had.

C.P. Snow, Kt., CBE (1905 – 1980), an English physical chemist and novelist
in 1959 Rede Lecture entitled “The Two Cultures”

C. P. Snow, English physicist, author, and diplomat.

Fundamental Difference Between Science and Politics

What I do know is that there is a fundamental difference between science and politics. In fact, I’ve come to view them more and more as opposites.

In science, progress is possible.

Nate Silver, statistician
in The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t (Penguin Books, 2012)

God Could Have Caused Birds to Fly With Their Bones Made of Solid Gold

Surely, God could have caused birds to fly with their bones made of solid gold, with their veins full of quicksilver, with their flesh heavier than lead, and with their wings exceedingly small. He did not, and that ought to show something. It is only in order to shield your ignorance that you put the Lord at every turn.

Salviati’s (Galileo’s voice) response to Simplicio (Pope Urban VIII)
Galileo Galilei in Dialogue Concerning the Two Chief World Systems