Renaissance for Information Theory in Biology

This year is the progenitor of what appears to be the biggest renaissance for the application of information theory to the area of biology since Hubert Yockey, Henry Quastler, and Robert L. Platzman’s “Symposium on Information Theory in Biology at Gatlinburg, Tennessee” in 1956. (I might argue it’s possibly even bigger than Claude Shannon’s Ph.D. thesis.)  It certainly portends to create a movement that will rapidly build upon and far surpass Norbert Weiner’s concept of Cybernetics and Ludwig von Bertalanffy’s concept of General Systems Theory.

This week John Baez has announced an upcoming three day workshop on “Entropy and Information in Biological Systems” to be hosted by the National Institute for Mathematical and Biological Synthesis in Knoxville, TN, tentatively scheduled for October 22-24, 2014.

Apparently unbeknownst to Baez, earlier this year Andrew Eckford, Toby Berger, and Peter Thomas announced a six day workshop on “Biological and Bio-Inspired Information Theory” to be hosted by the Banff International Research Station for Mathematical Innovation and Discovery scheduled for October 26-31, 2014 – just two days later!

What a bonanza!!

The BIRS workshop will be a bit more general in its approach while the NIMBioS workshop has a slightly tighter view specifically on maximum entropy as applied to biology.

Even more telling (and perhaps most promising) about the two workshops is the very heavy mathematical bent both intend to make their focus.  I have a theory that the bounds of science are held below the high water level of mathematics (aka are “bounded by” in mathematics-speak), so there is nothing more exciting than to see groups attempting to push the mathematics and its application further. It was both the lack of mathematical rigor and the general youth of biology (and specifically genetics and microbiology) in the 1950’s which heavily hampered the early growth of cybernetics as a movement. Fortunately this is no longer the case on either count. Now we just need more researchers who are more readily conversant in the two realms simultaneously.

Book Review: “Complexity: A Guided Tour” by Melanie Mitchell

Read Complexity: A Guided Tour by Melanie MitchellMelanie Mitchell (amzn.to)
Complexity: A Guided Tour Book Cover Complexity: A Guided Tour
Melanie Mitchell
Popular Science
Oxford University Press
May 28, 2009
Hardcover
366

This book provides an intimate, highly readable tour of the sciences of complexity, which seek to explain how large-scale complex, organized, and adaptive behavior can emerge from simple interactions among myriad individuals. The author, a leading complex systems scientist, describes the history of ideas, current research, and future prospects in this vital scientific effort.

This is handily one of the best, most interesting, and (to me at least) the most useful popularly written science books I’ve yet to come across. Most popular science books usually bore me to tears and end up being only pedantic for their historical backgrounds, but this one is very succinct with some interesting viewpoints (some of which I agree with and some of which my intuition says are terribly wrong) on the overall structure presented.

For those interested in a general and easily readable high-level overview of some of the areas of research I’ve been interested in (information theory, thermodynamics, entropy, microbiology, evolution, genetics, along with computation, dynamics, chaos, complexity, genetic algorithms, cellular automata, etc.) for the past two decades, this is really a lovely and thought-provoking book.

At the start I was disappointed that there were almost no equations in the book to speak of – and perhaps this is why I had purchased it when it came out and it’s subsequently been sitting on my shelf for so long. The other factor that prevented me from reading it was the depth and breadth of other more technical material I’ve read which covers the majority of topics in the book. I ultimately found myself not minding so much that there weren’t any/many supporting equations aside from a few hidden in the notes at the end of the text in most part because Dr. Mitchell does a fantastic job of pointing out some great subtleties within the various subjects which comprise the broader concept of complexity which one generally would take several years to come to on one’s own and at far greater expense of their time. Here she provides a much stronger picture of the overall subjects covered and this far outweighed the lack of specificity. I honestly wished I had read the book when it was released and it may have helped me to me more specific in my own research. Fortunately she does bring up several areas I will need to delve more deeply into and raised several questions which will significantly inform my future work.

In general, I wish there were more references I hadn’t read or been aware of yet, but towards the end there were a handful of topics relating to fractals, chaos, computer science, and cellular automata which I have been either ignorant of or which are further down my reading lists and may need to move closer to the top. I look forward to delving into many of these shortly. As a simple example, I’ve seen Zipf’s law separately from the perspectives of information theory, linguistics, and even evolution, but this is the first time I’ve seen it related to power laws and fractals.

I definitely appreciated the fact that Dr. Mitchell took the time to point out her own personal feelings on several topics and more so that she explicitly pointed them out as her own gut instincts instead of mentioning them passingly as if they were provable science which is what far too many other authors would have likely done. There are many viewpoints she takes which I certainly don’t agree with, but I suspect that it’s because I’m coming at things from the viewpoint of an electrical engineer with a stronger background in information theory and microbiology while hers is closer to that of computer science. She does mention that her undergraduate background was in mathematics, but I’m curious what areas she specifically studied to have a better understanding of her specific viewpoints.

Her final chapter looking at some of the pros and cons of the topic(s) was very welcome, particularly in light of previous philosophic attempts like cybernetics and general systems theory which I (also) think failed because of their lack of specificity. These caveats certainly help to place the scientific philosophy of complexity into a much larger context. I will generally heartily agree with her viewpoint (and that of others) that there needs to be a more rigorous mathematical theory underpinning the overall effort. I’m sure we’re all wondering “Where is our Newton?” or to use her clever aphorism that we’re “waiting for Carnot.” (Sounds like it should be a Tom Stoppard play title, doesn’t it?)

I might question her brief inclusion of her own Ph.D. thesis work in the text, but it did actually provide a nice specific and self-contained example within the broader context and also helped to tie several of the chapters together.

My one slight criticism of the work would be the lack of better footnoting within the text. Though many feel that footnote numbers within the text or inclusion at the bottom of the pages detracts from the “flow” of the work, I found myself wishing that she had done so here, particularly as I’m one of the few who actually cares about the footnotes and wants to know the specific references as I read. I hope that Oxford eventually publishes an e-book version that includes cross-linked footnotes in the future for the benefit of others.

I can heartily recommend this book to any fan of science, but I would specifically recommend it to any undergraduate science or engineering major who is unsure of what they’d specifically like to study and might need some interesting areas to take a look at. I will mention that one of the tough parts of the concept of complexity is that it is so broad and general that it encompasses over a dozen other fields of study each of which one could get a Ph.D. in without completely knowing the full depth of just one of them much less the full depth of all of them. The book is so well written that I’d even recommend it to senior researchers in any of the above mentioned fields as it is certainly sure to provide not only some excellent overview history of each, but it is sure to bring up questions and thoughts that they’ll want to include in their future researches in their own specific sub-areas of expertise.

How to Sidestep Mathematical Equations in Popular Science Books

In the publishing industry there is a general rule-of-thumb that every mathematical equation included in a book will cut the audience of science books written for a popular audience in half – presumably in a geometric progression. This typically means that including even a handful of equations will give you an effective readership of zero – something no author and certainly no editor or publisher wants.

I suspect that there is a corollary to this that every picture included in the text will help to increase your readership, though possibly not by as proportionally a large amount.

In any case, while reading Melanie Mitchell’s text Complexity: A Guided Tour [Cambridge University Press, 2009] this weekend, I noticed that, in what appears to be a concerted effort to include an equation without technically writing it into the text and to simultaneously increase readership by including a picture, she cleverly used a picture of Boltzmann’s tombstone in Vienna! Most fans of thermodynamics will immediately recognize Boltzmann’s equation for entropy, S = k log W , which appears engraved on the tombstone over his bust.

Page 51 of Melanie Mitchell's book "Complexity: A Guided Tour"
Page 51 of Melanie Mitchell’s book “Complexity: A Guided Tour” featuring Boltzmann’s tombstone in Vienna.

I hope that future mathematicians, scientists, and engineers will keep this in mind and have their tombstones engraved with key formulae to assist future authors in doing the same – hopefully this will help to increase the amount of mathematics that is deemed “acceptable” by the general public.

Book Review: Gregory Chaitin’s “Proving Darwin: Making Biology Mathematical”

Gregory Chaitin’s book Proving Darwin: Making Biology Mathematical combining biology, microbiology, mathematics, evolution and even information theory is directly in my wheelhouse. I had delayed reading it following a few initial poor reviews, and sadly I must confirm that I’m ultimately disappointed in the direct effort shown here, though there is some very significant value buried within. Unfortunately the full value is buried so deeply that very few, if any, will actually make the concerted effort to find it.

proving

This effort does seem to make a more high-minded and noble attempt than what I would call the “Brian Greene method” in which an academic seemingly gives up on serious science to publish multiple texts on a popular topic to cash in on public interest in that topic through sales of books. In this respect Chaitin is closer to Neil deGrasse Tyson in his effort to expound an interesting theory to the broader public and improve the public discourse, though I would admit he’s probably a bit more (self-) interested in pushing his own theory and selling books (or giving him the benefit of the doubt, perhaps the publisher has pushed him to this).

Though there is a reasonable stab at providing some philosophical background to fit the topic into the broader fabric of science and theory in the later chapters, most of it is rather poorly motivated and is covered far better in other non-technical works. While it is nice to have some semblance of Chaitin’s philosophy and feelings, the inclusion of this type of material only tends to soften the blow of his theoretical work and makes the text read more like pseudo-science or simple base philosophy without any actual rigorous underpinning.

I’m assuming that his purpose in writing the book is to make the theories he’s come up with in his primary paper on the topic more accessible to the broader community of science as well as the public itself. It’s easy for a groundbreaking piece of work to be hidden in the broader scientific literature, but Chaitin seems to be taking his pedestal as a reasonably popular science writer to increase the visibility of his work here. He admittedly mentions that his effort stems from his hobby as his primary area is algorithmic information theory and computer science and not biology or evolution, though his meager references in the text do at least indicate some facility with some of the “right” sources in these latter areas.

Speaking from a broad public perspective, there is certainly interest in this general topic to warrant such a book, though based on the reviews of others via Amazon, Goodreads, etc. the book has sadly missed it’s mark. He unfortunately sticks too closely to the rule that inclusion of mathematical equations is detrimental to the sale of ones books. Sadly, his broader point is seemingly lost on the broader public as his ability to analogize his work isn’t as strong as that of Brian Greene with respect to theoretical physics (string theory).

From the a higher perspective of a researcher who does work in all of the relevant areas relating to the topic, I was even more underwhelmed with the present text aside from the single URL link to the original much more technical paper which Chaitin wrote in 2010. To me this was the most valuable part of the entire text though he did provide some small amount of reasonable detail in his appendix.

I can certainly appreciate Chaitin’s enthusiastic following of John von Neumann but I’m disappointed in his lack of acknowledgement of Norbert Weiner or Claude Shannon who all collaborated in the mid part of the 20th century. I’m sure Chaitin is more than well aware of the father of information theory, but I’ll be willing to bet that although he’s probably read his infamous master’s thesis and his highly influential Bell Labs article on “A/The Mathematical Theory of Communication”, he is, like most, shamefully and wholly unaware of Shannon’s MIT doctoral thesis.

Given Chaitin’s own personal aim to further the acceptance of his own theories and work and the goal of the publisher to sell more copies, I would mention a few recommendations for future potential editions:

The greater majority of his broader audience will have at least a passably reasonable understanding of biology and evolution, but very little, if any, understanding of algorithmic information theory. He would be better off expounding upon this subject to bring people up to speed to better understand his viewpoint and his subsequent proof. Though I understand the need to be relatively light in regard to the number of equations and technicalities included, Chaitin could follow some of his heroes of mathematical exposition and do a slightly better job of explaining what is going on here. He could also go a long way toward adding some significant material to the appendices to help the higher end general readers and the specifically the biologists understand more of the technicalities of algorithmic information theory to better follow his proof which should appear in intricate glory in the appendix as well. I might also recommend excising some of the more philosophical material which tends to undermine his scientific “weight.” Though I found it interesting that he gives a mathematical definition of “intelligent design”, I have a feeling its intricacies were lost on most of his readership — this point alone could go a long way towards solidifying the position of evolution amongst non-scientists, particularly in America, and win the support of heavyweights like Dawkins himself.

I’ll agree wholeheartedly with one reviewer who said that Chaitin tends to “state small ideas repeatedly, and every time at the same shallow level with astonishing amount of redundancy (mostly consisting of chit-chat and self congratulations)”. This certainly detracted from my enjoyment of the work. Chaitin also includes an awful lot of name dropping of significant scientific figures tangential to the subject at hand. This may have been more impressive if he included the results of his discussions with them about the subject, but I’m left with the impression that he simply said hello, shook their hands, and at best was simply inspired by his having met them. It’s nice that he’s had these experiences, but it doesn’t help me to believe or follow his own work.

For the technically interested reader, save yourself some time and simply skim through chapter five and a portion of the appendix relating to his proof and then move on to his actual paper. For the non-technical reader, I expect you’ll get more out of reading Richard Dawkins’ early work (The Selfish Gene) or possibly Werner R. Loewenstein’s The Touchstone of Life: Molecular Information, Cell Communication, and the Foundations of Life.

Though I would certainly agree that we could use a mathematical proof of evolution, and that Chaitin has made a reasonable theoretical stab, this book sadly wasn’t the best one to motivate broader interest in such an effort. I’ll give him five stars for effort, three for general content, but in the end, for most it will have to be at most a 2 star work overall.

This review was originally published on June 17, 2013.

Axiom of Choice? “Would you rather be deaf or blind?”

Sir Michael Francis Atiyah, OMFRSFRSEFAA, a British mathematician
in Mathematics in the 20th Century

 

Book Review: Charles Seife’s “Proofiness: The Dark Arts of Mathematical Deception”

Read Proofiness: The Dark Arts of Mathematical Deception (Penguin)
Proofiness: The Dark Arts of Mathematical Deception Book Cover Proofiness: The Dark Arts of Mathematical Deception
Charles Seife
Mathematics, Popular Science
Penguin
September 23, 2010
Hardcover
320

The bestselling author of Zero shows how mathematical misinformation pervades-and shapes-our daily lives. According to MSNBC, having a child makes you stupid. You actually lose IQ points. Good Morning America has announced that natural blondes will be extinct within two hundred years. Pundits estimated that there were more than a million demonstrators at a tea party rally in Washington, D.C., even though roughly sixty thousand were there. Numbers have peculiar powers-they can disarm skeptics, befuddle journalists, and hoodwink the public into believing almost anything. "Proofiness," as Charles Seife explains in this eye-opening book, is the art of using pure mathematics for impure ends, and he reminds readers that bad mathematics has a dark side. It is used to bring down beloved government officials and to appoint undeserving ones (both Democratic and Republican), to convict the innocent and acquit the guilty, to ruin our economy, and to fix the outcomes of future elections. This penetrating look at the intersection of math and society will appeal to readers of Freakonomics and the books of Malcolm Gladwell.

Charles Seife doesn’t prove that mathematics is essential for a democracy, but he certainly shows how the lack of proper use of mathematics can fray heavily at the edges!

Proofiness was a great book to have read over a long Fourth of July holiday. Though many people may realize some of the broad general concepts in the book, it’s great to have a better structure for talking about concepts like Potemkin numbers, disestimation, fruit packing, cherry picking, apple polishing, comparing apples to oranges, causuistry, randnumbness, regression to the moon, tragedy of the commons, and moral hazard among others. If you didn’t think mathematics was important to daily life or our democratic society, this book will certainly change your mind.

Seife covers everything from polls, voting, politics, economics, marketing, law, and even health to show how numbers are misused in a modern world that can ill-afford to ignore what is really going on around us.

This is a fantastic book for nearly everyone in the general public, but I’d highly recommend it for high school students while taking civics.

Original review posted on GoodReads.com on 7/9/12.

Reading Progress
  • 07/07/12 marked as: currently reading
  • 07/07/12 23.0% #
  • 07/09/12 52.0%
  • 07/09/12 Finished book

You Cannot Learn Too Much Linear Algebra

Benedict Gross, Ph.D., George Vasmer Leverett Professor of Mathematics, Harvard University
in Abstract Algebra, Lecture 2 at 14:25 via Harvard Extension

 

Benedict Gross standing in front of chalkboard with equations from Abstract Algebra Class
Benedict Gross teaching abstract algebra

Mathematics in Popular Science Books | The Economist

Reposted Big bang (The Economist)
Popular physics has enjoyed a new-found regard. Now comes a brave attempt to inject mathematics into an otherwise fashionable subject
This review of Brian Cox and Jeff Forshaw’s forthcoming book The Quantum Universe: Everything That Can Happen Does Happen sounds intriguing. I’m highly impressed that so much of the review focuses on the author’s decision to include a more mathematical treatment of their subject for what is supposed to be a popular science book. I always wish books like these at least had the temerity to include much more in the way of the mathematical underpinnings of their subjects; I’m glad that the popular press (or at least The Economist in this case) is willing to be asking for the mathematics as well. Hopefully it will mark a broader trend in popular books on scientific topics!

Fundamental physics

Big bang

Popular physics has enjoyed a new-found regard. Now comes a brave attempt to inject mathematics into an otherwise fashionable subject

Nov 5th 2011 | from the print edition

The Quantum Universe: Everything That Can Happen Does Happen. By Brian Cox and Jeff Forshaw. Allen Lane; 255 pages; £20. To be published in America in January by Da Capo Press; $25.

PREVIOUSLY the preserve of dusty, tweed-jacketed academics, physics has enjoyed a surprising popular renaissance over the past few years. In America Michio Kaku, a string theorist, has penned several successful books and wowed television and radio audiences with his presentations on esoteric subjects such as the existence of wormholes and the possibility of alien life. In Britain Brian Cox, a former pop star whose music helped propel Tony Blair to power, has become the front man for physics, which recently regained its status as a popular subject in British classrooms, an effect many attribute to Mr Cox’s astonishing appeal.

Mr Cox, a particle physicist, is well-known as the presenter of two BBC television series that have attracted millions of viewers (a third series will be aired next year) and as a bestselling author and public speaker. His latest book, “The Quantum Universe”, which he co-wrote with Jeff Forshaw of the University of Manchester, breaks the rules of popular science-writing that were established over two decades ago by Stephen Hawking, who launched the modern genre with his famous book, “A Brief History of Time”.

Mr Hawking’s literary success was ascribed to his eschewing equations. One of his editors warned him that sales of the book would be halved by every equation he included; Mr Hawking inserted just one, E=mc2, and, even then, the volume acquired a sorry reputation for being bought but not read. By contrast, Mr Cox, whose previous book with Mr Forshaw investigated “Why does E=mc2?” (2009), has bravely sloshed a generous slug of mathematics throughout his texts.

The difficulties in explaining physics without using maths are longstanding. Einstein mused, “The eternal mystery of the world is its comprehensibility,” and “the fact that it is comprehensible is a miracle.” Yet the language in which the world is described is that of maths, a relatively sound grasp of which is needed to comprehend the difficulties that physicists are trying to resolve as well as the possible solutions. Mr Cox has secured a large fan base with his boyish good looks, his happy turns of phrase and his knack for presenting complex ideas using simple analogies. He also admirably shies away from dumbing down. “The Quantum Universe” is not a dry undergraduate text book, but nor is it a particularly easy read.

The subject matter is hard. Quantum mechanics, which describes in subatomic detail a shadowy world in which cats can be simultaneously alive and dead, is notoriously difficult to grasp. Its experiments yield bizarre results that can be explained only by embracing the maths that describe them, and its theories make outrageous predictions (such as the existence of antimatter) that have nevertheless later been verified. Messrs Cox and Forshaw say they have included the maths “mainly because it allows us to really explain why things are the way they are. Without it, we should have to resort to the physicist-guru mentality whereby we pluck profundities out of thin air, and neither author would be comfortable with guru status.”

That stance might comfort the authors, but to many readers they will nonetheless seem to pluck equations out of thin air. Yet their decision to include some of the hard stuff leaves open the possibility that some readers might actually engage in the slog that leads to higher pleasures. For non-sloggers alternative routes are offered: Messrs Cox and Forshaw use clockfaces to illustrate how particles interact with one another, a drawing of how guitar strings twang and a photograph of a vibrating drum. A diagram, rather than an equation, is used to explain one promising theory of how matter acquires mass, a question that experiments on the Large Hadron Collider at CERN, the European particle-physics laboratory near Geneva, will hopefully soon answer.

The authors have wisely chosen to leaven their tome with amusing tales of dysfunctional characters among scholars who developed quantum mechanics in the 1920s and beyond, as well as with accounts of the philosophical struggles with which they grappled and the occasional earthy aside. Where the subject matter is a trifle dull, Messrs Cox and Forshaw acknowledge it: of Heinrich Kayser, who a century ago completed a six-volume reference book documenting the spectral lines generated by every known element, they observe, “He must have been great fun at dinner parties.” And they make some sweeping generalisations about their colleagues who pore over equations, “Physicists are very lazy, and they would not go to all this trouble unless it saved time in the long run.”

Whether or not readers of “The Quantum Universe” will follow all the maths, the authors’ love for their subject shines through the book. “There is no better demonstration of the power of the scientific method than quantum theory,” they write. That may be so, but physicists all over the world, Messrs Cox and Forshaw included, are longing for the next breakthrough that will supersede the claim. Hopes are pinned on experiments currently under way at CERN that may force physicists to rethink their understanding of the universe, and inspire Messrs Cox and Forshaw to write their next book—equations and all.

from the print edition | Books and arts

John McCarthy on Arithmetic

John McCarthy (), an American computer scientist and cognitive scientist who was one of the founders of the discipline of artificial intelligence
in Computer Scientist Coined ‘Artificial Intelligence’ in The Wall Street Journal

 

The Response of the Schoolmaster

This must certainly be the quote of the week from English author Alan Bennett’s play Forty Years On:

Foster: I’m still a bit hazy about the Trinity, sir.
Schoolmaster: Three in one, one in three, perfectly straightforward.  Any doubts about that see your maths master.

 

HARASS SARAH is a PALINdrome, as well as a popular left-wing sport.

This is definitely the quote of the week:

Sol Golomb, mathematician and information theorist
via personal communication while discussing a palindromic word puzzle

Paul Halmos on Prerequisites

Definitely the quote of the day:

Paul Halmos (1916 – 2006, Hungarian-born American mathematician
in Measure Theory (1950)

 

This is essentially the mathematician’s equivalent of the adage “Fake it ’til you make it.”

Riemann’s On the Hypotheses Which Lie at the Foundations of Geometry

One must be truly enamored of the internet that it allows one to find and read a copy of Bernhard Riemann’s doctoral thesis Habilitation Lecture (in English translation) at the University of Göttingen from 1854!

His brief paper has created a tsunami of mathematical work and research in the ensuing 156 years. It has ultimately become one of the seminal works in the development of the algebra and calculus of n-dimensional manifolds.

Global classical solutions of the Boltzmann equation with long-range interactions

Bookmarked Global classical solutions of the Boltzmann equation with long-range interactions (pnas.org)
Finally, after 140 years, Robert Strain and Philip Gressman at the University of Pennsylvania have found a mathematical proof of Boltzmann’s equation, which predicts the motion of gas molecules.

Abstract

This is a brief announcement of our recent proof of global existence and rapid decay to equilibrium of classical solutions to the Boltzmann equation without any angular cutoff, that is, for long-range interactions. We consider perturbations of the Maxwellian equilibrium states and include the physical cross-sections arising from an inverse-power intermolecular potential r-(p-1) with p > 2, and more generally. We present here a mathematical framework for unique global in time solutions for all of these potentials. We consider it remarkable that this equation, derived by Boltzmann (1) in 1872 and Maxwell (2) in 1867, grants a basic example where a range of geometric fractional derivatives occur in a physical model of the natural world. Our methods provide a new understanding of the effects due to grazing collisions.

via pnas.org