## The Information Universe Conference

"The Information Universe" Conference in The Netherlands in October hits several of the sweet spots for areas involving information theory, physics, the origin of life, complexity, computer science, and microbiology.

Yesterday, via a notification from Lanyard, I came across a notice for the upcoming conference “The Information Universe” which hits several of the sweet spots for areas involving information theory, physics, the origin of life, complexity, computer science, and microbiology. It is scheduled to occur from October 7-9, 2015 at the Infoversum Theater in Groningen, The Netherlands.

I’ll let their site speak for itself below, but they already have an interesting line up of speakers including:

### Keynote speakers

• Erik Verlinde, Professor Theoretical Physics, University of Amsterdam, Netherlands
• Alex Szalay, Alumni Centennial Professor of Astronomy, The Johns Hopkins University, USA
• Gerard ‘t Hooft, Professor Theoretical Physics, University of Utrecht, Netherlands
• Gregory Chaitin, Professor Mathematics and Computer Science, Federal University of Rio de Janeiro, Brasil
• Charley Lineweaver, Professor Astronomy and Astrophysics, Australian National University, Australia
• Lude Franke, Professor System Genetics, University Medical Center Groningen, Netherlands

### Conference synopsis from their homepage:

Additional details about the conference including the participants, program, venue, and registration can also be found at their website.

## NIMBioS Workshop: Information Theory and Entropy in Biological Systems

Web resources for participants in the NIMBioS Worshop on Information Theory and Entropy in Biological Systems.

Over the next few days, I’ll be maintaining a Storify story covering information related to and coming out of the Information Theory and Entropy Workshop being sponsored by NIMBios at the Unviersity of Tennessee, Knoxville.

For those in attendance or participating by watching the live streaming video (or even watching the video after-the-fact), please feel free to use the official hashtag , and I’ll do my best to include your tweets, posts, and material into the story stream for future reference.

For journal articles and papers mentioned in/at the workshop, I encourage everyone to join the Mendeley.com group ITBio: Information Theory, Microbiology, Evolution, and Complexity and add them to the group’s list of papers. Think of it as a collaborative online journal club of sorts.

Those participating in the workshop are also encouraged to take a look at a growing collection of researchers and materials I maintain here. If you have materials or resources you’d like to contribute to the list, please send me an email or include them via the suggestions/submission form or include them in the comments section below.

## BIRS Workshop on Biological and Bio-Inspired Information Theory | Storify Stream

Over the span of the coming week, I'll be updating (and archiving) the stream of information coming out of the BIRS Workshop on Biological and Bio-Inspired Information Theory.

Over the span of the coming week, I’ll be updating (and archiving) the stream of information coming out of the BIRS Workshop on Biological and Bio-Inspired Information Theory.

Editor’s note: On 12/12/17 Storify announced they would be shutting down. As a result, I’m changing the embedded version of the original data served by Storify for an HTML copy which can be found below:

# BIRS: Biological and Bio-Inspired Information Theory

A 5 Day workshop on Biology and Information Theory hosted by the Banff International Research Station

1. I know where I’ll be in Oct 2014! Let’s hear it for Biology & Information Theory!  https://www.birs.ca/events/2014/5-day-workshops/14w5170  #ITBio #Banff @andreweckford
2. . @andreweckford You might be interested in this grouping of research papers:  http://www.mendeley.com/groups/2545131/itbio/  #ITBio #Banff
3. Wishing I was at the Gene Regulation and Information Theory meeting starting tomorrow  http://bit.ly/XnHRZs  #ITBio
4. #ITBio: @andreweckford has a new book on Molecular Communication available Oct 31.  http://bit.ly/15uEUzF
5. Mathematical and Statistical Models for Genetic Coding starts today.  http://www.am.hs-mannheim.de/genetic_code_2013.php?id=1  @andreweckford might borrow attendees for BIRS
6. Mathematical Foundations for Information Theory in Diffusion-Based Molecular Communications  http://bit.ly/1aTVR2c  #ITBio
7. Bill Bialek giving plenary talk “Information flow & order in real biological networks” at Feb 2014 workshop  http://mnd.ly/19LQH8f  #ITBio
8. Workshop on Information Theoretic Incentives for Artificial Life  http://jhu.md/1lM8tAn  #ITBio #ALife14 @alifeofficial @14thALIFE @cxdig
9. Researchers working in information theory & biology  http://jhu.md/1gieQGR  #ITBio @andreweckford @ChristophAdami @wbialek @johnhawks
10. #ITBio http://t.co/Ty8dEIXQUT"/>

CECAM Workshop: “Entropy in Biomolecular Systems” starts May 14 in Vienna.  http://jhu.md/1faLR8t  #ITBio pic.twitter.com/Ty8dEIXQUT
11. Currently organizing my Banff workshop on bio-information theory …  https://www.birs.ca/events/2014/5-day-workshops/14w5170
12. Last RT: wonder what the weather is going to be like at the end of October for my @BIRS_Math workshop
13. @JoVanEvery I’m organizing a workshop in Banff in October … hopefully this isn’t a sign of weather to come!
14. How information theory could hold the key to quantifying nature.  http://wrd.cm/1uy1xdX  by @vero_greenwood pic.twitter.com/ek5DUb2Ul9
15. Good morning from Banff. Current temp: -1 C
16. Banff takes its name from the town of Banff, Scotland, not to be confused with Bamff, also Scotland.
17. Good morning from beautiful Banff. How can you not love the mountains? pic.twitter.com/mxYBNz7yzl
18. “Not an obvious connection between utility and information, just as there is no obvious connection between energy and entropy” @BIRS_Math
19. Peter Thomas (Case Western Reserve University), Signal Transduction and Information Theory  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410270940-Thomas.mp4
20. Last RT: a lot of discussion of my signal transduction work with Peter Thomas.
21. Live now: Nicolo Michelusi of @USCViterbi on Stochastic Model for Electron Transfer in Bacterial Cables  http://www.birs.ca/live  #ITBio
22. Nicolo Michelusi (University of Southern California), A Stochastic Model for Electron Transfer in Bacterial Cables  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410271450-Michelusi.mp4
23. Listening to the always awesome @cnmirose talk about the ultimate limits of molecular communication.
24. “Timing is fundamental … subsumes time-varying concentration channel” @cnmirose @BIRS_Math
25. Chris Rose (Rutgers University), Molecular Communication Channels: timing vs. payload  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410271538-Rose.mp4
26. Standard opening quote of these talks: “I’m not a biologist, but …” @BIRS_Math
27. Stefan Moser (ETH Zurich), Capacity Bounds of the Memoryless AIGN Channel – a Toy-Model for Molecular Communicat…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410271610-Moser.mp4
28. Weisi Guo (University of Warwick), Communication Envelopes for Molecular Diffusion and Electromagnetic Wave Propag…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410271643-Guo.mp4
29. Terrific introduction of Canada/Banff by Andrew Eckford (York)The Landscape  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410270858-Eckford.mp4
30. Biological and Bio-Inspired Information Theory workshop videos!  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos  @BIRS_Math
31. .@ChrisAldrich @andreweckford @Storify @BIRS_Math Sounds like a fascinating workshop on bioinformation theory in Banff.
32. Toby Berger, winner of the 2002 Shannon award, speaking right now. @BIRS_Math
33. Naftali Tishby (Hebrew University of Jerusalem), Sensing and acting under information constraints – a principled a…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281032-Tishby.mp4
34. “…places such as BIRS and the Banff Centre exist to facilitate the exchange and pursuit of knowledge.” S. Sundaram  http://www.birs.ca/testimonials/#testimonial-1454
35. We’re going for a hike tomorrow. Many thanks to Lukas at the @ParksCanada info centre in Banff for helpful advice! @BIRS_Math
36. Behnaam Aazhang (Rice University), Real-Time Network Modulation for Intractable Epilepsy  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281337-Aazhang.mp4
37. Alexander Dimitrov (Washington State University), Invariant signal processing in auditory biological systems  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281416-Dimitrov.mp4
38. Joel Zylberberg (University of Washington), Communicating with noisy signals: lessons learned from the mammalian v…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281450-Zylberberg.mp4
39. Robert Schober (Universitat Erlangen-Nurnberg), Intersymbol interference mitigation in diffusive molecular communi…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281549-Schober.mp4
40. Rudolf Rabenstein (Friedrich-Alexander-Universitat Erlangen-Nurnberg (FAU)), Modelling Molecular Communication Cha…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281627-Rabenstein.mp4
42. THis week @BIRS_Math ” Biological and Bio-Inspired Information Theory ” @thebanffcentre #biology #math @NSF
43. “Your theory might match the data, but the data might be wrong” – Crick @BIRS_Math
44. So information theory seems to be a big deal in ecology. @BIRS_Math
45. Tom Schneider (National Institutes of Health), Three Principles of Biological States: Ecology and Cancer  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410290904-Schneider.mp4
46. “In biodiversity, the entropy of an ecosystem is the expected … information we gain about an organism by learning its species” @BIRS_Math
47. Seriously, I’m blown away by this work in information theory in ecology. Huge body of work; I had no idea. @BIRS_Math
48. .@andreweckford @BIRS_Math Harte’s book Maximum Entropy & Ecology is excellent in this area  http://amzn.to/1DwIl3V  pic.twitter.com/EIBDpM35uf
49. .@andreweckford @QuantaMagazine had a nice overview of some of John Harte’s work in September  http://bit.ly/1DwIWCD  @BIRS_Math
50. Chan-Byoung Chae (Yonsei University), Molecular MIMO: From Theory to Practice  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410281705-Chae.mp4
51. John Baez (University of California, Riverside), Biodiversity, entropy and thermodynamics  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410291038-Baez.mp4
52. I encourage @BIRS_Math attendees at Biological & Bio-Inspired Information Theory to contribute references here:  http://bit.ly/1jQwObk
53. Christoph Adami (Michigan State University), Some Information-Theoretic Musings Concerning the Origin and Evolutio…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410291114-Adami.mp4
54. #ITBio http://t.co/VA8komuuSW"/>

.@ChristophAdami talk Some Information-Theoretic Musings Concerning the Origin of Life @BIRS_Math this morning #ITBio pic.twitter.com/VA8komuuSW
55. ICYMI @ChristophAdami had great paper: Information-theoretic Considerations on Origin of Life on arXiv  http://bit.ly/1yIhK2Q  @BIRS_Math
56. Johnston Canyon selfie @BIRS_Math pic.twitter.com/MEKeY5To5s
57. Baez has a post on Tishby’s talk “Sensing & Acting Under Information Constraints”  http://bit.ly/1yIDonR  @BIRS_Math pic.twitter.com/dFuiVLFSGC
58. INFORMATION THEORY is the new central …
59. I’m listening to a talk on the origin of life at a workshop on Biological and Bio-Inspired Information Theory. …  https://plus.google.com/117562920675666983007/posts/gqFL7XY3quF
60. Ilya Nemenman @EmoryUniversity on Predictive information  http://bit.ly/1titfOw
61. Now accepting applications for the #Research Collaboration Workshop for Women in #MathBio at NIMBioS  http://ow.ly/DzeZ7
62. Inkpots selfie from yesterday’s hike. @BIRS_Math pic.twitter.com/0A6ZQsQVwE
63. On the way home from Inkpots. @BIRS_Math pic.twitter.com/1XhO8mLOkq
64. Toby Berger (University of Virginia), Neruoscience Applications of GIG Distributions  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410280914-Berger.mp4
65. We removed a faulty microphone from our lecture room this morning. We’re now fixing the audio buzz in this week’s videos, and reposting.
66. Daniel Polani (University of Hertfordshire), Informational Principles in Perception-Action Loops  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301038-Polani.mp4
67. Didn’t get enough information theory & biology this week @BIRS_Math? Apply for NIMBioS workshop in April 2015  http://bit.ly/1yIeiWe  #ITBio
68. Amin Emad (University of Illinois at Urbana-Champaign), Applications of Discrete Mathematics in Bioinformatics  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301329-Emad.mp4
69. Paul Bogdan (University of Southern California), Multiscale Analysis Reveals Complex Behavior in Bacteria Populati…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301401-Bogdan.mp4
70. Robert Shaw (ProtoLife Inc.), Information and Causality in a Reaction-Diffusion System  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301434-Shaw.mp4
71. Lubomir Kostal (Institute of Physiology, Academy of Sciences of the Czech Republic), Efficient information transmi…  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301534-Kostal.mp4
72. Nima Soltani (Stanford University), Applications of Directed Information to Neuroscience  http://www.birs.ca/events/2014/5-day-workshops/14w5170/videos/watch/201410301647-Soltani.mp4
73. Banff ☀️❄️🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲❤️
74. @lrvarshney I shoulda invited you to this BIRS workshop …
75. @conservativelez I’m a big fan of your dad’s research & was reminded of much of it via a workshop on Biological Information Theory
76. @conservativelez Though he may not have been able to attend, he can catch most of the talks online if he’d like  https://www.birs.ca/events/2014/5-day-workshops/14w5170
77. Depressed that @BIRS_Math Workshop on Biological & Bio-Inspired Information Theory is over? Relive it here:  http://bit.ly/1rF3G4B  #ITBio
78. Kudos @andreweckford, Toby Berger, Peter Thomas, @NGhoussoub, @BIRS_Math & friends on a fantastic workshop!  http://bit.ly/1ckttZq
79. This @BIRS_Math Workshop was biggest thing in #informationtheory & #biology since the Gatlinburg Symposium in 1956.  http://bit.ly/1rF4RRr
80. See you later Calgary. pic.twitter.com/mkmU6yrmVz
81. A few thoughts about that workshop while I wait for my flight back to Toronto.
82. 1/ Everyone I talked to said it was the best workshop they’d ever been to, and they’d like to do a follow-up workshop @BIRS_Math
83. 2/ There is an amazing diversity of work under the umbrella of “information theory”. @BIRS_Math
84. 3/ Much of this work is outside the IT mainstream, and an issue is that people use different terms for related concepts. @BIRS_Math
85. 4/ Some community building is in order. I think this workshop was a good first step. @BIRS_Math
86. 5/ Many many thanks to @BIRS_Math and huge kudos to @NGhoussoub for excellent service to the Canadian scientific community. BIRS is a gem.
87. 6/ Also many thanks to the participants for their excellent talks, and to @ChrisAldrich for maintaining a Storify.

## Information Theory is the New Central Discipline

Replied to Information Theory is the new central discipline. by Nassim Nicholas Taleb (facebook.com)

I’m coming to this post a bit late as I’m playing a bit of catch up, but agree with it wholeheartedly.

In particular, applications to molecular biology and medicine are really beginning to come to a heavy boil in just the past five years. This particular year is the progenitor of what appears to be the biggest renaissance for the application of information theory to the area of biology since Hubert Yockey, Henry Quastler, and Robert L. Platzman’s “Symposium on Information Theory in Biology at Gatlinburg, Tennessee” in 1956.

Upcoming/recent conferences/workshops on information theory in biology include:

At the beginning of September, Christoph Adami posted an awesome and very sound paper on arXiv entitled “Information-theoretic considerations concerning the origin of life”  which truly portends to turn the science of the origin of life on its head.

I’ll note in passing, for those interested, that Claude Shannon’s infamous master’s thesis at MIT (in which he applied Boolean Algebra to electric circuits allowing the digital revolution to occur) and his subsequent “The Theory of Mathematical Communication” were so revolutionary, nearly everyone forgets his MIT Ph.D. Thesis “An Algebra for Theoretical Genetics” which presaged the areas of cybernetics and the current applications of information theory to microbiology and are probably as seminal as Sir R.A Fisher’s applications of statistics to science in general and biology in particular.

For those commenting on the post who were interested in a layman’s introduction to information theory, I recommend John Robinson Pierce’s An Introduction to Information Theory: Symbols, Signals and Noise (Dover has a very inexpensive edition.) After this, one should take a look at Claude Shannon’s original paper. (The MIT Press printing includes some excellent overview by Warren Weaver along with the paper itself.) The mathematics in the paper really aren’t too technical, and most of it should be comprehensible by most advanced high school students.

For those that don’t understand the concept of entropy, I HIGHLY recommend Arieh Ben-Naim’s book Entropy Demystified The Second Law Reduced to Plain Common Sense with Seven Simulated Games. He really does tear the concept down into its most basic form in a way I haven’t seen others come remotely close to and which even my mother can comprehend (with no mathematics at all).  (I recommend this presentation to even those with Ph.D.’s in physics because it is so truly fundamental.)

For the more advanced mathematicians, physicists, and engineers Arieh Ben-Naim does a truly spectacular job of extending ET Jaynes’ work on information theory and statistical mechanics and comes up with a more coherent mathematical theory to conjoin the entropy of physics/statistical mechanics with that of Shannon’s information theory in A Farewell to Entropy: Statistical Thermodynamics Based on Information.

For the advanced readers/researchers interested in more at the intersection of information theory and biology, I’ll also mention that I maintain a list of references, books, and journal articles in a Mendeley group entitled “ITBio: Information Theory, Microbiology, Evolution, and Complexity.”

## How to Sidestep Mathematical Equations in Popular Science Books

In the publishing industry there is a general rule-of-thumb that every mathematical equation included in a book will cut the audience of science books written for a popular audience in half – presumably in a geometric progression. This typically means that including even a handful of equations will give you an effective readership of zero – something no author and certainly no editor or publisher wants.

I suspect that there is a corollary to this that every picture included in the text will help to increase your readership, though possibly not by as proportionally a large amount.

In any case, while reading Melanie Mitchell’s text Complexity: A Guided Tour [Cambridge University Press, 2009] this weekend, I noticed that, in what appears to be a concerted effort to include an equation without technically writing it into the text and to simultaneously increase readership by including a picture, she cleverly used a picture of Boltzmann’s tombstone in Vienna! Most fans of thermodynamics will immediately recognize Boltzmann’s equation for entropy, $S = k log W$, which appears engraved on the tombstone over his bust.

I hope that future mathematicians, scientists, and engineers will keep this in mind and have their tombstones engraved with key formulae to assist future authors in doing the same – hopefully this will help to increase the amount of mathematics that is deemed “acceptable” by the general public.

## Book Review: John Avery’s “Information Theory and Evolution”

Information Theory and Evolution
John Avery
Non-fiction, Popular Science
World Scientific
January 1, 2003
paperback
217

This highly interdisciplinary book discusses the phenomenon of life, including its origin and evolution (and also human cultural evolution), against the background of thermodynamics, statistical mechanics, and information theory. Among the central themes is the seeming contradiction between the second law of thermodynamics and the high degree of order and complexity produced by living systems. This paradox has its resolution in the information content of the Gibbs free energy that enters the biosphere from outside sources, as the author shows. The role of information in human cultural evolution is another focus of the book. One of the final chapters discusses the merging of information technology and biotechnology into a new discipline — bio-information technology.

This is a fantastic book which, for the majority of people, I’d give a five star review. For my own purposes, however, I was expecting far more on the theoretical side of information theory and statistical mechanics as applied to microbiology that it didn’t live up to, so I’m giving it three stars from a purely personal perspective.

I do wish that someone had placed it in my hands and forced me to read it when I was a freshman in college entering the study of biomedical and electrical engineering. It is far more an impressive book at this level and for those in the general public who are interested in the general history of science and philosophy of the topics. The general reader may be somewhat scared by a small amount of mathematics in chapter 4, but there is really no loss of continuity by skimming through most of it. For those looking for a bit more rigor, Avery provides some additional details in appendix A, but for the specialist, the presentation is heavily lacking.

The book opens with a facile but acceptable overview of the history of the development for the theory of evolution whereas most other texts would simply begin with Darwin’s work and completely skip the important philosophical and scientific contributions of Aristotle, Averroes, Condorcet, Linnaeus, Erasmus Darwin, Lamarck, or the debates between Cuvier and St. Hilaire.

For me, the meat of the book was chapters 3-5 and appendix A which collectively covered molecular biology, evolution, statistical mechanics, and a bit of information theory, albeit from a very big picture point of view. Unfortunately the rigor of the presentation and the underlying mathematics were skimmed over all too quickly to accomplish what I had hoped to gain from the text. On the other hand, the individual sections of “suggestions for further reading” throughout the book seem well researched and offer an acceptable launching pad for delving into topics in places where they may be covered more thoroughly.

The final several chapters become a bit more of an overview of philosophy surrounding cultural evolution and information technology which are much better covered and discussed in James Gleick’s recent book The Information.

Overall, Avery has a well laid out outline of the broad array of subjects and covers it all fairly well in an easy to read and engaging style.

• Started book on 07/11/11
• Finished book on 08/14//11
Bookmarked Information Theory and Statistical Mechanics by E. T. Jaynes (Physical Review, 106, 620 – Published 15 May 1957)

Information theory provides a constructive criterion for setting up probability distributions on the basis of partial knowledge, and leads to a type of statistical inference which is called the maximum-entropy estimate. It is the least biased estimate possible on the given information; i.e., it is maximally noncommittal with regard to missing information. If one considers statistical mechanics as a form of statistical inference rather than as a physical theory, it is found that the usual computational rules, starting with the determination of the partition function, are an immediate consequence of the maximum-entropy principle. In the resulting "subjective statistical mechanics," the usual rules are thus justified independently of any physical argument, and in particular independently of experimental verification; whether or not the results agree with experiment, they still represent the best estimates that could have been made on the basis of the information available.

It is concluded that statistical mechanics need not be regarded as a physical theory dependent for its validity on the truth of additional assumptions not contained in the laws of mechanics (such as ergodicity, metric transitivity, equal a priori probabilities, etc.). Furthermore, it is possible to maintain a sharp distinction between its physical and statistical aspects. The former consists only of the correct enumeration of the states of a system and their properties; the latter is a straightforward example of statistical inference.

DOI:https://doi.org/10.1103/PhysRev.106.620

## Global classical solutions of the Boltzmann equation with long-range interactions

Bookmarked Global classical solutions of the Boltzmann equation with long-range interactions (pnas.org)

Finally, after 140 years, Robert Strain and Philip Gressman at the University of Pennsylvania have found a mathematical proof of Boltzmann’s equation, which predicts the motion of gas molecules.