Information is a precise concept that can be defined mathematically, but its relationship to what we call "knowledge" is not always made clear. Furthermore, the concepts "entropy" and "information", while deeply related, are distinct and must be used with care, something that is not always achieved in the literature. In this elementary introduction, the concepts of entropy and information are laid out one by one, explained intuitively, but defined rigorously. I argue that a proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
Comments: 19 pages, 2 figures. To appear in Philosophical Transaction of the Royal Society A
Subjects: Adaptation and Self-Organizing Systems (nlin.AO); Information Theory (cs.IT); Biological Physics (physics.bio-ph); Quantitative Methods (q-bio.QM)
Cite as:arXiv:1601.06176 [nlin.AO] (or arXiv:1601.06176v1 [nlin.AO] for this version)From: Christoph Adami
[v1] Fri, 22 Jan 2016 21:35:44 GMT (151kb,D) [.pdf]
Tag: ITBio
Forthcoming ITBio-related book from Sean Carroll: “The Big Picture: On the Origins of Life, Meaning, and the Universe Itself”
Prior to the holidays Sean wrote a blogpost that contains a full overview table of contents, which will give everyone a stronger idea of its contents. For convenience I’ll excerpt it below.
I’ll post a review as soon as a copy arrives, but it looks like a strong new entry in the category of popular science books on information theory, biology and complexity as well as potentially the areas of evolution, the origin of life, and physics in general.
As a side bonus, for those reading this today (1/15/16), I’ll note that Carroll’s 12 part lecture series from The Great Courses The Higgs Boson and Beyond (The Learning Company, February 2015) is 80% off.
THE BIG PICTURE: ON THE ORIGINS OF LIFE, MEANING, AND THE UNIVERSE ITSELF
0. Prologue
* Part One: Cosmos
- 1. The Fundamental Nature of Reality
- 2. Poetic Naturalism
- 3. The World Moves By Itself
- 4. What Determines What Will Happen Next?
- 5. Reasons Why
- 6. Our Universe
- 7. Time’s Arrow
- 8. Memories and Causes
* Part Two: Understanding
- 9. Learning About the World
- 10. Updating Our Knowledge
- 11. Is It Okay to Doubt Everything?
- 12. Reality Emerges
- 13. What Exists, and What Is Illusion?
- 14. Planets of Belief
- 15. Accepting Uncertainty
- 16. What Can We Know About the Universe Without Looking at It?
- 17. Who Am I?
- 18. Abducting God
* Part Three: Essence
- 19. How Much We Know
- 20. The Quantum Realm
- 21. Interpreting Quantum Mechanics
- 22. The Core Theory
- 23. The Stuff of Which We Are Made
- 24. The Effective Theory of the Everyday World
- 25. Why Does the Universe Exist?
- 26. Body and Soul
- 27. Death Is the End
* Part Four: Complexity
- 28. The Universe in a Cup of Coffee
- 29. Light and Life
- 30. Funneling Energy
- 31. Spontaneous Organization
- 32. The Origin and Purpose of Life
- 33. Evolution’s Bootstraps
- 34. Searching Through the Landscape
- 35. Emergent Purpose
- 36. Are We the Point?
* Part Five: Thinking
- 37. Crawling Into Consciousness
- 38. The Babbling Brain
- 39. What Thinks?
- 40. The Hard Problem
- 41. Zombies and Stories
- 42. Are Photons Conscious?
- 43. What Acts on What?
- 44. Freedom to Choose
* Part Six: Caring
- 45. Three Billion Heartbeats
- 46. What Is and What Ought to Be
- 47. Rules and Consequences
- 48. Constructing Goodness
- 49. Listening to the World
- 50. Existential Therapy
- Appendix: The Equation Underlying You and Me
- Acknowledgments
- Further Reading
- References
- Index
Donald Forsdyke Indicates the Concept of Information in Biology Predates Claude Shannon
These two historical references predate Claude Shannon’s mathematical formalization of information in A Mathematical Theory of Communication (The Bell System Technical Journal, 1948) and even Erwin Schrödinger‘s lecture (1943) and subsequent book What is Life (1944).
For those interested in reading more on this historical tidbit, I’ve dug up a copy of the primary Forsdyke reference which first appeared on arXiv (prior to its ultimate publication in History of Psychiatry [.pdf]):
🔖 [1406.1391] ‘A Vehicle of Symbols and Nothing More.’ George Romanes, Theory of Mind, Information, and Samuel Butler by Donald R. Forsdyke [1]
Submitted on 4 Jun 2014 (v1), last revised 13 Nov 2014 (this version, v2)
Abstract: Today’s ‘theory of mind’ (ToM) concept is rooted in the distinction of nineteenth century philosopher William Clifford between ‘objects’ that can be directly perceived, and ‘ejects,’ such as the mind of another person, which are inferred from one’s subjective knowledge of one’s own mind. A founder, with Charles Darwin, of the discipline of comparative psychology, George Romanes considered the minds of animals as ejects, an idea that could be generalized to ‘society as eject’ and, ultimately, ‘the world as an eject’ – mind in the universe. Yet, Romanes and Clifford only vaguely connected mind with the abstraction we call ‘information,’ which needs ‘a vehicle of symbols’ – a material transporting medium. However, Samuel Butler was able to address, in informational terms depleted of theological trappings, both organic evolution and mind in the universe. This view harmonizes with insights arising from modern DNA research, the relative immortality of ‘selfish’ genes, and some startling recent developments in brain research.
Comments: Accepted for publication in History of Psychiatry. 31 pages including 3 footnotes. Based on a lecture given at Santa Clara University, February 28th 2014, at a Bannan Institute Symposium on ‘Science and Seeking: Rethinking the God Question in the Lab, Cosmos, and Classroom.’
The original arXiv article also referenced two lectures which are appended below:
http://www.youtube.com/watch?v=a3yNbTUCPd4
[Original Draft of this was written on December 14, 2015.]
References
Quantum Biological Information Theory by Ivan B. Djordjevic | Springer
From the publisher’s website:
This book is a self-contained, tutorial-based introduction to quantum information theory and quantum biology. It serves as a single-source reference to the topic for researchers in bioengineering, communications engineering, electrical engineering, applied mathematics, biology, computer science, and physics. The book provides all the essential principles of the quantum biological information theory required to describe the quantum information transfer from DNA to proteins, the sources of genetic noise and genetic errors as well as their effects.
- Integrates quantum information and quantum biology concepts;
- Assumes only knowledge of basic concepts of vector algebra at undergraduate level;
- Provides a thorough introduction to basic concepts of quantum information processing, quantum information theory, and quantum biology;
- Includes in-depth discussion of the quantum biological channel modelling, quantum biological channel capacity calculation, quantum models of aging, quantum models of evolution, quantum models on tumor and cancer development, quantum modeling of bird navigation compass, quantum aspects of photosynthesis, quantum biological error correction.
Source: Quantum Biological Information Theory | Ivan B. Djordjevic | Springer
I’ll note that it looks like it also assumes some reasonable facility with quantum mechanics in addition to the material listed above.
Springer also has a downloadable copy of the preface and a relatively extensive table of contents for those looking for a preview. Dr. Djordjevic has been added to the ever growing list of researchers doing work at the intersection of information theory and biology.
The Information Theory of Life | Quanta Magazine
The Information Theory of Life: The polymath Christoph Adami is investigating life’s origins by reimagining living things as self-perpetuating information strings.
Winter Q-BIO Quantitative Biology Meeting February 15-18, 2016
The Winter Q-BIO Quantitative Biology Meeting is coming up at the Sheraton Waikiki in Oahu, HI, USA
A predictive understanding of living systems is a prerequisite for designed manipulation in bioengineering and informed intervention in medicine. Such an understanding requires quantitative measurements, mathematical analysis, and theoretical abstraction. The advent of powerful measurement technologies and computing capacity has positioned biology to drive the next scientific revolution. A defining goal of Quantitative Biology (qBIO) is the development of general principles that arise from networks of interacting elements that initially defy conceptual reasoning. The use of model organisms for the discovery of general principles has a rich tradition in biology, and at a fundamental level the philosophy of qBIO resonates with most molecular and cell biologists. New challenges arise from the complexity inherent in networks, which require mathematical modeling and computational simulation to develop conceptual “guideposts” that can be used to generate testable hypotheses, guide analyses, and organize “big data.”
The Winter q-bio meeting welcomes scientists and engineers who are interested in all areas of q-bio. For 2016, the meeting will be hosted at the Sheraton Waikiki, which is located in Honolulu, on the island of Oahu. The resort is known for its breathtaking oceanfront views, a first-of-its-kind recently opened “Superpool” and many award-winning dining venues. Registration and accommodation information can be found via the links at the top of the page.
Breaking the code | The Economist
Brief book overview of Matthew Cobb's "Life’s Greatest Secret" from The Economist.
Life’s Greatest Secret: The Story of the Race to Crack the Genetic Code. By Matthew Cobb. Basic Books; 434 pages; $29.99. Profile Books; £25.
Molecular Programming Project
“The Molecular Programming Project aims to develop computer science principles for programming information-bearing molecules like DNA and RNA to create artificial biomolecular programs of similar complexity. Our long-term vision is to establish molecular programming as a subdiscipline of computer science — one that will enable a yet-to-be imagined array of applications from chemical circuitry for interacting with biological molecules to nanoscale computing and molecular robotics.”
Source: MPP: Home
Why Information Grows: The Evolution of Order, from Atoms to Economies
From the book description:
“What is economic growth? And why, historically, has it occurred in only a few places? Previous efforts to answer these questions have focused on institutions, geography, finances, and psychology. But according to MIT’s antidisciplinarian César Hidalgo, understanding the nature of economic growth demands transcending the social sciences and including the natural sciences of information, networks, and complexity. To understand the growth of economies, Hidalgo argues, we first need to understand the growth of order.
At first glance, the universe seems hostile to order. Thermodynamics dictates that over time, order–or information–will disappear. Whispers vanish in the wind just like the beauty of swirling cigarette smoke collapses into disorderly clouds. But thermodynamics also has loopholes that promote the growth of information in pockets. Our cities are pockets where information grows, but they are not all the same. For every Silicon Valley, Tokyo, and Paris, there are dozens of places with economies that accomplish little more than pulling rocks off the ground. So, why does the US economy outstrip Brazil’s, and Brazil’s that of Chad? Why did the technology corridor along Boston’s Route 128 languish while Silicon Valley blossomed? In each case, the key is how people, firms, and the networks they form make use of information.
Seen from Hidalgo’s vantage, economies become distributed computers, made of networks of people, and the problem of economic development becomes the problem of making these computers more powerful. By uncovering the mechanisms that enable the growth of information in nature and society, Why Information Grows lays bear the origins of physical order and economic growth. Situated at the nexus of information theory, physics, sociology, and economics, this book propounds a new theory of how economies can do, not just more, but more interesting things.”
The Information Universe Conference
I’ll let their site speak for itself below, but they already have an interesting line up of speakers including:
Keynote speakers
- Erik Verlinde, Professor Theoretical Physics, University of Amsterdam, Netherlands
- Alex Szalay, Alumni Centennial Professor of Astronomy, The Johns Hopkins University, USA
- Gerard ‘t Hooft, Professor Theoretical Physics, University of Utrecht, Netherlands
- Gregory Chaitin, Professor Mathematics and Computer Science, Federal University of Rio de Janeiro, Brasil
- Charley Lineweaver, Professor Astronomy and Astrophysics, Australian National University, Australia
- Lude Franke, Professor System Genetics, University Medical Center Groningen, Netherlands
Conference synopsis from their homepage:
Additional details about the conference including the participants, program, venue, and registration can also be found at their website.
Videos from the NIMBioS Workshop on Information and Entropy in Biological Systems
John Baez, one of the organizers of the workshop, is also going through them and adding some interesting background and links on his Azimuth blog as well for those who are looking for additional details and depth
Additonal resources from the Workshop:
- NIMBios Workshop page
- Participants list
- Workshop Agenda [.pdf download]
- Information and Entropy WordPress site
- YouTube playlist of videos
- Storify archive from the workshop (The storify service shuttered in May 2018. I’ve archived much of the content and links.)
https://www.youtube.com/playlist?list=PLRyq_4VPZ9g-3869ozbY_eEp6jZhWL0UE
Popular Science Books on Information Theory, Biology, and Complexity
Introductory / General Readership / Popular Science Books
These books are written on a generally non-technical level and give a broad overview of their topics with occasional forays into interesting or intriguing subtopics. They include little, if any, mathematical equations or conceptualization. Typically, any high school student should be able to read, follow, and understand the broad concepts behind these books. Though often non-technical, these texts can give some useful insight into the topics at hand, even for the most advanced researchers.
Complexity: A Guided Tour by Melanie Mitchell (review)
Possibly one of the best places to start, this text gives a great overview of most of the major areas of study related to these fields.
Entropy Demystified: The Second Law Reduced to Plain Common Sense by Arieh Ben-Naim
One of the best books on the concept of entropy out there. It can be read even by middle school students with no exposure to algebra and does a fantastic job of laying out the conceptualization of how entropy underlies large areas of the broader subject. Even those with Ph.D.’s in statistical thermodynamics can gain something useful from this lovely volume.
The Information: A History, a Theory, a Flood by James Gleick (review)
A relatively recent popular science volume covering various conceptualizations of what information is and how it’s been dealt with in science and engineering. Though it has its flaws, its certainly a good introduction to the beginner, particularly with regard to history.
The Origin of Species by Charles Darwin
One of the most influential pieces of writing known to man, this classical text is the basis from which major strides in biology have been made as a result. A must read for everyone on the planet.
Information, Entropy, Life and the Universe: What We Know and What We Do Not Know by Arieh Ben-Naim
Information Theory and Evolution by John Avery
The Touchstone of Life: Molecular Information, Cell Communication, and the Foundations of Life by Werner R. Loewenstein (review)
Information Theory, Evolution, and the Origin of Life by Hubert P. Yockey
The four books above have a significant amount of overlap. Though one could read all of them, I recommend that those pressed for time choose Ben-Naim first. As I write this I’ll note that Ben-Naim’s book is scheduled for release on May 30, 2015, but he’s been kind enough to allow me to read an advance copy while it was in process; it gets my highest recommendation in its class. Loewenstein covers a bit more than Avery who also has a more basic presentation. Most who continue with the subject will later come across Yockey’s Information Theory and Molecular Biology which is similar to his text here but written at a slightly higher level of sophistication. Those who finish at this level of sophistication might want to try Yockey third instead.
The Red Queen: Sex and the Evolution of Human Nature by Matt Ridley
Grammatical Man: Information, Entropy, Language, and Life by Jeremy Campbell
Life’s Ratchet: How Molecular Machines Extract Order from Chaos by Peter M. Hoffmann
Complexity: The Emerging Science at the Edge of Order and Chaos by M. Mitchell Waldrop
The Big Picture: On the Origins of Life, Meaning, and the Universe Itself (Dutton, May 10, 2016)
In the coming weeks/months, I’ll try to continue putting recommended books on the remainder of the rest of the spectrum, the balance of which follows in outline form below. As always, I welcome suggestions and recommendations based on others’ experiences as well. If you’d like to suggest additional resources in any of the sections below, please do so via our suggestion box. For those interested in additional resources, please take a look at the ITBio Resources page which includes information about related research groups; references and journal articles; academic, research institutes, societies, groups, and organizations; and conferences, workshops, and symposia.
Lower Level Undergraduate
These books are written at a level that can be grasped and understood by most with a freshmen or sophomore university level. Coursework in math, science, and engineering will usually presume knowledge of calculus, basic probability theory, introductory physics, chemistry, and basic biology.
Upper Level Undergraduate
These books are written at a level that can be grasped and understood by those at a junior or senor university level. Coursework in math, science, and engineering may presume knowledge of probability theory, differential equations, linear algebra, complex analysis, abstract algebra, signal processing, organic chemistry, molecular biology, evolutionary theory, thermodynamics, advanced physics, and basic information theory.
Graduate Level
These books are written at a level that can be grasped and understood by most working at the level of a master’s level at most universities. Coursework presumes all the previously mentioned classes, though may require a higher level of sub-specialization in one or more areas of mathematics, physics, biology, or engineering practice. Because of the depth and breadth of disciplines covered here, many may feel the need to delve into areas outside of their particular specialization.
Nicolas Perony: Puppies! Now that I’ve got your attention, complexity theory | TED
[ted id=1916]
NIMBioS Workshop: Information Theory and Entropy in Biological Systems
For those in attendance or participating by watching the live streaming video (or even watching the video after-the-fact), please feel free to use the official hashtag #entropyWS, and I’ll do my best to include your tweets, posts, and material into the story stream for future reference.
For journal articles and papers mentioned in/at the workshop, I encourage everyone to join the Mendeley.com group ITBio: Information Theory, Microbiology, Evolution, and Complexity and add them to the group’s list of papers. Think of it as a collaborative online journal club of sorts.
Those participating in the workshop are also encouraged to take a look at a growing collection of researchers and materials I maintain here. If you have materials or resources you’d like to contribute to the list, please send me an email or include them via the suggestions/submission form or include them in the comments section below.
Resources for Information Theory and Biology
- Researchers
- References and Journal Articles
- Books
- Related Academic, Research Institutes, Societies, Groups, and Organizations
- Conferences, Workshops, and Symposia
- Bionet.Info-Theory (Google Group/Usenet Group)
- #ITBio on Twitter
RSS Feed for BoffoSocko posts tagged with #ITBio
Probability Models for DNA Sequence Evolution
I’ll also give him a shout out for being a mathematician with a fledgling blog: Rick’s Ramblings.