“Quantumbased demons” sound like they'd be at home in 'Stranger Things.'
Tag: entropy
Physicists Hunt For The Big Bang’s Triangles  Quanta Magazine
“The notion that counting more shapes in the sky will reveal more details of the Big Bang is implied in a central principle of quantum physics known as “unitarity.” Unitarity dictates that the probabilities of all possible quantum states of the universe must add up to one, now and forever; thus, information, which is stored in quantum states, can never be lost — only scrambled. This means that all information about the birth of the cosmos remains encoded in its present state, and the more precisely cosmologists know the latter, the more they can learn about the former.”
Introduction to Information Theory  SFI’s Complexity Explorer
Many readers often ask me for resources for delving into the basics of information theory. I hadn’t posted it before, but the Santa Fe Institute recently had an online course Introduction to Information Theory through their Complexity Explorer, which has some other excellent offerings. It included videos, fora, and other resources and was taught by the esteemed physicist and professor Seth Lloyd. There are a number of currently active students still learning and posting there.
Introduction to Information Theory
About the Tutorial:
This tutorial introduces fundamental concepts in information theory. Information theory has made considerable impact in complex systems, and has in part coevolved with complexity science. Research areas ranging from ecology and biology to aerospace and information technology have all seen benefits from the growth of information theory.
In this tutorial, students will follow the development of information theory from bits to modern application in computing and communication. Along the way Seth Lloyd introduces valuable topics in information theory such as mutual information, boolean logic, channel capacity, and the natural relationship between information and entropy.
Lloyd coherently covers a substantial amount of material while limiting discussion of the mathematics involved. When formulas or derivations are considered, Lloyd describes the mathematics such that less advanced math students will find the tutorial accessible. Prerequisites for this tutorial are an understanding of logarithms, and at least a year of highschool algebra.
About the Instructor(s):
Professor Seth Lloyd is a principal investigator in the Research Laboratory of Electronics (RLE) at the Massachusetts Institute of Technology (MIT). He received his A.B. from Harvard College in 1982, the Certificate of Advanced Study in Mathematics (Part III) and an M. Phil. in Philosophy of Science from Cambridge University in 1983 and 1984 under a Marshall Fellowship, and a Ph.D. in Physics in 1988 from Rockefeller University under the supervision of Professor Heinz Pagels.
From 1988 to 1991, Professor Lloyd was a postdoctoral fellow in the High Energy Physics Department at the California Institute of Technology, where he worked with Professor Murray GellMann on applications of information to quantummechanical systems. From 1991 to 1994, he was a postdoctoral fellow at Los Alamos National Laboratory, where he worked at the Center for Nonlinear Systems on quantum computation. In 1994, he joined the faculty of the Department of Mechanical Engineering at MIT. Since 1988, Professor Lloyd has also been an adjunct faculty member at the Sante Fe Institute.
Professor Lloyd has performed seminal work in the fields of quantum computation and quantum communications, including proposing the first technologically feasible design for a quantum computer, demonstrating the viability of quantum analog computation, proving quantum analogs of Shannon’s noisy channel theorem, and designing novel methods for quantum error correction and noise reduction.
Professor Lloyd is a member of the American Physical Society and the Amercian Society of Mechanical Engineers.
Tutorial Team:
Yoav Kallus is an Omidyar Fellow at the Santa Fe Institute. His research at the boundary of statistical physics and geometry looks at how and when simple interactions lead to the formation of complex order in materials and when preferred local order leads to systemwide disorder. Yoav holds a B.Sc. in physics from Rice University and a Ph.D. in physics from Cornell University. Before joining the Santa Fe Institute, Yoav was a postdoctoral fellow at the Princeton Center for Theoretical Science in Princeton University.
How to use Complexity Explorer: How to use Complexity Explore
Prerequisites: At least one year of highschool algebra
Like this tutorial? Donate to help fund more like it
Syllabus
 Introduction
 Forms of Information
 Information and Probability
 Fundamental Formula of Information
 Computation and Logic: Information Processing
 Mutual Information
 Communication Capacity
 Shannon’s Coding Theorem
 The Manifold Things Information Measures
 Homework
Devourer of Encyclopedias: Stanislaw Lem’s “Summa Technologiae”
A review of Summa Technologiae by Stanislaw Lem by David Auerbach from the Los Angeles Review of Books.
Summa Technologiae
AT LAST WE have it in English. Summa Technologiae, originally published in Polish in 1964, is the cornerstone of Stanislaw Lem’s oeuvre, his consummate work of speculative nonfiction. Trained in medicine and biology, Lem synthesizes the current science of the day in ways far ahead of most science fiction of the time.
His subjects, among others, include:
 Virtual reality
 Artificial intelligence
 Nanotechnology and biotechnology
 Evolutionary biology and evolutionary psychology
 Artificial life
 Information theory
 Entropy and thermodynamics
 Complexity theory, probability, and chaos
 Population and ecological catastrophe
 The “singularity” and “transhumanism”
Source: Devourer of Encyclopedias: Stanislaw Lem’s “Summa Technologiae” – The Los Angeles Review of Books
I came across this book review quite serendipitously today via an Auerbach article in Slate, which I’ve bookmarked. I found a copy of the book and have added it to the top of my reading pile. As I’m currently reading an advance reader edition of Sean Carroll’s The Big Picture, I can only imagine how well the two may go together despite being written nearly 60 years apart.
What is Information? by Christoph Adami
Information is a precise concept that can be defined mathematically, but its relationship to what we call "knowledge" is not always made clear. Furthermore, the concepts "entropy" and "information", while deeply related, are distinct and must be used with care, something that is not always achieved in the literature. In this elementary introduction, the concepts of entropy and information are laid out one by one, explained intuitively, but defined rigorously. I argue that a proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
Comments: 19 pages, 2 figures. To appear in Philosophical Transaction of the Royal Society A
Subjects: Adaptation and SelfOrganizing Systems (nlin.AO); Information Theory (cs.IT); Biological Physics (physics.bioph); Quantitative Methods (qbio.QM)
Cite as:arXiv:1601.06176 [nlin.AO] (or arXiv:1601.06176v1 [nlin.AO] for this version)From: Christoph Adami
[v1] Fri, 22 Jan 2016 21:35:44 GMT (151kb,D) [.pdf]
A proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
Forthcoming ITBiorelated book from Sean Carroll: “The Big Picture: On the Origins of Life, Meaning, and the Universe Itself”
In catching up on blogs/reading from the holidays, I’ve noticed that physicist Sean Carroll has a forthcoming book entitled The Big Picture: On the Origins of Life, Meaning, and the Universe Itself (Dutton, May 10, 2016) that will be of interest to many of our readers. One can already preorder the book via Amazon.
Prior to the holidays Sean wrote a blogpost that contains a full overview table of contents, which will give everyone a stronger idea of its contents. For convenience I’ll excerpt it below.
I’ll post a review as soon as a copy arrives, but it looks like a strong new entry in the category of popular science books on information theory, biology and complexity as well as potentially the areas of evolution, the origin of life, and physics in general.
As a side bonus, for those reading this today (1/15/16), I’ll note that Carroll’s 12 part lecture series from The Great Courses The Higgs Boson and Beyond (The Learning Company, February 2015) is 80% off.
THE BIG PICTURE: ON THE ORIGINS OF LIFE, MEANING, AND THE UNIVERSE ITSELF
0. Prologue
* Part One: Cosmos
 1. The Fundamental Nature of Reality
 2. Poetic Naturalism
 3. The World Moves By Itself
 4. What Determines What Will Happen Next?
 5. Reasons Why
 6. Our Universe
 7. Time’s Arrow
 8. Memories and Causes
* Part Two: Understanding
 9. Learning About the World
 10. Updating Our Knowledge
 11. Is It Okay to Doubt Everything?
 12. Reality Emerges
 13. What Exists, and What Is Illusion?
 14. Planets of Belief
 15. Accepting Uncertainty
 16. What Can We Know About the Universe Without Looking at It?
 17. Who Am I?
 18. Abducting God
* Part Three: Essence
 19. How Much We Know
 20. The Quantum Realm
 21. Interpreting Quantum Mechanics
 22. The Core Theory
 23. The Stuff of Which We Are Made
 24. The Effective Theory of the Everyday World
 25. Why Does the Universe Exist?
 26. Body and Soul
 27. Death Is the End
* Part Four: Complexity
 28. The Universe in a Cup of Coffee
 29. Light and Life
 30. Funneling Energy
 31. Spontaneous Organization
 32. The Origin and Purpose of Life
 33. Evolution’s Bootstraps
 34. Searching Through the Landscape
 35. Emergent Purpose
 36. Are We the Point?
* Part Five: Thinking
 37. Crawling Into Consciousness
 38. The Babbling Brain
 39. What Thinks?
 40. The Hard Problem
 41. Zombies and Stories
 42. Are Photons Conscious?
 43. What Acts on What?
 44. Freedom to Choose
* Part Six: Caring
 45. Three Billion Heartbeats
 46. What Is and What Ought to Be
 47. Rules and Consequences
 48. Constructing Goodness
 49. Listening to the World
 50. Existential Therapy
 Appendix: The Equation Underlying You and Me
 Acknowledgments
 Further Reading
 References
 Index
Donald Forsdyke Indicates the Concept of Information in Biology Predates Claude Shannon
As it was published, I had read Kevin Hartnett’s article and interview with Christoph Adami The Information Theory of Life in Quanta Magazine. I recently revisited it and read through the commentary and stumbled upon an interesting quote relating to the history of information in biology:
These two historical references predate Claude Shannon’s mathematical formalization of information in A Mathematical Theory of Communication (The Bell System Technical Journal, 1948) and even Erwin Schrödinger‘s lecture (1943) and subsequent book What is Life (1944).
For those interested in reading more on this historical tidbit, I’ve dug up a copy of the primary Forsdyke reference which first appeared on arXiv (prior to its ultimate publication in History of Psychiatry [.pdf]):
🔖 [1406.1391] ‘A Vehicle of Symbols and Nothing More.’ George Romanes, Theory of Mind, Information, and Samuel Butler by Donald R. Forsdyke [1]
Submitted on 4 Jun 2014 (v1), last revised 13 Nov 2014 (this version, v2)
Abstract: Today’s ‘theory of mind’ (ToM) concept is rooted in the distinction of nineteenth century philosopher William Clifford between ‘objects’ that can be directly perceived, and ‘ejects,’ such as the mind of another person, which are inferred from one’s subjective knowledge of one’s own mind. A founder, with Charles Darwin, of the discipline of comparative psychology, George Romanes considered the minds of animals as ejects, an idea that could be generalized to ‘society as eject’ and, ultimately, ‘the world as an eject’ – mind in the universe. Yet, Romanes and Clifford only vaguely connected mind with the abstraction we call ‘information,’ which needs ‘a vehicle of symbols’ – a material transporting medium. However, Samuel Butler was able to address, in informational terms depleted of theological trappings, both organic evolution and mind in the universe. This view harmonizes with insights arising from modern DNA research, the relative immortality of ‘selfish’ genes, and some startling recent developments in brain research.
Comments: Accepted for publication in History of Psychiatry. 31 pages including 3 footnotes. Based on a lecture given at Santa Clara University, February 28th 2014, at a Bannan Institute Symposium on ‘Science and Seeking: Rethinking the God Question in the Lab, Cosmos, and Classroom.’
The original arXiv article also referenced two lectures which are appended below:
[Original Draft of this was written on December 14, 2015.]
References
The Information Universe Conference
Yesterday, via a notification from Lanyard, I came across a notice for the upcoming conference “The Information Universe” which hits several of the sweet spots for areas involving information theory, physics, the origin of life, complexity, computer science, and microbiology. It is scheduled to occur from October 79, 2015 at the Infoversum Theater in Groningen, The Netherlands.
I’ll let their site speak for itself below, but they already have an interesting line up of speakers including:
Keynote speakers
 Erik Verlinde, Professor Theoretical Physics, University of Amsterdam, Netherlands
 Alex Szalay, Alumni Centennial Professor of Astronomy, The Johns Hopkins University, USA
 Gerard ‘t Hooft, Professor Theoretical Physics, University of Utrecht, Netherlands
 Gregory Chaitin, Professor Mathematics and Computer Science, Federal University of Rio de Janeiro, Brasil
 Charley Lineweaver, Professor Astronomy and Astrophysics, Australian National University, Australia
 Lude Franke, Professor System Genetics, University Medical Center Groningen, Netherlands
Conference synopsis from their homepage:
Additional details about the conference including the participants, program, venue, and registration can also be found at their website.
Videos from the NIMBioS Workshop on Information and Entropy in Biological Systems
Videos from the April 810, 2015, NIMBioS workshop on Information and Entropy in Biological Systems are slowly starting to appear on YouTube.
John Baez, one of the organizers of the workshop, is also going through them and adding some interesting background and links on his Azimuth blog as well for those who are looking for additional details and depth
Additonal resources from the Workshop:
 NIMBios Workshop page
 Participants list
 Workshop Agenda [.pdf download]
 Information and Entropy WordPress site
 YouTube playlist of videos
 Storify archive from the workshop (The storify service shuttered in May 2018. I’ve archived much of the content and links.)
NIMBioS Workshop: Information Theory and Entropy in Biological Systems
Over the next few days, I’ll be maintaining a Storify story covering information related to and coming out of the Information Theory and Entropy Workshop being sponsored by NIMBios at the Unviersity of Tennessee, Knoxville.
For those in attendance or participating by watching the live streaming video (or even watching the video afterthefact), please feel free to use the official hashtag #entropyWS, and I’ll do my best to include your tweets, posts, and material into the story stream for future reference.
For journal articles and papers mentioned in/at the workshop, I encourage everyone to join the Mendeley.com group ITBio: Information Theory, Microbiology, Evolution, and Complexity and add them to the group’s list of papers. Think of it as a collaborative online journal club of sorts.
Those participating in the workshop are also encouraged to take a look at a growing collection of researchers and materials I maintain here. If you have materials or resources you’d like to contribute to the list, please send me an email or include them via the suggestions/submission form or include them in the comments section below.
Resources for Information Theory and Biology
 Researchers
 References and Journal Articles
 Books
 Related Academic, Research Institutes, Societies, Groups, and Organizations
 Conferences, Workshops, and Symposia
 Bionet.InfoTheory (Google Group/Usenet Group)
 #ITBio on Twitter
RSS Feed for BoffoSocko posts tagged with #ITBio
BIRS Workshop on Biological and BioInspired Information Theory  Storify Stream
Over the span of the coming week, I’ll be updating (and archiving) the stream of information coming out of the BIRS Workshop on Biological and BioInspired Information Theory.
Editor’s note: On 12/12/17 Storify announced they would be shutting down. As a result, I’m changing the embedded version of the original data served by Storify for an HTML copy which can be found below:
Edit
BIRS: Biological and BioInspired Information Theory
A 5 Day workshop on Biology and Information Theory hosted by the Banff International Research Station

I know where I’ll be in Oct 2014! Let’s hear it for Biology & Information Theory! https://www.birs.ca/events/2014/5dayworkshops/14w5170 … #ITBio #Banff @andreweckford

. @andreweckford You might be interested in this grouping of research papers: http://www.mendeley.com/groups/2545131/itbio/ … #ITBio #Banff

Wishing I was at the Gene Regulation and Information Theory meeting starting tomorrow http://bit.ly/XnHRZs #ITBio

#ITBio: @andreweckford has a new book on Molecular Communication available Oct 31. http://bit.ly/15uEUzF

Mathematical and Statistical Models for Genetic Coding starts today. http://www.am.hsmannheim.de/genetic_code_2013.php?id=1 … @andreweckford might borrow attendees for BIRS

John Baez has announced a workshop on Entropy & Information in Biological Systems http://johncarlosbaez.wordpress.com/2013/11/02/entropyandinformationinbiologicalsystems/ … #NIMBioS #ITBio @andreweckford

Mathematical Foundations for Information Theory in DiffusionBased Molecular Communications http://bit.ly/1aTVR2c #ITBio

Bill Bialek giving plenary talk “Information flow & order in real biological networks” at Feb 2014 workshop http://mnd.ly/19LQH8f #ITBio

Workshop on Information Theoretic Incentives for Artificial Life http://jhu.md/1lM8tAn #ITBio #ALife14 @alifeofficial @14thALIFE @cxdig

Researchers working in information theory & biology http://jhu.md/1gieQGR #ITBio @andreweckford @ChristophAdami @wbialek @johnhawks

Currently organizing my Banff workshop on bioinformation theory … https://www.birs.ca/events/2014/5dayworkshops/14w5170 …

Last RT: wonder what the weather is going to be like at the end of October for my @BIRS_Math workshop

@JoVanEvery I’m organizing a workshop in Banff in October … hopefully this isn’t a sign of weather to come!

Special issue of Entropy: “Information Theoretic Incentives for Cognitive Systems” http://boffosocko.com/2014/09/19/specialissueofentropyinformationtheoreticincentivesforcognitivesystems/ … #ITBio #CallForSubmissions

Next week @BIRS_Math: Biological and BioInspired Information Theory http://www.birs.ca/events/2014/5dayworkshops/14w5170 …  Live @ http://www.birs.ca/live #bioinformatics

Andrew Eckford (York University), The Landscape http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410270858Eckford.mp4 …

“Not an obvious connection between utility and information, just as there is no obvious connection between energy and entropy” @BIRS_Math

Peter Thomas (Case Western Reserve University), Signal Transduction and Information Theory http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410270940Thomas.mp4 …

Following @andreweckford @BIRS_Math 4 updates/video from Biological & BioInspired Information Theory https://storify.com/ChrisAldrich/biologicalandbioinspiredinformationtheory … @cxdig #ITBio

Live now: Nicolo Michelusi of @USCViterbi on Stochastic Model for Electron Transfer in Bacterial Cables http://www.birs.ca/live #ITBio

Nicolo Michelusi (University of Southern California), A Stochastic Model for Electron Transfer in Bacterial Cables http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410271450Michelusi.mp4 …

“Timing is fundamental … subsumes timevarying concentration channel” @cnmirose @BIRS_Math

Chris Rose (Rutgers University), Molecular Communication Channels: timing vs. payload http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410271538Rose.mp4 …

Standard opening quote of these talks: “I’m not a biologist, but …” @BIRS_Math

Stefan Moser (ETH Zurich), Capacity Bounds of the Memoryless AIGN Channel – a ToyModel for Molecular Communicat… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410271610Moser.mp4 …

Biodiversity, Entropy and ThermodynamicsI’m giving a short 30minute talk at a workshop on Biological and BioInspired Information Theory at the Banff International Research Institute. I’ll say more about the workshop later, but here’s my talk: * Biodiversity, entropy and thermodynamics. Most of the people at this workshop study neurobiology and cell signalling, not evolutionary game theory or…

Weisi Guo (University of Warwick), Communication Envelopes for Molecular Diffusion and Electromagnetic Wave Propag… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410271643Guo.mp4 …


.@Storify stream coming out of @BIRS_Math: Biological & BioInspired Information Theory this week: http://boffosocko.com/2014/10/27/birsworkshoponbiologicalandbioinspiredinformationtheorystorifystream/ …


Terrific introduction of Canada/Banff by Andrew Eckford (York)The Landscape http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410270858Eckford.mp4 …”

@andreweckford “The Landscape” to become a permanent resource for @BIRS_Math participants http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410270858Eckford.mp4 …”

Biological and BioInspired Information Theory workshop videos! http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos … @BIRS_Math

.@ChrisAldrich @andreweckford @Storify @BIRS_Math Sounds like a fascinating workshop on bioinformation theory in Banff.

Mathematical Modeling of Biological Processes (by Avner Friedman & ChiuYen Kao) http://www.amazon.com/MathematicalModelingBiologicalProcessesModelling/dp/3319083139/ref=as_sl_pc_ss_til?tag=compldiges20&linkCode=w01&linkId=3H5F3QNHI6IDWAFQ&creativeASIN=3319083139 …

Toby Berger, winner of the 2002 Shannon award, speaking right now. @BIRS_Math

Naftali Tishby (Hebrew University of Jerusalem), Sensing and acting under information constraints – a principled a… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281032Tishby.mp4 …

“…places such as BIRS and the Banff Centre exist to facilitate the exchange and pursuit of knowledge.” S. Sundaram http://www.birs.ca/testimonials/#testimonial1454 …

We’re going for a hike tomorrow. Many thanks to Lukas at the @ParksCanada info centre in Banff for helpful advice! @BIRS_Math

Behnaam Aazhang (Rice University), RealTime Network Modulation for Intractable Epilepsy http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281337Aazhang.mp4 …

Alexander Dimitrov (Washington State University), Invariant signal processing in auditory biological systems http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281416Dimitrov.mp4 …

Joel Zylberberg (University of Washington), Communicating with noisy signals: lessons learned from the mammalian v… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281450Zylberberg.mp4 …

Robert Schober (Universitat ErlangenNurnberg), Intersymbol interference mitigation in diffusive molecular communi… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281549Schober.mp4 …

Rudolf Rabenstein (FriedrichAlexanderUniversitat ErlangenNurnberg (FAU)), Modelling Molecular Communication Cha… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281627Rabenstein.mp4 …

THis week @BIRS_Math ” Biological and BioInspired Information Theory ” @thebanffcentre #biology #math @NSF

“Your theory might match the data, but the data might be wrong” – Crick @BIRS_Math

So information theory seems to be a big deal in ecology. @BIRS_Math

Tom Schneider (National Institutes of Health), Three Principles of Biological States: Ecology and Cancer http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410290904Schneider.mp4 …

“In biodiversity, the entropy of an ecosystem is the expected … information we gain about an organism by learning its species” @BIRS_Math

Seriously, I’m blown away by this work in information theory in ecology. Huge body of work; I had no idea. @BIRS_Math

.@andreweckford @QuantaMagazine had a nice overview of some of John Harte’s work in September http://bit.ly/1DwIWCD @BIRS_Math

ChanByoung Chae (Yonsei University), Molecular MIMO: From Theory to Practice http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410281705Chae.mp4 …

John Baez (University of California, Riverside), Biodiversity, entropy and thermodynamics http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410291038Baez.mp4 …

I encourage @BIRS_Math attendees at Biological & BioInspired Information Theory to contribute references here: http://bit.ly/1jQwObk

Christoph Adami (Michigan State University), Some InformationTheoretic Musings Concerning the Origin and Evolutio… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410291114Adami.mp4 …

Twitter Lists for @BIRS_Math workshop #Biology #InformationTheory:
#ITBIO: http://bit.ly/1yIfzfJ
#complexitytheory http://bit.ly/1scfZbw 
ICYMI @ChristophAdami had great paper: Informationtheoretic Considerations on Origin of Life on arXiv http://bit.ly/1yIhK2Q @BIRS_Math

Sensing and Acting Under Information ConstraintsI’m having a great time at a workshop on Biological and BioInspired Information Theory in Banff, Canada. You can see videos of the talks online. There have been lots of good talks so far, but this one really blew my mind: * Naftali Tishby, Sensing and acting under information constraints—a principled approach to biology and…

I’m listening to a talk on the origin of life at a workshop on Biological and BioInspired Information Theory. … https://plus.google.com/117562920675666983007/posts/gqFL7XY3quF …

Ilya Nemenman @EmoryUniversity on Predictive information http://bit.ly/1titfOw
Follow his research: http://bit.ly/1tithFW @BIRS_Math 

Ilya Nemenman (Emory University), Predictive information http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410300907Nemenman.mp4 …

Toby Berger (University of Virginia), Neruoscience Applications of GIG Distributions http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410280914Berger.mp4 …

Daniel Polani (University of Hertfordshire), Informational Principles in PerceptionAction Loops http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301038Polani.mp4 …

Fascinating stuff! #Information Theory #Entropy #Thermodynamics #Chemistry #Physics & lastly #Life itself: https://medium.com/thephysicsarxivblog/informationtheoryandtheoriginoflife4cf6b93d156c …

Didn’t get enough information theory & biology this week @BIRS_Math? Apply for NIMBioS workshop in April 2015 http://bit.ly/1yIeiWe #ITBio

Amin Emad (University of Illinois at UrbanaChampaign), Applications of Discrete Mathematics in Bioinformatics http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301329Emad.mp4 …

Paul Bogdan (University of Southern California), Multiscale Analysis Reveals Complex Behavior in Bacteria Populati… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301401Bogdan.mp4 …

Robert Shaw (ProtoLife Inc.), Information and Causality in a ReactionDiffusion System http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301434Shaw.mp4 …

Lubomir Kostal (Institute of Physiology, Academy of Sciences of the Czech Republic), Efficient information transmi… http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301534Kostal.mp4 …

Nima Soltani (Stanford University), Applications of Directed Information to Neuroscience http://www.birs.ca/events/2014/5dayworkshops/14w5170/videos/watch/201410301647Soltani.mp4 …

@lrvarshney I shoulda invited you to this BIRS workshop …

@conservativelez I’m a big fan of your dad’s research & was reminded of much of it via a workshop on Biological Information Theory

@conservativelez Though he may not have been able to attend, he can catch most of the talks online if he’d like https://www.birs.ca/events/2014/5dayworkshops/14w5170 …

Depressed that @BIRS_Math Workshop on Biological & BioInspired Information Theory is over? Relive it here: http://bit.ly/1rF3G4B #ITBio

Kudos @andreweckford, Toby Berger, Peter Thomas, @NGhoussoub, @BIRS_Math & friends on a fantastic workshop! http://bit.ly/1ckttZq

This @BIRS_Math Workshop was biggest thing in #informationtheory & #biology since the Gatlinburg Symposium in 1956. http://bit.ly/1rF4RRr

1/ Everyone I talked to said it was the best workshop they’d ever been to, and they’d like to do a followup workshop @BIRS_Math

2/ There is an amazing diversity of work under the umbrella of “information theory”. @BIRS_Math

3/ Much of this work is outside the IT mainstream, and an issue is that people use different terms for related concepts. @BIRS_Math

4/ Some community building is in order. I think this workshop was a good first step. @BIRS_Math

5/ Many many thanks to @BIRS_Math and huge kudos to @NGhoussoub for excellent service to the Canadian scientific community. BIRS is a gem.

6/ Also many thanks to the participants for their excellent talks, and to @ChrisAldrich for maintaining a Storify.

@andreweckford Perhaps we could consider tying in our friends from @sfiscience? @MelMitchell1 https://www.birs.ca/events/2014/5dayworkshops/14w5170 … @BIRS_Math #complexity

Entropy and Information in Biological Systems (Part 1)John Harte is an ecologist who uses maximum entropy methods to predict the distribution, abundance and energy usage of species. Marc Harper uses information theory in bioinformatics and evolutionary game theory. Harper, Harte and I are organizing a workshop on entropy and information in biological systems, and I’m really excited about it!

Entropy and Information in Biological Systems (Part 2)John Harte, Marc Harper and I are running a workshop! Now you can apply here to attend: * Information and entropy in biological systems, National Institute for Mathematical and Biological Synthesis, Knoxville Tennesee, WednesdayFriday, 810 April 2015. Click the link, read the stuff and scroll down to “CLICK HERE” to apply.

BioInspired Information TheoryThere will be a 5day workshop on Biological and BioInspired Information Theory at BIRS from Sunday the 26th to Friday the 31st of October, 2014. It’s being organized by * Toby Berger (University of Virginia) * Andrew Eckford (York University) * Peter Thomas (Case Western Reserve University) BIRS is the Banff International Research Station,…

Andrew Eckford: The BlogHow does it feel to (co)write a book and hold the finished product in your hands? About like this: Many, many thanks to my excellent coauthors, Tadashi Nakano and Tokuko Haraguchi, for their hard work; thanks to Cambridge for accepting this project and managing it well; and thanks to Satoshi Hiyama for writing a nice blurb.

Andrew Eckford: The BlogYou may have seen our PLOS ONE paper about tabletop molecular communication, which received loads of media coverage. One of the goals of this paper was to show that anyone can do experiments in molecular communication, without any wet labs or expensive apparatus.

Meeting: Gene Regulation and Information Theoryhttp://www.uniulm.de/~sschober/fachgruppe/Meeting2013/Meeting2013.shtml
Information Theory is the New Central Discipline
[My comments posted to the original Facebook post follow below.]
I’m coming to this post a bit late as I’m playing a bit of catch up, but agree with it wholeheartedly.
In particular, applications to molecular biology and medicine are really beginning to come to a heavy boil in just the past five years. This particular year is the progenitor of what appears to be the biggest renaissance for the application of information theory to the area of biology since Hubert Yockey, Henry Quastler, and Robert L. Platzman’s “Symposium on Information Theory in Biology at Gatlinburg, Tennessee” in 1956.
Upcoming/recent conferences/workshops on information theory in biology include:
 BIRS Workshop: Biological and BioInspired Information Theory
 Entropy and Information in Biological Systems at NIMBios
 CECAM Workshop: Entropy in Biomolecular Systems
 ALife breakout session on Information Theoretic Incentives for Artificial Life (which will also spawn off a special issue of the journal Entropy):
At the beginning of September, Christoph Adami posted an awesome and very sound paper on arXiv entitled “Informationtheoretic considerations concerning the origin of life” which truly portends to turn the science of the origin of life on its head.
I’ll note in passing, for those interested, that Claude Shannon’s infamous master’s thesis at MIT (in which he applied Boolean Algebra to electric circuits allowing the digital revolution to occur) and his subsequent “The Theory of Mathematical Communication” were so revolutionary, nearly everyone forgets his MIT Ph.D. Thesis “An Algebra for Theoretical Genetics” which presaged the areas of cybernetics and the current applications of information theory to microbiology and are probably as seminal as Sir R.A Fisher’s applications of statistics to science in general and biology in particular.
For those commenting on the post who were interested in a layman’s introduction to information theory, I recommend John Robinson Pierce’s An Introduction to Information Theory: Symbols, Signals and Noise (Dover has a very inexpensive edition.) After this, one should take a look at Claude Shannon’s original paper. (The MIT Press printing includes some excellent overview by Warren Weaver along with the paper itself.) The mathematics in the paper really aren’t too technical, and most of it should be comprehensible by most advanced high school students.
For those that don’t understand the concept of entropy, I HIGHLY recommend Arieh BenNaim’s book Entropy Demystified The Second Law Reduced to Plain Common Sense with Seven Simulated Games. He really does tear the concept down into its most basic form in a way I haven’t seen others come remotely close to and which even my mother can comprehend (with no mathematics at all). (I recommend this presentation to even those with Ph.D.’s in physics because it is so truly fundamental.)
For the more advanced mathematicians, physicists, and engineers Arieh BenNaim does a truly spectacular job of extending ET Jaynes’ work on information theory and statistical mechanics and comes up with a more coherent mathematical theory to conjoin the entropy of physics/statistical mechanics with that of Shannon’s information theory in A Farewell to Entropy: Statistical Thermodynamics Based on Information.
For the advanced readers/researchers interested in more at the intersection of information theory and biology, I’ll also mention that I maintain a list of references, books, and journal articles in a Mendeley group entitled “ITBio: Information Theory, Microbiology, Evolution, and Complexity.”
Announcement: Special issue of Entropy: “Information Theoretic Incentives for Cognitive Systems”
Editor’s Note: Dr. Christoph Salge asked me to crosspost the following notice from the Entropy site here. I’m sure many of the readers of the blog will find the topic to be of interest.
Dear Colleagues,
In recent years, ideas such as “life is information processing” or “information holds the key to understanding life” have become more common. However, how can information, or more formally Information Theory, increase our understanding of life, or lifelike systems?
Information Theory not only has a profound mathematical basis, but also typically provides an intuitive understanding of processes, such as learning, behavior and evolution terms of information processing.
In this special issue, we are interested in both:
 the informationtheoretic formalization and quantification of different aspects of life, such as driving forces of learning and behavior generation, information flows between neurons, swarm members and social agents, and information theoretic aspects of evolution and adaptation, and
 the simulation and creation of lifelike systems with previously identified principles and incentives.
Topics with relation to artificial and natural systems:
 information theoretic intrinsic motivations
 information theoretic quantification of behavior
 information theoretic guidance of artificial evolution
 information theoretic guidance of selforganization
 information theoretic driving forces behind learning
 information theoretic driving forces behind behavior
 information theory in swarms
 information theory in social behavior
 information theory in evolution
 information theory in the brain
 information theory in systemenvironment distinction
 information theory in the perception action loop
 information theoretic definitions of life
Submission
Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. Papers will be published continuously (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.
Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are refereed through a peerreview process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peerreviewed Open Access monthly journal published by MDPI.
Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1200 CHF (Swiss Francs).
Deadline for manuscript submissions: 28 February 2015
Special Issue Editors
Guest Editor
Dr. Christoph Salge
Adaptive Systems Research Group,University of Hertfordshire, College Lane, AL10 9AB Hatfield, UK
Website: http://homepages.stca.herts.ac.uk/~cs08abi
EMail: c.salge@herts.ac.uk
Phone: +44 1707 28 4490
Interests: Intrinsic Motivation (Empowerment); SelfOrganization; Guided SelfOrganization; InformationTheoretic Incentives for Social Interaction; InformationTheoretic Incentives for Swarms; Information Theory and Computer Game AI
Guest Editor
Dr. Georg Martius
Cognition and Neurosciences, Max Planck Institute for Mathematics in the Sciences Inselstrasse 22, 04103 Leipzig, Germany
Website: http://www.mis.mpg.de/jjost/members/georgmartius.html
EMail: martius@mis.mpg.de
Phone: +49 341 9959 545
Interests: Autonomous Robots; SelfOrganization; Guided SelfOrganization; Information Theory; Dynamical Systems; Machine Learning; Neuroscience of Learning; Optimal Control
Guest Editor
Dr. Keyan GhaziZahedi
Information Theory of Cognitive Systems, Max Planck Institute for Mathematics in the Sciences Inselstrasse 22, 04103 Leipzig, Germany
Website: http://personalhomepages.mis.mpg.de/zahedi
EMail: zahedi@mis.mpg.de
Phone: +49 341 9959 535
Interests: Embodied Artificial Intelligence; Information Theory of the Sensorimotor Loop; Dynamical Systems; Cybernetics; Selforganisation; Synaptic plasticity; Evolutionary Robotics
Guest Editor
Dr. Daniel Polani
Department of Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK
Website: http://homepages.feis.herts.ac.uk/~comqdp1/
EMail: d.polani@herts.ac.uk
Interests: artificial intelligence; artificial life; information theory for intelligent information processing; sensor Evolution; collective and multiagent systems
Workshop Announcement: Entropy and Information in Biological Systems
I rarely, if ever, reblog anything here, but this particular post from John Baez’s blog Azimuth is so ontopic, that attempting to embellish it seems silly.
Entropy and Information in Biological Systems (Part 2)
John Harte, Marc Harper and I are running a workshop! Now you can apply here to attend:
• Information and entropy in biological systems, National Institute for Mathematical and Biological Synthesis, Knoxville Tennesee, WednesdayFriday, 810 April 2015.
Click the link, read the stuff and scroll down to “CLICK HERE” to apply. The deadline is 12 November 2014.
Financial support for travel, meals, and lodging is available for workshop attendees who need it. We will choose among the applicants and invite 1015 of them.
The idea
Information theory and entropy methods are becoming powerful tools in biology, from the level of individual cells, to whole ecosystems, to experimental design, modelbuilding, and the measurement of biodiversity. The aim of this investigative workshop is to synthesize different ways of applying these concepts to help systematize and unify work in biological systems. Early attempts at “grand syntheses” often misfired, but applications of information theory and entropy to specific highly focused topics in biology have been increasingly successful. In ecology, entropy maximization methods have proven successful in predicting the distribution and abundance of species. Entropy is also widely used as a measure of biodiversity. Work on the role of information in game theory has shed new light on evolution. As a population evolves, it can be seen as gaining information about its environment. The principle of maximum entropy production has emerged as a fascinating yet controversial approach to predicting the behavior of biological systems, from individual organisms to whole ecosystems. This investigative workshop will bring together top researchers from these diverse fields to share insights and methods and address some longstanding conceptual problems.So, here are the goals of our workshop:
 To study the validity of the principle of Maximum Entropy Production (MEP), which states that biological systems – and indeed all open, nonequilibrium systems – act to produce entropy at the maximum rate.
 To familiarize all the participants with applications to ecology of the MaxEnt method: choosing the probabilistic hypothesis with the highest entropy subject to the constraints of our data. We will compare MaxEnt with competing approaches and examine whether MaxEnt provides a sufficient justification for the principle of MEP.
 To clarify relations between known characterizations of entropy, the use of entropy as a measure of biodiversity, and the use of MaxEnt methods in ecology.
 To develop the concept of evolutionary games as “learning” processes in which information is gained over time.
 To study the interplay between information theory and the thermodynamics of individual cells and organelles.
For more details, go here.
If you’ve got colleagues who might be interested in this, please let them know. You can download a PDF suitable for printing and putting on a bulletin board by clicking on this:
CECAM Workshop: “Entropy in Biomolecular Systems”
On Friday, I had an excellent and stimulating conversation with Arieh BenNaim about his recent writing and work, and he mentioned in passing that he had been invited to a conference relating to entropy and biology in Vienna. A quick websearch found it quickly, and not having heard about it myself yet, I thought I’d pass it along to others who are regular readers and interested in the area.
The workshop on “Entropy in Biomolecular Systems” is being hosted by the Centre Européen de Calcul Atomique et Moléculaire (CECAM)
Location: DACAM, Max F. Perutz Laboratories, University of Vienna, Dr. Bohrgasse 9, A1030, Vienna, Austria
Dates: May 14, 2014 to May 17, 2014
The workshop is being organized by:
 Richard Henchman (University of Manchester, United Kingdom)
 Bojan Zagrovic (University of Vienna, Austria)
 Michel Cuendet (Swiss Institute of Bioinformatics, Lausanne, Switzerland and Weill Cornell Medical College, New York, USA)
 Chris Oostenbrink (University of Natural Resources and Life Sciences, Austria)
It’s being supported by CECAM, the European Research Council, and the Royal Society of Chemistry’s Statistical Mechanics and Thermodynamics Group.
I’ll note that the registration deadline is on April 21 with a payment deadline of April 30, so check in quickly if you haven’t already.
The summary from the workshop website states:
This workshop brings together the world’s experts to address the challenges of determining the entropy of biomolecular systems, either by experiment or computer simulation. Entropy is one the main driving forces for any biological process such as binding, folding, partitioning and reacting. Our deficient understandng of entropy, however, means that such important processes remain controversial and only partially understood. Contributions of water, ions, cofactors, and biomolecular flexibility are actively examined but yet to be resolved. The stateoftheart of each entropy method will be presented and explained, highlighting its capabilities and deficiencies. This will be followed by intensive discussion on the main areas that need improving, leading suitable actions and collaborations to address the main biological and industrial questions.
Further details on the workshop can be found on the CECAM website.
As always, details on other upcoming workshops and conferences relating to information theory and biology can be found on our ITBio Conferences/Workshops page.