Von J.‐M. Lehn. VCH Verlagsgesellschaft, Weinheim, 1995. 271 S., geb. 128.00 DM/Broschur 58.00 DM. ‐ ISBN 3‐527‐29312‐4/3‐527‐29311‐6 https://doi.org/10.1002/ange.19951072130
In the early ’90s, Hank Rowan gave $100 million to a university in New Jersey, an act of extraordinary generosity that helped launch the greatest explosion in educational philanthropy since the days of Andrew Carnegie and the Rockefellers. But Rowan gave his money to Glassboro State University, a tiny, almost bankrupt school in South Jersey, while almost all of the philanthropists who followed his lead made their donations to elite schools such as Harvard and Yale. Why did no one follow Rowan’s example?
“My Little Hundred Million” is the third part of Revisionist History’s educational miniseries. It looks at the hidden ideologies behind giving and how a strange set of ideas has hijacked educational philanthropy.
The key idea laid out stunningly here is strong links versus weak links.
I’m generally flabbergasted by the general idea proposed here and will have to do some more research in the near future to play around further with the ideas presented. Fortunately, in addition to the education specific idea presented, Gladwell also comes up with an additional few examples in sports by using the differences between soccer and basketball to show the subtle differences.
If he and his lab aren’t aware of the general concept, I would recommend this particular podcast and the concept of strong and weak links to César Hidalgo (t) who might actually have some troves of economics data to use to play around with some general modeling to expand upon these ideas. I’ve been generally enamored of Hidalgo’s general thesis about the overall value of links as expressed in Why Information Grows: The Evolution of Order, from Atoms to Economies1. I often think of it with relation to political economies and how the current administration seems to be (often quietly) destroying large amounts of value by breaking down a variety of economic, social, and political links within the United States as well as between our country and others.
I wonder if the additional ideas about the differences between strong and weak links might further improve these broader ideas. The general ideas behind statistical mechanics and statistics make me think that Gladwell, like Hidalgo, is certainly onto a strong idea which can be continued to be refined to improve billions of lives. I’ll have to start some literature searches now…
Paleoclimate records are extremely rich sources of information about the past history of the Earth system. We take an information-theoretic approach to analyzing data from the WAIS Divide ice core, the longest continuous and highest-resolution water isotope record yet recovered from Antarctica. We use weighted permutation entropy to calculate the Shannon entropy rate from these isotope measurements, which are proxies for a number of different climate variables, including the temperature at the time of deposition of the corresponding layer of the core. We find that the rate of information production in these measurements reveals issues with analysis instruments, even when those issues leave no visible traces in the raw data. These entropy calculations also allow us to identify a number of intervals in the data that may be of direct relevance to paleoclimate interpretation, and to form new conjectures about what is happening in those intervals—including periods of abrupt climate change.
Saw reference in Predicting unpredictability: Information theory offers new way to read ice cores 
At two miles long and five inches in diameter, the West Antarctic Ice Sheet Divide (WAIS) ice core is a tangible record of the last 68,000 years of our planet's climate.
Hundreds of researchers in a collaborative project called "It from Qubit" say space and time may spring up from the quantum entanglement of tiny bits of information.
A Problem with Transcripts
In the past few weeks, I’ve seen dozens of news outlets publish multi-paragraph excerpts of speeches from Donald Trump and have been appalled that I was unable to read them in any coherent way. I could not honestly follow or discern any coherent thought or argument in the majority of them. I was a bit shocked because in listening to him, he often sounds like he has some kind of point, though he seems to be spouting variations on one of ten one-liners he’s been using for over a year now. There’s apparently a flaw in our primal reptilian brains that seems to be tricking us into thinking that there’s some sort of substance in his speech when there honestly is none. I’m going to have to spend some time reading more on linguistics and cognitive neuroscience. Maybe Stephen Pinker knows of an answer?
The situation got worse this week as I turned to news sources for fact-checking of the recent presidential debate. While it’s nice to have web-based annotation tools like Genius and Hypothes.is to mark up these debates, it becomes another thing altogether to understand the meaning of what’s being said in order to actually attempt to annotate it. I’ve included some links so that readers can attempt the exercise for themselves.
Recent transcripts (some with highlights/annotations):
- Fact Check: Trump And Clinton Debate For The First Time
- The first Trump-Clinton presidential debate transcript, annotated
- Transcript: The most important exchanges of the presidential debate, annotated
- Transcript: Donald Trump’s full immigration speech, annotated
- Transcript: Here are words Trump just used to talk about ‘the cyber’
Doubletalk and Doublespeech
It’s been a while since Americans were broadly exposed to actual doubletalk. For the most part our national experience with it has been a passing curiosity highlighted by comedians.
n. (NORTH AMERICAN)
a deliberately unintelligible form of speech in which inappropriate, invented or nonsense syllables are combined with actual words. This type of speech is commonly used to give the appearance of knowledge and thereby confuse, amuse, or entertain the speaker’s audience.
another term for doublespeak
see also n. doubletalk 
Since the days of vaudeville (and likely before), comedians have used doubletalk to great effect on stage, in film, and on television. Some comedians who have historically used the technique as part of their acts include Al Kelly, Cliff Nazarro, Danny Kaye, Gary Owens, Irwin Corey, Jackie Gleason, Sid Caesar, Stanley Unwin, and Reggie Watts. I’m including some short video clips below as examples.
A well-known, but foreshortened, form of it was used by Dana Carvey in his Saturday Night Live performances caricaturizing George H.W. Bush by using a few standard catch phrases with pablum in between: “Not gonna do it…”, “Wouldn’t be prudent at this juncture”, and “Thousand Points of Light…”. These snippets in combination with some creative hand gestures (pointing, lacing fingers together), along with a voice melding of Mr. Rogers and John Wayne were the simple constructs that largely transformed a diminutive comedian convincingly into a president.
Doubletalk also has a more “educated” sibling known as technobabble. Engineers are sure to recall a famous (and still very humorous) example of both doubletalk and technobabble in the famed description of the Turboencabulator. (See also, the short videos below.)
Doubletalk comedy examples
Al Kelly on Ernie Kovaks
Rockwell Turbo Encabulator Version 2
And of course doubletalk and technobabble have closely related cousins named doublespeak and politicobabble. These are far more dangerous than the others because they move over the line of comedy into seriousness and are used by people who make decisions effecting hundreds of thousands to millions, if not billions, of people on the planet. I’m sure an archeo-linguist might be able to discern where exactly politicobabble emerged and managed to evolve into a non-comedic form of speech which people manage to take far more seriously than its close ancestors. One surely suspects some heavy influence from George Orwell’s corpus of work:
While politicobabble is nothing new, I did find a very elucidating passage from the 1992 U.S. Presidential Election cycle which seems to be a major part of the Trump campaign playbook:
In the continuation of the article, Jacobs goes on to give a variety of examples of the term as well as a “translation” guide for some of the common politicobabble words from that particular election. I’ll leave it to the capable hands of others (perhaps in the comments, below?) to come up with the translation guide for our current political climate.
The interesting evolutionary change I’ll note for the current election cycle is that Trump hasn’t delved into any depth on any of his themes to offend anyone significantly enough. This has allowed him to stay with the dozen or so themes he started out using and therefore hasn’t needed to change them as in campaigns of old.
Filling in the Blanks
These forms of pseudo-speech area all meant to fool us into thinking that something of substance is being discussed and that a conversation is happening, when in fact, nothing is really being communicated at all. Most of the intended meaning and reaction to such speech seems to stem from the demeanor of the speaker as well as, in some part, to the reaction of the surrounding interlocutor and audience. In reading Donald Trump transcripts, an entirely different meaning (or lack thereof) is more quickly realized as the surrounding elements which prop up the narrative have been completely stripped away. In a transcript version, gone is the hypnotizing element of the crowd which is vehemently sure that the emperor is truly wearing clothes.
In many of these transcripts, in fact, I find so little is being said that the listener is actually being forced to piece together the larger story in their head. Being forced to fill in the blanks in this way leaves too much of the communication up to the listener who isn’t necessarily engaged at a high level. Without more detail or context to understand what is being communicated, the listener is far more likely to fill in the blanks to fit a story that doesn’t create any cognitive dissonance for themselves — in part because Trump is usually smiling and welcoming towards his adoring audiences.
One will surely recall that Trump even wanted Secretary Clinton to be happy during the debate when he said, “Now, in all fairness to Secretary Clinton — yes, is that OK? Good. I want you to be very happy. It’s very important to me.” (This question also doubles as an example of a standard psychological sales tactic of attempting to get the purchaser to start by saying ‘yes’ as a means to keep them saying yes while moving them towards making a purchase.)
His method of communicating by leaving large holes in his meaning reminds me of the way our brain smooths out information as indicated in this old internet meme :
I cdn’uolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg: the phaonmneel pweor of the hmuan mnid. Aoccdrnig to a rseearch taem at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer be in the rghit pclae. The rset can be a taotl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Scuh a cdonition is arpppoiatrely cllaed typoglycemia.
I’m also reminded of the biases and heuristics research carried out in part (and the remainder cited) by Daniel Kahneman in his book Thinking, Fast and Slow  in which he discusses the mechanics of how system 1 and system 2 work in our brains. Is Trump taking advantage of the deficits of language processing in our brains in something akin to system 1 biases to win large blocks of votes? Is he creating a virtual real-time Choose-Your-Own-Adventure to subvert the laziness of the electorate? Kahneman would suggest the the combination of what Trump does say and what he doesn’t leaves it up to every individual listener to create their own story. Their system 1 is going to default to the easiest and most palatable one available to them: a happy story that fits their own worldview and is likely to encourage them to support Trump.
Ten Word Answers
As an information theorist, I know all too well that there must be a ‘linguistic Shannon limit’ to the amount of semantic meaning one can compress into a single word.  One is ultimately forced to attempt to form sentences to convey more meaning. But usually the less politicians say, the less trouble they can get into — a lesson hard won through generations of political fighting.
I’m reminded of a scene from The West Wing television series. In season 4, episode 6 which aired on October 30, 2002 on NBC, Game On had a poignant moment (video clip below) which is germane to our subject: 
Moderator: Governor Ritchie, many economists have stated that the tax cut, which is the centrepiece of your economic agenda, could actually harm the economy. Is now really the time to cut taxes?
Governor Ritchie, R-FL: You bet it is. We need to cut taxes for one reason – the American people know how to spend their money better than the federal government does.
Moderator: Mr. President, your rebuttal.
President Bartlet: There it is…
That’s the 10 word answer my staff’s been looking for for 2 weeks. There it is.
10 word answers can kill you in political campaigns — they’re the tip of the sword.
Here’s my question: What are the next 10 words of your answer?
“Your taxes are too high?” So are mine…
Give me the next 10 words: How are we going to do it?
Give me 10 after that — I’ll drop out of the race right now.
Every once in a while — every once in a while, there’s a day with an absolute right and an absolute wrong, but those days almost always include body counts. Other than that there aren’t very many un-nuanced moments in leading a country that’s way too big for 10 words.
I’m the President of the United States, not the president of the people who agree with me. And by the way, if the left has a problem with that, they should vote for somebody else.
As someone who studies information theory and complexity theory and even delves into sub-topics like complexity and economics, I can agree wholeheartedly with the sentiment. Though again, here I can also see the massive gaps between system 1 and 2 that force us to want to simplify things down to such a base level that we don’t have to do the work to puzzle them out.
(And yes, that is Jennifer Anniston’s father playing the moderator.)
One can’t but wonder why Mr. Trump doesn’t seem to have ever gone past the first ten words? Is it because he isn’t capable? interested? Or does he instinctively know better? It would seem that he’s been doing business by using the uncertainty inherent in his speech for decades, but always operating by using what he meant (or thought he wanted to mean) than what the other party heard and thought they understood. If it ain’t broke, don’t fix it.
Idiocracy or Something Worse?
In our increasingly specialized world, people eventually have to give in and quit doing some tasks that everyone used to do for themselves. Yesterday I saw a lifeworn woman in her 70s pushing a wheeled wire basket with a 5 gallon container of water from the store to her home. As she shuffled along, I contemplated Thracian people from fourth century BCE doing the same thing except they likely carried amphorae possibly with a yoke and without the benefit of the $10 manufactured custom shopping cart. 20,000 years before that people were still carrying their own water, but possibly without even the benefit of earthenware containers. Things in human history have changed very slowly for the most part, but as we continually sub-specialize further and further, we need to remember that we can’t give up one of the primary functions that makes us human: the ability to think deeply and analytically for ourselves.
I suspect that far too many people are too wrapped up in their own lives and problems to listen to more than the ten word answers our politicians are advertising to us. We need to remember to ask for the next ten words and the ten after that.
Otherwise there are two extreme possible outcomes:
We’re either at the beginning of what Mike Judge would term Idiocracy. 
Or we’re headed to what Michiko Kakutani is “subtweeting” about in her recent review In ‘Hitler’ an Ascent from ‘Dunderhead’ to Demagogue  of Volker Ulrich’s new book Hitler: Ascent 1889-1939. 
— Mark Harris (@MarkHarrisNYC) September 28, 2016
Here, one is tempted to quote George Santayana’s famous line (from The Life of Reason, 1905), “Those who cannot remember the past are condemned to repeat it.” However, I far prefer the following as more apropos to our present national situation:
If Cliff Navarro comes back to run for president, I hope no one falls for his joke just because he wasn’t laughing as he acted it out. If his instructions for fixing the wagon (America) are any indication, the voters who are listening and making the repairs will be in severe pain.
Went on vacation or fell asleep at the internet wheel this week? Here’s some of the interesting stuff you missed.
Science & Math
- Context Specific and Differential Gene Co-expression Networks via Bayesian Biclustering | PLOS Computational Biology
- The Competing Incentives of Academic Research in Mathematics
- [1607.08473] Quantum circuits and low-degree polynomials over F_2
- This Physics Pioneer Walked Away from it All | Nautilus
- Monumental proof to torment mathematicians for years to come: Conference on Shinichi Mochizuki’s work inspires cautious optimism. | Nature
- What Your Brain Looks Like When It Solves a Math Problem | New York Times
- Habits of Highly Mathematical People
- Why You Should Care About High-Dimensional Sphere Packing | Roots of Unity
- Initial steps toward reproducible research
- Bridging the Curation Gap between Research and Libraries – A Case Study
- Quantum steampunk: Quantum information applied to thermodynamics
- How Vector Space Mathematics Reveals the Hidden Sexism in Language
- How Sound Can Make Food Taste Better | Nautilus
- Top 10 algorithms of 20th century numerical analysis, from a talk by Alex Townsend
- UK vs. US: Who’s got the right way to teach math(s)? | Math with Bad Drawin
- Physics & Caffeine: Stop Motion Film Uses a Cup of Coffee to Explain Key Co
- The Water Kingdom: A Secret History of China by Philip Ball (review)
- The master of them all: Book review for”Leonhard Euler: Mathematical Genius in the Enlightenment” | The Economist
- Biologists Search for New Model Organisms: The bulk of biological research is centered on a handful of species. Are we missing a huge chunk of life’s secrets?
- One-sentence proof of Fermat’s theorem on sums of two squares | Fermat’s Library
- This protein designer aims to revolutionize medicines and materials
- Our last common ancestor inhaled hydrogen from underwater volcanoes
- Meet Luca, The Ancestor of All Living Things | New York Times
- *Disconnected, fragmented, or united? a trans-disciplinary review of network
- What’s Behind A Science vs. Philosophy Fight? | Big Think
- What is a “Neutral Network” Anyway? An Exploration and Rediscovery of the Aims of Net Neutrality in Theory and Practice
- The Brachistochrone Curve: The Problem of Quickest Descent | Fermat’s Library
- In what sense is Quantum Mechanics a theory of information? | Quora
- Major transitions in information technology | Philosophical Transactions of
- Human brain mapped in unprecedented detail: Nearly 100 previously unidentified brain areas revealed by examination of the cerebral cortex. | Nature
- Cell biologists should specialize, not hybridize: Dry cell biologists, who bridge computer science and cell biology, should have a pivotal role in driving effective team science, says Assaf Zaritsky | Nature
- Internet 3.0: How we take back control from the giants | New Scientist
- How a Guy From a Montana Trailer Park Overturned 150 Years of Biology | The Atlantic
- People can sense single photons | Nature News & Comment
- Defining synergy thermodynamically using quantitative measurements of entropy and free energy
- A Prime Case of Chaos | AMS.org
- Murray Gell-Mann (video interviews) – YouTube
- Mathematics & Chalk: A teary goodbye to Hagomoro | Jeremy Kun
- Want to Change Academic Publishing? Just Say No | Chronicle
- Textbooks Show Aging Signs: Curated Guides Are Next – 10+ Disruptive Factors Transforming the World of Education and Learning — Consequences, Opportunities, Tools
- Simon & Schuster, Penguin, Random House Don’t Want to Talk About Their Ebook Sales
- Amazon Sales Rank: Taming the Algorithm | Self-Publishing Author Advice
- What Authors Should Know About Advance Review Copies
- Ingram Launches Ingram Academic Services
- How a Publishing House Designs a Book Cover
- How Indie Bookstores Help Drive Book Discoverability
- How to Grow Your Email List
- 3 Ways Indie Publishers Sell Books | Digital Book World
- 10 Self-Publishing Trends to Watch
- Ingram Launches Academic Services for University Presses and Academic Publishers
- Indigo Goes Where Amazon, B&N, Goodreads, and a Dozen Publishers and Startus Have Dared to Tread
- How To Make An Ebook Feel More Like A Real Book
- Looking for open digital collections – Wynken de Worde
Indieweb, Internet, Identity, Blogging, Social Media
- What is Open Source?
- My Text Editor is Absolutely Sublime | Devon Zuegel
- My zsh aliases | Devon Zuegel
- XOXO Festival
- Web Design in 4 minutes
- Custom Elements
- Design Principles
- Infographic: The Optimal Length for Every Social Media Update
- Notes For New (and Potential) Twitter Followers | Whatever
- How Blogs Work Today – Whatever
- My reply to: How Blogs Work Today | Whatever
- Unicode Character ‘ZERO WIDTH SPACE’ (U 200B)
- A Book Apart, Practical SVG
- Gillmor Gang Trumpathon
- The best news aggregation service – The Sweet Setup
- Social Startup Sprinklr Is Now Valued At $1.8 Billion After $105 Million Raise | Forbes
- Epeus’ epigone: Digital publics, Conversations and Twitter
- The New Meaning of Success
- 7 Lessons from the Future of Content: Part One — Tools Are Cheap, Time Is Expensive
- 7 Lessons from the Future of Content: Part Two — Let’s Play Risk
- Aron Pilhofer Joining Temple University School of Media and Communication
- Secrets and agents: George Akerlof’s 1970 paper, “The Market for Lemons”, is a foundation stone of information economics. The first in our series on seminal economic ideas | The Economist
- John Oliver has the takedown of Donald Trump’s Republican convention
- Reference: New Interactive Map Of 100,000 Photos and Videos Reveal “Lost London in the Victorian Era”
- “better modifiers than “insane(ly)” (Grammar and Usage)
- A lesson in the errors of statistical thinking: Nate Silver on Trump
- Trump & Putin. Yes, It’s Really a Thing
- Charlie Parker Plays with Dizzy Gillespie in Only Footage Capturing the “Bird” in True Live Performance
- Let Me Remind You Fuckers Who I Am (Shit HRC Can’t Say/satire)
Methods originally developed in Information Theory have found wide applicability in computational neuroscience. Beyond these original methods there is a need to develop novel tools and approaches that are driven by problems arising in neuroscience. A number of researchers in computational/systems neuroscience and in information/communication theory are investigating problems of information representation and processing. While the goals are often the same, these researchers bring different perspectives and points of view to a common set of neuroscience problems. Often they participate in different fora and their interaction is limited. The goal of the workshop is to bring some of these researchers together to discuss challenges posed by neuroscience and to exchange ideas and present their latest work. The workshop is targeted towards computational and systems neuroscientists with interest in methods of information theory as well as information/communication theorists with interest in neuroscience.
I was getting concerned that I hadn’t heard back from Sol for a while, particularly after emailing him late last week, and then I ran across this notice through ITSOC & the IEEE:
Solomon W. Golomb (May 30, 1932 – May 1, 2016)
Shannon Award winner and long-time ITSOC member Solomon W. Golomb passed away on May 1, 2016.
Solomon W. Golomb was the Andrew Viterbi Chair in Electrical Engineering at the University of Southern California (USC) and was at USC since 1963, rising to the rank of University and Distinguished Professor. He was a member of the National Academies of Engineering and Science, and was awarded the National Medal of Science, the Shannon Award, the Hamming Medal, and numerous other accolades. As USC Dean Yiannis C. Yortsos wrote, “With unparalleled scholarly contributions and distinction to the field of engineering and mathematics, Sol’s impact has been extraordinary, transformative and impossible to measure. His academic and scholarly work on the theory of communications built the pillars upon which our modern technological life rests.”
In addition to his many contributions to coding and information theory, Professor Golomb was one of the great innovators in recreational mathematics, contributing many articles to Scientific American and other publications. More recent Information Theory Society members may be most familiar with his mathematics puzzles that appeared in the Society Newsletter, which will publish a full remembrance later.
A quick search a moment later revealed this sad confirmation along with some great photos from an award Sol received just a week ago:
— Yannis C. Yortsos (@DeanYortsos) May 2, 2016
— Yannis C. Yortsos (@DeanYortsos) April 22, 2016
— Yannis C. Yortsos (@DeanYortsos) April 21, 2016
As is common in academia, I’m sure it will take a few days for the news to drip out, but the world has certainly lost one of its greatest thinkers, and many of us have lost a dear friend, colleague, and mentor.
I’ll try touch base with his family and pass along what information sniff I can. I’ll post forthcoming obituaries as I see them, and will surely post some additional thoughts and reminiscences of my own in the coming days.
“The notion that counting more shapes in the sky will reveal more details of the Big Bang is implied in a central principle of quantum physics known as “unitarity.” Unitarity dictates that the probabilities of all possible quantum states of the universe must add up to one, now and forever; thus, information, which is stored in quantum states, can never be lost — only scrambled. This means that all information about the birth of the cosmos remains encoded in its present state, and the more precisely cosmologists know the latter, the more they can learn about the former.”
The 2016 School of information will be hosted at Duke University, June 21-23. It is sponsored by the IEEE Information Theory Society, Duke University, the Center for Science of Information, and the National Science Foundation. The school provides a venue where doctoral and postdoctoral students can learn from distinguished professors in information theory, meet with fellow researchers, and form collaborations.
Program and Lectures
The daily schedule will consist of morning and afternoon lectures separated by a lunch break with poster sessions. Students from all research areas are welcome to attend and present their own research via a poster during the school. The school will host lectures on core areas of information theory and interdisciplinary topics. The following lecturers are confirmed:
- Helmut Bölcskei (ETH Zurich): The Mathematics of Deep Learning
- Natasha Devroye (University of Illinois, Chicago): The Interference Channel
- René Vidal (Johns Hopkins University): Global Optimality in Deep Learning and Beyond
- Tsachy Weissman (Stanford University): Information Processing under Logarithmic Loss
- Aylin Yener (Pennsylvania State University): Information-Theoretic Security
Applications will be available on March 15 and will be evaluated starting April 1. Accepted students must register by May 15, 2016. The registration fee of $200 will include food and 3 nights accommodation in a single-occupancy room. We suggest that attendees fly into the Raleigh-Durham (RDU) airport located about 30 minutes from the Duke campus. Housing will be available for check-in on the afternoon of June 20th. The main part of the program will conclude after lunch on June 23rd so that attendees can fly home that evening.
Administrative Contact: Kathy Peterson, email@example.com
Henry Pfister (chair) (Duke University), Dror Baron (North Carolina State University), Matthieu Bloch (Georgia Tech), Rob Calderbank (Duke University), Galen Reeves (Duke University). Advisors: Gerhard Kramer (Technical University of Munich) and Andrea Goldsmith (Stanford)
Many readers often ask me for resources for delving into the basics of information theory. I hadn’t posted it before, but the Santa Fe Institute recently had an online course Introduction to Information Theory through their Complexity Explorer, which has some other excellent offerings. It included videos, fora, and other resources and was taught by the esteemed physicist and professor Seth Lloyd. There are a number of currently active students still learning and posting there.
Introduction to Information Theory
About the Tutorial:
This tutorial introduces fundamental concepts in information theory. Information theory has made considerable impact in complex systems, and has in part co-evolved with complexity science. Research areas ranging from ecology and biology to aerospace and information technology have all seen benefits from the growth of information theory.
In this tutorial, students will follow the development of information theory from bits to modern application in computing and communication. Along the way Seth Lloyd introduces valuable topics in information theory such as mutual information, boolean logic, channel capacity, and the natural relationship between information and entropy.
Lloyd coherently covers a substantial amount of material while limiting discussion of the mathematics involved. When formulas or derivations are considered, Lloyd describes the mathematics such that less advanced math students will find the tutorial accessible. Prerequisites for this tutorial are an understanding of logarithms, and at least a year of high-school algebra.
About the Instructor(s):
Professor Seth Lloyd is a principal investigator in the Research Laboratory of Electronics (RLE) at the Massachusetts Institute of Technology (MIT). He received his A.B. from Harvard College in 1982, the Certificate of Advanced Study in Mathematics (Part III) and an M. Phil. in Philosophy of Science from Cambridge University in 1983 and 1984 under a Marshall Fellowship, and a Ph.D. in Physics in 1988 from Rockefeller University under the supervision of Professor Heinz Pagels.
From 1988 to 1991, Professor Lloyd was a postdoctoral fellow in the High Energy Physics Department at the California Institute of Technology, where he worked with Professor Murray Gell-Mann on applications of information to quantum-mechanical systems. From 1991 to 1994, he was a postdoctoral fellow at Los Alamos National Laboratory, where he worked at the Center for Nonlinear Systems on quantum computation. In 1994, he joined the faculty of the Department of Mechanical Engineering at MIT. Since 1988, Professor Lloyd has also been an adjunct faculty member at the Sante Fe Institute.
Professor Lloyd has performed seminal work in the fields of quantum computation and quantum communications, including proposing the first technologically feasible design for a quantum computer, demonstrating the viability of quantum analog computation, proving quantum analogs of Shannon’s noisy channel theorem, and designing novel methods for quantum error correction and noise reduction.
Professor Lloyd is a member of the American Physical Society and the Amercian Society of Mechanical Engineers.
Yoav Kallus is an Omidyar Fellow at the Santa Fe Institute. His research at the boundary of statistical physics and geometry looks at how and when simple interactions lead to the formation of complex order in materials and when preferred local order leads to system-wide disorder. Yoav holds a B.Sc. in physics from Rice University and a Ph.D. in physics from Cornell University. Before joining the Santa Fe Institute, Yoav was a postdoctoral fellow at the Princeton Center for Theoretical Science in Princeton University.
- Forms of Information
- Information and Probability
- Fundamental Formula of Information
- Computation and Logic: Information Processing
- Mutual Information
- Communication Capacity
- Shannon’s Coding Theorem
- The Manifold Things Information Measures
Yesterday I ran across this nice little video explaining some recent research on global language networks. It’s not only interesting in its own right, but is a fantastic example of science communication as well.
I’m interested in some of the information theoretic aspects of this as well as the relation of this to the area of corpus linguistics. I’m also curious if one could build worthwhile datasets like this for the ancient world (cross reference some of the sources I touch on in relation to the Dickinson College Commentaries within Latin Pedagogy and the Digital Humanities) to see what influences different language cultures have had on each other. Perhaps the historical record could help to validate some of the predictions made in relation to the future?
The paper “Global distribution and drivers of language extinction risk” indicates that of all the variables tested, economic growth was most strongly linked to language loss.
This research also has some interesting relation to the concept of “Collective Learning” within the realm of a Big History framework via David Christian, Fred Spier, et al. I’m curious to revisit my hypothesis: Collective learning has potentially been growing at the expense of a shrinking body of diverse language some of which was informed by the work of Jared Diamond.
Some of the discussion in the video is reminiscent to me of some of the work Stuart Kauffman lays out in At Home in the Universe: The Search for the Laws of Self-Organization and Complexity (Oxford, 1995). Particularly in chapter 3 in which Kauffman discusses the networks of life. The analogy of this to the networks of language here indicate to me that some of Cesar Hidalgo’s recent work in Why Information Grows: The Evolution of Order, From Atoms to Economies (MIT Press, 2015) is even more interesting in helping to show the true value of links between people and firms (information sources which he measures as personbytes and firmbytes) within economies.
Finally, I can also only think about how this research may help to temper some of the xenophobic discussion that occurs in American political life with respect to fears relating to Mexican immigration issues as well as the position of China in the world economy.
Those intrigued by the video may find the website set up by the researchers very interesting. It contains links to the full paper as well as visualizations and links to the data used.
Languages vary enormously in global importance because of historical, demographic, political, and technological forces. However, beyond simple measures of population and economic power, there has been no rigorous quantitative way to define the global influence of languages. Here we use the structure of the networks connecting multilingual speakers and translated texts, as expressed in book translations, multiple language editions of Wikipedia, and Twitter, to provide a concept of language importance that goes beyond simple economic or demographic measures. We find that the structure of these three global language networks (GLNs) is centered on English as a global hub and around a handful of intermediate hub languages, which include Spanish, German, French, Russian, Portuguese, and Chinese. We validate the measure of a language’s centrality in the three GLNs by showing that it exhibits a strong correlation with two independent measures of the number of famous people born in the countries associated with that language. These results suggest that the position of a language in the GLN contributes to the visibility of its speakers and the global popularity of the cultural content they produce.
Citation: Ronen S, Goncalves B, Hu KZ, Vespignani A, Pinker S, Hidalgo CA
Links that speak: the global language network and its association with global fame, Proceedings of the National Academy of Sciences (PNAS) (2014), 10.1073/pnas.1410931111
- Why learning Spanish, not Mandarin, is the best way to globalize your ideas | Quartz
- Want to influence the world? Map reveals the best languages to speak | Science
“A language like Dutch — spoken by 27 million people — can be a disproportionately large conduit, compared with a language like Arabic, which has a whopping 530 million native and second-language speakers,” Science reports. “This is because the Dutch are very multilingual and very online.”
Information is a precise concept that can be defined mathematically, but its relationship to what we call "knowledge" is not always made clear. Furthermore, the concepts "entropy" and "information", while deeply related, are distinct and must be used with care, something that is not always achieved in the literature. In this elementary introduction, the concepts of entropy and information are laid out one by one, explained intuitively, but defined rigorously. I argue that a proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
A proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
Comments: 19 pages, 2 figures. To appear in Philosophical Transaction of the Royal Society A
Subjects: Adaptation and Self-Organizing Systems (nlin.AO); Information Theory (cs.IT); Biological Physics (physics.bio-ph); Quantitative Methods (q-bio.QM)
Cite as:arXiv:1601.06176 [nlin.AO] (or arXiv:1601.06176v1 [nlin.AO] for this version)
As it was published, I had read Kevin Hartnett’s article and interview with Christoph Adami The Information Theory of Life in Quanta Magazine. I recently revisited it and read through the commentary and stumbled upon an interesting quote relating to the history of information in biology:
These two historical references predate Claude Shannon’s mathematical formalization of information in A Mathematical Theory of Communication (The Bell System Technical Journal, 1948) and even Erwin Schrödinger‘s lecture (1943) and subsequent book What is Life (1944).
For those interested in reading more on this historical tidbit, I’ve dug up a copy of the primary Forsdyke reference which first appeared on arXiv (prior to its ultimate publication in History of Psychiatry [.pdf]):
🔖 [1406.1391] ‘A Vehicle of Symbols and Nothing More.’ George Romanes, Theory of Mind, Information, and Samuel Butler by Donald R. Forsdyke 
Submitted on 4 Jun 2014 (v1), last revised 13 Nov 2014 (this version, v2)
Abstract: Today’s ‘theory of mind’ (ToM) concept is rooted in the distinction of nineteenth century philosopher William Clifford between ‘objects’ that can be directly perceived, and ‘ejects,’ such as the mind of another person, which are inferred from one’s subjective knowledge of one’s own mind. A founder, with Charles Darwin, of the discipline of comparative psychology, George Romanes considered the minds of animals as ejects, an idea that could be generalized to ‘society as eject’ and, ultimately, ‘the world as an eject’ – mind in the universe. Yet, Romanes and Clifford only vaguely connected mind with the abstraction we call ‘information,’ which needs ‘a vehicle of symbols’ – a material transporting medium. However, Samuel Butler was able to address, in informational terms depleted of theological trappings, both organic evolution and mind in the universe. This view harmonizes with insights arising from modern DNA research, the relative immortality of ‘selfish’ genes, and some startling recent developments in brain research.
Comments: Accepted for publication in History of Psychiatry. 31 pages including 3 footnotes. Based on a lecture given at Santa Clara University, February 28th 2014, at a Bannan Institute Symposium on ‘Science and Seeking: Rethinking the God Question in the Lab, Cosmos, and Classroom.’
The original arXiv article also referenced two lectures which are appended below:
[Original Draft of this was written on December 14, 2015.]