Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann-Gibbs form of the entropy ensures that probability distributions inferred satisfy the multiplication rule of probability for independent events in the absence of data coupling such events. Other types of entropies that violate the Shore and Johnson axioms, including nonadditive entropies such as the Tsallis entropy, violate this basic consistency requirement. Here we use the axiomatic framework of Shore and Johnson to show how such nonadditive entropy functions generate biases in probability distributions that are not warranted by the underlying data.
Category: Information Theory
Zero-width characters are invisible, ‘non-printing’ characters that are not displayed by the majority of applications. For example, I’ve inserted 10 zero-width spaces into this sentence, can you tell? (Hint: paste the sentence into Diff Checker to see the locations of the characters!). These characters can be used to ‘fingerprint’ text for certain users.
A cool little trick with text for embedded steganography, security, or other communication purposes.
This could also be used for pseudo-private communication via Webmention even. Just hide your messages inside of public messages.
ᔥ bookmark Be careful what you copy: Invisibly inserting usernames into text with Zero-Width Characters (medium.com) ()
ITBio – Chris Aldrich (feed) ❧
Hey, wait! He’s not only following me, but a very distinct subset of my posts! 🙂
This is the first time I’ve ever seen someone indicate that they’ve done this in the wild.
I’ll also admit that this is a really great looking blogroll too! I’m going to have to mine it for the bunch of feeds that I’m not already following.
The media's "epistemic crisis," algorithmic biases, and the radio's inherent, historical misogyny.
In hearings this week, House Democrats sought to highlight an emerging set of facts concerning the President’s conduct. On this week’s On the Media, a look at why muddying the waters remains a viable strategy for Trump’s defenders. Plus, even the technology we trust for its clarity isn’t entirely objective, especially the algorithms that drive decisions in public and private institutions. And, how early radio engineers designed broadcast equipment to favor male voices and make women sound "shrill."
1. David Roberts [@drvox], writer covering energy for Vox, on the "epistemic crisis" at the heart of our bifurcated information ecosystem. Listen.
2. Cathy O'Neil [@mathbabedotorg], mathematician and author of Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, on the biases baked into our algorithms. Listen.
3. Tina Tallon [@ttallon], musician and professor, on how biases built into radio technology have shaped how we hear women speak. Listen.
Some great discussion on the idea of women being “shrill” and ad hominem attacks instead of attacks on ideas.
Cathy O’Neil has a great interview on her book Weapons of Math Distraction. I highly recommend everyone read it, but if for some reason you can’t do it this month, this interview is a good starting place for repairing that deficiency.
In section three, I’ll note that I’ve studied the areas of signal processing and information theory in great depth, but never run across the fascinating history of how we physically and consciously engineered women out of radio and broadcast in quite the way discussed here. I recall the image of “Lena” being nudged out of image processing recently, but the engineering wrongs here are far more serious and pernicious.
President and William H. Miller Professor of Complex Systems
Sean Carroll from preposterousuniverse.comMaxwell’s Demon is a famous thought experiment in which a mischievous imp uses knowledge of the velocities of gas molecules in a box to decrease the entropy of the gas, which could then be used to do useful work such as pushing a piston. This is a classic example of converting information (what the gas molecules are doing) into work. But of course that kind of phenomenon is much more widespread — it happens any time a company or organization hires someone in order to take advantage of their know-how. César Hidalgo has become an expert in this relationship between information and work, both at the level of physics and how it bubbles up into economies and societies. Looking at the world through the lens of information brings new insights into how we learn things, how economies are structured, and how novel uses of data will transform how we live.
César Hidalgo received his Ph.D. in physics from the University of Notre Dame. He currently holds an ANITI Chair at the University of Toulouse, an Honorary Professorship at the University of Manchester, and a Visiting Professorship at Harvard’s School of Engineering and Applied Sciences. From 2010 to 2019, he led MIT’s Collective Learning group. He is the author of Why Information Grows and co-author of The Atlas of Economic Complexity. He is a co-founder of Datawheel, a data visualization company whose products include the Observatory of Economic Complexity.
It was interesting to hear Cesar Hidalgo use the concept of “big history” a few times in this episode. I’m not 100% sure he meant it in the David Christian sense of the words, but it at least felt right.
I was also piqued at the mention of Lynne Kelly’s work, which I’m now knee deep into. I suspect it could dramatically expand on what we think of as the capacity of a personbyte, though the limit of knowledge there still exists. The idea of mnemotechniques within indigenous cultures certainly expands on the way knowledge worked in prehistory and what we classically think of and frame collective knowledge or collective learning.
I also think there are some interesting connections with Dr. Kelly’s mentions of social equity in prehistorical cultures and the work that Hidalgo mentions in the middle of the episode.
There are a small handful of references I’ll want to delve into after hearing this, though it may take time to pull them up unless they’re linked in the show notes.
hat-tip: Complexity Digest for the reminder that this is in my podcatcher. 🔖 November 22, 2019 at 03:28PM
👓 Humane Ingenuity 9: GPT-2 and You | Dan Cohen | Buttondown
Dan Cohen (buttondown.email)This newsletter has not been written by a GPT-2 text generator, but you can now find a lot of artificially created text that has been.
For those not familiar with GPT-2, it is, according to its creators OpenAI (a socially conscious artificial intelligence lab overseen by a nonprofit entity), “a large-scale unsupervised language model which generates coherent paragraphs of text.” Think of it as a computer that has consumed so much text that it’s very good at figuring out which words are likely to follow other words, and when strung together, these words create fairly coherent sentences and paragraphs that are plausible continuations of any initial (or “seed”) text.
This isn’t a very difficult problem and the underpinnings of it are well laid out by John R. Pierce in *[An Introduction to Information Theory: Symbols, Signals and Noise](https://amzn.to/32JWDSn)*. In it he has a lot of interesting tidbits about language and structure from an engineering perspective including the reason why crossword puzzles work.
November 13, 2019 at 08:33AM
The most interesting examples have been the weird ones (cf. HI7), where the language model has been trained on narrower, more colorful sets of texts, and then sparked with creative prompts. Archaeologist Shawn Graham, who is working on a book I’d like to preorder right now, An Enchantment of Digital Archaeology: Raising the Dead with Agent Based Models, Archaeogaming, and Artificial Intelligence, fed GPT-2 the works of the English Egyptologist Flinders Petrie (1853-1942) and then resurrected him at the command line for a conversation about his work. Robin Sloan had similar good fun this summer with a focus on fantasy quests, and helpfully documented how he did it.
Circle back around and read this when it comes out.
Similarly, these other references should be an interesting read as well.
November 13, 2019 at 08:36AM
From this perspective, GPT-2 says less about artificial intelligence and more about how human intelligence is constantly looking for, and accepting of, stereotypical narrative genres, and how our mind always wants to make sense of any text it encounters, no matter how odd. Reflecting on that process can be the source of helpful self-awareness—about our past and present views and inclinations—and also, some significant enjoyment as our minds spin stories well beyond the thrown-together words on a page or screen.
And it’s not just happening with text, but it also happens with speech as I’ve written before: Complexity isn’t a Vice: 10 Word Answers and Doubletalk in Election 2016 In fact, in this mentioned case, looking at transcripts actually helps to reveal that the emperor had no clothes because there’s so much missing from the speech that the text doesn’t have enough space to fill in the gaps the way the live speech did.
November 13, 2019 at 08:43AM
🔖 GLTR (glitter) v0.5
This demo enables forensic inspection of the visual footprint of a language model on input text to detect whether a text could be real or fake.
🔖 The En-Gedi Scroll (2016) | Internet Archive
The data and virtual unwrapping results on the En-Gedi scroll.
See the following papers for more information:Seales, William Brent, et al. "From damage to discovery via virtual unwrapping: Reading the scroll from En-Gedi." Science advances 2.9 (2016): e1601247. (Web Article)Segal, Michael, et al. "An Early Leviticus Scroll From En-Gedi: Preliminary Publication." Textus 26 (2016): 1-30. (PDF)
🔖 Digital Restoration Initiative
The written word has been used throughout history to chronicle and contemplate the human experience, but many valuable texts are “lost” to us due to damage. The words of these documents and the knowledge they seek to impart are locked behind the destruction and decay wrought by time and injury, while the physical manuscripts themselves form an “invisible library” of sorts — closeted away on dark shelves, well-protected but prevented from proffering knowledge and encouraging inquiry. For more than 20 years, Dr. Seales has been working to create and use hi-tech, non-invasive tools to rescue these lost texts from the blink of oblivion and restore them to humanity. We call this innovative process “virtual unwrapping.”
h/t Dan Cohen newsletter #1
📺 EDUCE: Imaging the Herculaneum Scrolls | YouTube
The eruption of Mt. Vesuvius covered the city of Herculaneum in twenty meters of lava, simultaneously destroying the Herculaneum scrolls through carbonization and preserving the scrolls by protecting them from the elements. Unwrapping the scrolls would damage them, but researchers are anxious to read the texts. Researchers from the University of Kentucky collaborated with the Institut de France and SkyScan to digitally unwrap and preserve the scrolls. To learn more about the EDUCE project, go to http://cs.uky.edu/dri.
They haven’t finished the last mile, but having high resolution scans of the objects is great. I’m not sure why they’re handling these items manually when they could very likely be secured in better external casings and still imaged the same way.
He became a mark because there was an imbalance of information between himself and the grifters who had targeted him.
The key to dealing with information is having some sort of memory for it.
👓 What can Schrödinger’s cat say about 3D printers on Mars? | Aeon | Aeon Essays
A cat is alive, a sofa is not: that much we know. But a sofa is also part of life. Information theory tells us why
A nice little essay in my area, but I’m not sure there’s anything new in it for me. It is nice that they’re trying to break some of the problem down into smaller components before building it back up into something else. Reframing things can always be helpful. Here, in particular, they’re reframing the definitions of life and alive.
🔖 Origins Of Life | Complexity Explorer
About the Course:
This course aims to push the field of Origins of Life research forward by bringing new and synthetic thinking to the question of how life emerged from an abiotic world.
This course begins by examining the chemical, geological, physical, and biological principles that give us insight into origins of life research. We look at the chemical and geological environment of early Earth from the perspective of likely environments for life to originate.
Taking a look at modern life we ask what it can tell us about the origin of life by winding the clock backwards. We explore what elements of modern life are absolutely essential for life, and ask what is arbitrary? We ponder how life arose from the huge chemical space and what this early 'living chemistry'may have looked like.
We examine phenomena, that may seem particularly life like, but are in fact likely to arise given physical dynamics alone. We analyze what physical concepts and laws bound the possibilities for life and its formation.
Insights gained from modern evolutionary theory will be applied to proto-life. Once life emerges, we consider how living systems impact the geosphere and evolve complexity.
The study of Origins of Life is highly interdisciplinary - touching on concepts and principles from earth science, biology, chemistry, and physics. With this we hope that the course can bring students interested in a broad range of fields to explore how life originated.
The course will make use of basic algebra, chemistry, and biology but potentially difficult topics will be reviewed, and help is available in the course discussion forum and instructor email. There will be pointers to additional resources for those who want to dig deeper.
This course is Complexity Explorer's first Frontiers Course. A Frontiers Course gives students a tour of an active interdisciplinary research area. The goals of a Frontiers Course are to share the excitement and uncertainty of a scientific area, inspire curiosity, and possibly draw new people into the research community who can help this research area take shape!
I’m totally in for this!
Hat tip for the reminder to:
A new online course on The Origin of Life is just starting at Complexity Explorer @ComplexExplorer, a program of the Santa Fe Institute @sfiscience. I’m enrolled!https://t.co/lZG4bY1T0X
— Sean Carroll (@seanmcarroll) June 14, 2019