👓 The Man Who Tried to Redeem the World with Logic | Issue 21: Information – Nautilus

Read The Man Who Tried to Redeem the World with Logic (Nautilus)
Walter Pitts was used to being bullied. He’d been born into a tough family in Prohibition-era Detroit, where his father, a boiler-maker,…

Highlights, Quotes, Annotations, & Marginalia

McCulloch was a confident, gray-eyed, wild-bearded, chain-smoking philosopher-poet who lived on whiskey and ice cream and never went to bed before 4 a.m.  

Now that is a business card title!

March 03, 2019 at 06:01PM

McCulloch and Pitts were destined to live, work, and die together. Along the way, they would create the first mechanistic theory of the mind, the first computational approach to neuroscience, the logical design of modern computers, and the pillars of artificial intelligence.  

tl;dr

March 03, 2019 at 06:06PM

Gottfried Leibniz. The 17th-century philosopher had attempted to create an alphabet of human thought, each letter of which represented a concept and could be combined and manipulated according to a set of logical rules to compute all knowledge—a vision that promised to transform the imperfect outside world into the rational sanctuary of a library.  

I don’t think I’ve ever heard this quirky story…

March 03, 2019 at 06:08PM

Which got McCulloch thinking about neurons. He knew that each of the brain’s nerve cells only fires after a minimum threshold has been reached: Enough of its neighboring nerve cells must send signals across the neuron’s synapses before it will fire off its own electrical spike. It occurred to McCulloch that this set-up was binary—either the neuron fires or it doesn’t. A neuron’s signal, he realized, is a proposition, and neurons seemed to work like logic gates, taking in multiple inputs and producing a single output. By varying a neuron’s firing threshold, it could be made to perform “and,” “or,” and “not” functions.  

I’m curious what year this was, particularly in relation to Claude Shannon’s master’s thesis in which he applied Boolean algebra to electronics.
Based on their meeting date, it would have to be after 1940.And they published in 1943: https://link.springer.com/article/10.1007%2FBF02478259

March 03, 2019 at 06:14PM

McCulloch and Pitts alone would pour the whiskey, hunker down, and attempt to build a computational brain from the neuron up.  

A nice way to pass the time to be sure. Naturally mathematicians would have been turning “coffee into theorems” instead of whiskey.

March 03, 2019 at 06:15PM

“an idea wrenched out of time.” In other words, a memory.  

March 03, 2019 at 06:17PM

McCulloch and Pitts wrote up their findings in a now-seminal paper, “A Logical Calculus of Ideas Immanent in Nervous Activity,” published in the Bulletin of Mathematical Biophysics.  

March 03, 2019 at 06:21PM

I really like this picture here. Perhaps for a business card?
colorful painting of man sitting with abstract structure around him
  
March 03, 2019 at 06:23PM

it had been Wiener who discovered a precise mathematical definition of information: The higher the probability, the higher the entropy and the lower the information content.  

Oops, I think this article is confusing Wiener with Claude Shannon?

March 03, 2019 at 06:34PM

By the fall of 1943, Pitts had moved into a Cambridge apartment, was enrolled as a special student at MIT, and was studying under one of the most influential scientists in the world.  

March 03, 2019 at 06:32PM

Thus formed the beginnings of the group who would become known as the cyberneticians, with Wiener, Pitts, McCulloch, Lettvin, and von Neumann its core.  

Wiener always did like cyberneticians for it’s parallelism with mathematicians….

March 03, 2019 at 06:38PM

In the entire report, he cited only a single paper: “A Logical Calculus” by McCulloch and Pitts.  

First Draft of a Report on EDVAC by jon von Neumann

March 03, 2019 at 06:43PM

Oliver Selfridge, an MIT student who would become “the father of machine perception”; Hyman Minsky, the future economist; and Lettvin.  

March 03, 2019 at 06:44PM

at the Second Cybernetic Conference, Pitts announced that he was writing his doctoral dissertation on probabilistic three-dimensional neural networks.  

March 03, 2019 at 06:44PM

In June 1954, Fortune magazine ran an article featuring the 20 most talented scientists under 40; Pitts was featured, next to Claude Shannon and James Watson.  

March 03, 2019 at 06:46PM

Lettvin, along with the young neuroscientist Patrick Wall, joined McCulloch and Pitts at their new headquarters in Building 20 on Vassar Street. They posted a sign on the door: Experimental Epistemology.  

March 03, 2019 at 06:47PM

“The eye speaks to the brain in a language already highly organized and interpreted,” they reported in the now-seminal paper “What the Frog’s Eye Tells the Frog’s Brain,” published in 1959.  

March 03, 2019 at 06:50PM

There was a catch, though: This symbolic abstraction made the world transparent but the brain opaque. Once everything had been reduced to information governed by logic, the actual mechanics ceased to matter—the tradeoff for universal computation was ontology. Von Neumann was the first to see the problem. He expressed his concern to Wiener in a letter that anticipated the coming split between artificial intelligence on one side and neuroscience on the other. “After the great positive contribution of Turing-cum-Pitts-and-McCulloch is assimilated,” he wrote, “the situation is rather worse than better than before. Indeed these authors have demonstrated in absolute and hopeless generality that anything and everything … can be done by an appropriate mechanism, and specifically by a neural mechanism—and that even one, definite mechanism can be ‘universal.’ Inverting the argument: Nothing that we may know or learn about the functioning of the organism can give, without ‘microscopic,’ cytological work any clues regarding the further details of the neural mechanism.”  

March 03, 2019 at 06:54PM

Nature had chosen the messiness of life over the austerity of logic, a choice Pitts likely could not comprehend. He had no way of knowing that while his ideas about the biological brain were not panning out, they were setting in motion the age of digital computing, the neural network approach to machine learning, and the so-called connectionist philosophy of mind.  

March 03, 2019 at 06:55PM

by stringing them together exactly as Pitts and McCulloch had discovered, you could carry out any computation.  

I feel like this is something more akin to what may have been already known from Boolean algebra and Whitehead/Russell by this time. Certainly Shannon would have known of it?

March 03, 2019 at 06:58PM

👓 Henry Quastler | Wikipedia

Read Henry Quastler (Wikipedia)
Henry Quastler (November 11, 1908 – July 4, 1963) was an Austrian physician and radiologist who became a pioneer in the field of information theory applied to biology after emigrating to America. His work with Sidney Dancoff led to the publication of what is now commonly called Dancoff's Law.
Spent a moment to make a few additions to the page as well…

👓 Dr. Hubert Yockey of Bel Air, Director of APG Reactor; Manhattan Project Nuclear Physicist, Dies at 99 | The Dagger

Read Dr. Hubert Yockey of Bel Air, Director of APG Reactor; Manhattan Project Nuclear Physicist, Dies at 99 (The Dagger - Local News with an Edge)
Hubert Palmer Yockey, 99, died peacefully under hospice care at his home in Bel Air, MD, on January 31, 2016, with his daughter, Cynthia Yockey, at his side. Born in Alexandria, Minnesota, he was t…

Acquired The Emergence of Biological Organization by Henry Quastler

Acquired The Emergence of Biological Organization by Henry Quastler (Yale Univ Press; First Edition edition (1964))

In 1964 Quastler's book The Emergence of Biological Organization was published posthumously. In 2002, Harold J. Morowitz described it as a "remarkably prescient book" which is "surprisingly contemporary in outlook". In it Quastler pioneers a theory of emergence, developing model of "a series of emergences from probionts to prokaryotes".

The work is based on lectures given by Quastler during the spring term of 1963, when he was Visiting Professor of Theoretical Biology at Yale University. In these lectures Quastler argued that the formation of single-stranded polynucleotides was well within the limits of probability of what could have occurred during the pre-biologic period of the Earth. However, he noted that polymerization of a single-stranded polymer from mononucleotides is slow, and its hydrolysis is fast; therefore in a closed system consisting only of mononucleotides and their single-stranded polymers, only a small fraction of the available molecules will be polymerized. However, a single-stranded polymer may form a double-stranded one by complementary polymerization, using a single-stranded polynucleotide as a template. Such a process is relatively fast and the resulting double-stranded polynucleotide is much more stable than the single single-stranded one since each monomer is bound not only along the sugar phosphate backbone, but also through inter-strand bonding between the bases.

The capability for self-replication, a fundamental feature of life, emerged when double-stranded polynucleotides disassociated into single-stranded ones and each of these served as a template for synthesis of a complementary strand, producing two double-stranded copies. Such a system is mutable since random changes of individual bases may occur and be propagated. Individual replicators with different nucleotide sequences may also compete with each other for nucleotide precursors. Mutations that influence the folding state of polynucleotides may affect the ratio of association of strands to dissociation and thus the ability to replicate. The folding state would also affect the stability of the molecule. These ideas were then developed to speculate on the emergence of genetic information, protein synthesis and other general features of life.

Lily E. Kay says that Quastler's works "are an illuminating example of a well reasoned epistemic quest and a curious disciplinary failure". Quastler's aspiration to create an information based biology was innovative, but his work was "plagued by problems: outdated data, unwarranted assumptions, some dubious numerology, and, most importantly, an inability to generate an experimental agenda." However Quastler's "discursive framework" survived.

Forty-five years after Quastler's 1964 proposal, Lincoln and Joyce described a cross-catalytic system that involves two RNA enzymes (ribosymes) that catalyze each other's synthesis from a total of four component substrates. This synthesis occurred in the absence of protein and could provide the basis for an artificial genetic system.

The Emergence of Biological Organization

There was a single used copy in the UK for $12.49 and all the rest are $149.00+ so I snapped it up. Should be an interesting read in and of itself, but I suspect it’s got an interesting niche of the history of science covered with respect to bit history, complexity, and biological organization.

Should arrive some time between March 13 – March 25.

👓 Book review by Nicolas Rashevsky of Information theory in biology | The bulletin of mathematical biophysics

Read Book review of Information theory in biology by Nicolas Rashevsky (The bulletin of mathematical biophysics, June 1954, Volume 16, Issue 2, pp 183–185)
While sifting through some old bookmarks from CiteULike which is going to disappear from the web soon, I ran across one for this book review of Henry Quastler’s book Information Theory in Biology (1953).

The last page of the review had an interesting information theoretical take on not only book reviews, but the level of information they contain with respect for improved teaching and learning in an era prior to Mihaly Csikszentmihalyi’s ideas about “flow”.

As it isn’t the easiest thing to track down, I’ll quote the relevant paragraphs from page 185:

The purpose of a scientific book (we at least hope!) is to store and convey information in a given field. The purpose of a review is to convey  information about a book. It is therefore legitimate to attempt a mathematical theory of writing books and to find the optimal conditions which make a book good. At first it may seem that the optimal conditions consist of maximizing the amount of information per page, that is, in minimizing the redundancy. But a certain amount of redundancy may not only be desirable, but necessary. When presenting a new subject to young students who have never heard of it, a judicious amount of repetition is good pedagogy. Giving an exact abstract definition and then illustrating it by an example already constitutes a logical redundancy. But how useful it frequently is! The minimum of redundancy that is found in some well-known and excellent mathematical books (nomina sunt odiosa!) occasionally makes those books difficult to read even for mathematicians.
The optimum amount of redundancy is a function of the information and intelligence of the reader for whom the book is written. The analytical form of this function is to be determined by an appropriate mathematical theory of learning. Writing a book even in a field which belongs entirely to the domains of Her Majesty the Queen of Sciences is, alas, still more an art than a science. Is it not possible, however, that in the future it may become an exact science?
If a reviewer’s information and intelligence are exactly equal to the value for which the book has been optimized, then he will perceive as defects in the book only deviations from the optimal conditions. His criticism will be objective and unbiased. If, however, the reviewer’s information and intelligence deviate in any direction from the value for which the book is intended, then he will perceive shortcomings which are not due to the deviation of the book from the optimum, but to the reviewer’s personal characteristics. He may also perceive some advantages in the same way. If in the society of the future every individual will be tagged, through appropriate tests, as to his information and intelligence at a given time, expressed in appropriate units, then a reviewer will be able to calculate the correction for his personal bias. These are fantastic dreams of today, which may become reality in the future.

Some of this is very indicative of why one has to spend some significant time finding and recommending the right textbooks [1][2] for students and why things like personalized learning and improvements in pedagogy are so painfully difficult. Sadly on the pedagogy side we haven’t come as far as he may have hoped in nearly 70 ears, and, in fact, we may have regressed.

I’ve often seen web developers in the IndieWeb community mention the idea that “naming things is hard”, so I can’t help but noticing that this 1950’s reviewer uses the Latin catchphrase nomina sunt odiosa which translates as “names are odious”, which has a very similar, but far older sentiment about naming. It was apparently a problem for the ancients as well.

📑 Walter Pitts by Neil Smalheiser | Journal Perspectives in Biology and Medicine

Bookmarked Walter Pitts by Neil SmalheiserNeil Smalheiser (Journal Perspectives in Biology and Medicine. Volume 43. Issue 2. Page 217 - 226.)
Walter Pitts was pivotal in establishing the revolutionary notion of the brain as a computer, which was seminal in the development of computer design, cybernetics, artificial intelligence, and theoretical neuroscience. He was also a participant in a large number of key advances in 20th-century science.  
This looks like an interesting bio to read.

🔖 Information Theory in Biology by Henry Quastler (editor) | University of Illinois Press (1953)

Bookmarked Information Theory in Biology (University of Illinois Press (1953))
I’d love to have a copy of this book that I don’t think I’d heard of before. I’ve got his later Symposium of Information Theory In Biology (1958) already. That volume gives credit to this prior book as inspiration for the symposium.

I suspect based on the Wikipedia article for Quastler that this may also be the same book as the slightly differently titled Essays on the Use of Information Theory in Biology. (Urbana: University of Illinois Press, 1953). There’s also a 1955 review of the text with this name available as well.

Google uses the first title with 273 pages and the Symposium text specifically cites Information Theory in Biology as the correct title several times.

The tough part seems to be that there are very few copies available online and the ones that are are certainly used, in poor condition, and priced at $100+. Ugh…

📖 Read pages 54-60 of 251 of The Demon in the Machine by Paul Davies

📖 Read pages 54-60 of 251 of The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life by Paul Davies

I’ve seen a few places in the text where he references “group(s) of Japanese scientists” in a collective way where as when the scientists are from the West he tends to name at least a principle investigator if not multiple members of a team. Is this implicit bias? I hope it’s not, but it feels very conspicuous and regular to me and I wish it weren’t there.

Photo of the book The Demon in the Machine by Paul Davies sitting on a wooden table. The cover is primarily the title in a large font superimposed on a wireframe of a bird in which the wireframe is meant to look like nodes in a newtowrk

📖 Read pages 27-54 of 251 of The Demon in the Machine by Paul Davies

📖 Read pages 27-54 of 251 of The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life by Paul Davies

Photo of the book The Demon in the Machine by Paul Davies sitting on a wooden table. The cover is primarily the title in a large font superimposed on a wireframe of a bird in which the wireframe is meant to look like nodes in a newtowrk

📖 Read pages 1-26 of 251 of The Demon in the Machine by Paul Davies

📖 Read pages 1-26 of 251 of The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life by Paul Davies

He seems to have a reasonable opening history here. He references several researchers I’m familiar with and trust at least.

Photo of the book The Demon in the Machine by Paul Davies sitting on a wooden table. The cover is primarily the title in a large font superimposed on a wireframe of a bird in which the wireframe is meant to look like nodes in a newtowrk

Acquired The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life by Paul Davies

Acquired The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life by Paul DaviesPaul Davies (Allen Lane)

How does life create order from chaos? And just what is life, anyway? Leading physicist Paul Davies argues that to find the answers, we must first answer a deeper question: 'What is information?' To understand the origins and nature of life, Davies proposes a radical vision of biology which sees the underpinnings of life as similar to circuits and electronics, arguing that life as we know it should really be considered a phenomenon of information storage. In an extraordinary deep dive into the real mechanics of what we take for granted, Davies reveals how biological processes, from photosynthesis to birds' navigation abilities, rely on quantum mechanics, and explores whether quantum physics could prove to be the secret key of all life on Earth. Lively and accessible, Demons in the Machine boils down intricate interdisciplinary developments to take readers on an eye-opening journey towards the ultimate goal of science: unifying all theories of the living and the non-living, so that humanity can at last understand its place in the universe.

book cover The Demon in the Machine

Ordered from Amazon on February 4th and had it shipped from the UK because I wasn’t sure when the book was going to finally be released in the US.

🔖 The Demon in the Machine by Paul Davies | Allen Lane (2018)

Bookmarked The Demon in the Machine by Paul Davies (Allen Lane)

How does life create order from chaos? And just what is life, anyway? Leading physicist Paul Davies argues that to find the answers, we must first answer a deeper question: 'What is information?' To understand the origins and nature of life, Davies proposes a radical vision of biology which sees the underpinnings of life as similar to circuits and electronics, arguing that life as we know it should really be considered a phenomenon of information storage. In an extraordinary deep dive into the real mechanics of what we take for granted, Davies reveals how biological processes, from photosynthesis to birds' navigation abilities, rely on quantum mechanics, and explores whether quantum physics could prove to be the secret key of all life on Earth. Lively and accessible, Demons in the Machine boils down intricate interdisciplinary developments to take readers on an eye-opening journey towards the ultimate goal of science: unifying all theories of the living and the non-living, so that humanity can at last understand its place in the universe.

book cover of The Demon in the Machine by Paul Davies

Found via review.

👓 ‘I predict a great revolution’: inside the struggle to define life | the Guardian

Read 'I predict a great revolution': inside the struggle to define life by Ian Sample (the Guardian)
Paul Davies thinks combining physics and biology will reveal a pattern of information management
hat tip: Philip Ball

Reply to Andy Gonzalez about NIMBIOS Workshop

Replied to a tweet by Andy GonzalezAndy Gonzalez (Twitter)
Andrew Eckford et al. hosted a related conference a few months prior at BIRS which also has some great videos:

Biological and Bio-Inspired Information Theory(14w5170)

Perhaps it’s time for a follow up conference?

🔖 Theory Of Self Reproducing Automata by John Von Neumann, Arthur W. Burks (Editor) | 9780252727337

Bookmarked Theory Of Self Reproducing Automata by John von Neumann (University of Illinois Press)
Waiting for the price of some of these to drop.

Digital copy available on Archive.org.