Saddened to hear of the passing of Sir David J.C. MacKay, FRS

Earlier this morning, I was saddened to hear that one of my information theory heroes passed away today.

David MacKay blackboard

I’ve been following a Google Alert for “information theory,” and so on an almost a daily basis for over 15 years I’ve seen thousands of notices and references to his excellent textbook Information Theory, Inference, and Learning Algorithms, which he kindly chose to freely share with the world. It’s really a great little textbook, and I recommend that everyone download it or purchase it and give it a read. In addition he has a fabulous series of video lectures to go with it as well. (Someone had actually asked me for information theory lectures on Quora last week, and his are some of the best.)

An instant classic, covering everything from Shannon’s fundamental theorems to the postmodern theory of LDPC codes. You’ll want two copies of this astonishing book, one for the office and one for the fireside at home.

Bob McEliece, information theorist and professor, California Institute of Technology

 
Information Theory, Inference and Learning Algorithms

Sir David J.C. MacKay was the Regius Professor of Engineering at Cambridge University and a former professor of natural philosophy in the Department of Physics at at Cavendish Laboratory, University of Cambridge. He was also a leading figure in energy and climate change having written the accessible and highly praised book Sustainable Energy: Without all the Hot Air, which is also available for free on his site. In 2009 he was appointed to a five year term as Chief Scientific Advisor of the Department of Energy and Climate Change, United Kingdom.

His TED talk will give you an idea of some of his work in this area:

MacKay was elected a Fellow of the Royal Society in 2009. His nomination reads:

David MacKay introduced more efficient types of error-correcting code that are now used in satellite communications, digital broadcasting and magnetic recording. He advanced the field of Machine Learning by providing a sound Bayesian foundation for artificial neural networks. Using this foundation, he significantly improved their performance, allowing them to be used for designing new types of steel that are now used in power stations. He used his expertise in information theory to design a widely used interface called “dasher” that allows disabled people to write efficiently using a single finger or head-mounted pointer.

Sir David MacKay was knighted in the 2016 New Year Honours for services to scientific advice in government and to science outreach.

For those interested, he a great little blog. Here’s his last blogpost.

Below, from a variety of information theorists, mathematicians, and scientists is just the beginning of the outpouring of loss the world is experiencing today:



RIP David MacKay, former DECC Chief Scientific Adviser. He was passionate, original, brave. A truly good man. Deep condolences to his family

— Ed Miliband (@Ed_Miliband) April 14, 2016

Online Lectures in Information Theory

Replied to Where can I find good online lectures in information theory? (quora.com)
There aren’t a lot of available online lectures on the subject of information theory, but here are the ones I’m currently aware of:

Introductory

Advanced

Fortunately, most are pretty reasonable, though vary in their coverage of topics. The introductory lectures don’t require as much mathematics and can probably be understood by those at the high school level with just a small amount of basic probability theory and an understanding of the logarithm.

The top three in the advanced section (they generally presume a prior undergraduate level class in probability theory and some amount of mathematical sophistication) are from professors who’ve written some of the most commonly used college textbooks on the subject. If I recall a first edition of the Yeung text was available via download through his course interface. MacKay’s text is available for free download from his site as well.

Feel free to post other video lectures or resources you may be aware of in the comments below.

Editor’s Update: With sadness, I’ll note that David MacKay died just days after this was originally posted.