[1609.02422] What can logic contribute to information theory?

Bookmarked [1609.02422] What can logic contribute to information theory? by David EllermanDavid Ellerman (128.84.21.199)
Logical probability theory was developed as a quantitative measure based on Boole's logic of subsets. But information theory was developed into a mature theory by Claude Shannon with no such connection to logic. But a recent development in logic changes this situation. In category theory, the notion of a subset is dual to the notion of a quotient set or partition, and recently the logic of partitions has been developed in a parallel relationship to the Boolean logic of subsets (subset logic is usually mis-specified as the special case of propositional logic). What then is the quantitative measure based on partition logic in the same sense that logical probability theory is based on subset logic? It is a measure of information that is named "logical entropy" in view of that logical basis. This paper develops the notion of logical entropy and the basic notions of the resulting logical information theory. Then an extensive comparison is made with the corresponding notions based on Shannon entropy.
Ellerman is visiting at UC Riverside at the moment. Given the information theory and category theory overlap, I’m curious if he’s working with John Carlos Baez, or what Baez is aware of this.

Based on a cursory look of his website(s), I’m going to have to start following more of this work.

Randomness And Complexity, from Leibniz To Chaitin | World Scientific Publishing

Bookmarked Randomness And Complexity, from Leibniz To Chaitin (amzn.to)
The book is a collection of papers written by a selection of eminent authors from around the world in honour of Gregory Chaitin s 60th birthday. This is a unique volume including technical contributions, philosophical papers and essays. Hardcover: 468 pages; Publisher: World Scientific Publishing Company (October 18, 2007); ISBN: 9789812770820

Peter Webb’s A Course in Finite Group Representation Theory

Bookmarked A Course in Finite Group Representation Theory by Peter WebbPeter Webb (math.umn.edu)
Download a pre-publication version of the book which will be published by Cambridge University Press. The book arises from notes of courses taught at the second year graduate level at the University of Minnesota and is suitable to accompany study at that level.

“Why should we want to know about representations over rings that are not fields of characteristic zero? It is because they arise in many parts of mathematics. Group representations appear any time we have a group of symmetries where there is some linear structure present, over some commutative ring. That ring need not be a field of characteristic zero.

Here are some examples.

  • […]
  • In the theory of error-correcting codes many important codes have a non-trivial symmetry group and are vector spaces over a finite field, thereby providing a representation of the group over that field.”
Peter Webb, February 23, 2016, Professor of Mathematics, University of Minnesota
in A Course in Finite Group Representation Theory to be published soon by Cambridge University Press

 

Introduction to Complex Analysis | UCLA Extension

Dr. Michael Miller has announced his Autumn mathematics course, and it is…

Introduction to Complex Analysis

Course Description

Complex analysis is one of the most beautiful and useful disciplines of mathematics, with applications in engineering, physics, and astronomy, as well as other branches of mathematics. This introductory course reviews the basic algebra and geometry of complex numbers; develops the theory of complex differential and integral calculus; and concludes by discussing a number of elegant theorems, including many–the fundamental theorem of algebra is one example–that are consequences of Cauchy’s integral formula. Other topics include De Moivre’s theorem, Euler’s formula, Riemann surfaces, Cauchy-Riemann equations, harmonic functions, residues, and meromorphic functions. The course should appeal to those whose work involves the application of mathematics to engineering problems as well as individuals who are interested in how complex analysis helps explain the structure and behavior of the more familiar real number system and real-variable calculus.

Prerequisites

Basic calculus or familiarity with differentiation and integration of real-valued functions.

Details

MATH X 451.37 – 268651  Introduction to Complex Analysis
Fall 2016
Time 7:00PM to 10:00PM
Dates Tuesdays, Sep 20, 2016 to Dec 06, 2016
Contact Hours 33.00
Location: UCLA, Math Sciences Building
Standard credit (3.9 units) $453.00
Instructor: Michael Miller
Register Now at UCLA

For many who will register, this certainly won’t be their first course with Dr. Miller — yes, he’s that good! But for the newcomers, I’ve written some thoughts and tips to help them more easily and quickly settle in and adjust:
Dr. Michael Miller Math Class Hints and Tips | UCLA Extension

I often recommend people to join in Mike’s classes and more often hear the refrain: “I’ve been away from math too long”, or “I don’t have the prerequisites to even begin to think about taking that course.” For people in those categories, you’re in luck! If you’ve even had a soupcon of calculus, you’ll be able to keep up here. In fact, it was a similar class exactly a decade ago by Mike Miller that got me back into mathematics. (Happy 10th math anniversary to me!)

I look forward to seeing everyone in the Fall!

Update 9/1/16

Textbook

Dr. Miller is back from summer vacation and emailed me this morning to say that he’s chosen the textbook for the class. We’ll be using Complex Analysis with Applications by Richard A. Silverman [1]

Complex Analysis with Applications by Richard A. Silverman

(Note that there’s another introductory complex analysis textbook from Silverman that’s offered through Dover, so be sure to choose the correct one.)

As always in Dr. Miller’s classes, the text is just recommended (read: not required) and in-class notes are more than adequate. To quote him directly, “We will be using as a basic guide, but, as always, supplemented by additional material and alternate ways of looking at things.”

The bonus surprise of his email: He’s doing two quarters of Complex Analysis! So we’ll be doing both the Fall and Winter Quarters to really get some depth in the subject!

Alternate textbooks

If you’re like me, you’ll probably take a look at some of the other common (and some more advanced) textbooks in the area. Since I’ve already compiled a list, I’ll share it:

Undergraduate

More advanced

References

[1]
R. A. Silverman, Complex Analysis with Applications, 1st ed. Dover Publications, Inc., 2010, pp. 304–304 [Online]. Available: http://amzn.to/2c7KaQy
[2]
J. Bak and D. J. Newman, Complex Analysis, 3rd ed. Springer, 2010, pp. 328–328 [Online]. Available: http://amzn.to/2bLPW89
[3]
T. Gamelin, Complex Analysis. Springer, 2003, pp. 478–478 [Online]. Available: http://amzn.to/2bGNQct
[4]
J. Brown and R. V. Churchill, Complex Variables and Applications, 8th ed. McGraw-Hill, 2008, pp. 468–468 [Online]. Available: http://amzn.to/2bLQWcu
[5]
E. B. Saff and A. D. Snider, Fundamentals of Complex Analysis with Applications to Engineering, Science, and Mathematics, 3rd ed. Pearson, 2003, pp. 563–563 [Online]. Available: http://amzn.to/2f3Nyj6
[6]
L. V. Ahlfors, Complex Analysis, 3rd ed. McGraw-Hill, 1979, pp. 336–336 [Online]. Available: http://amzn.to/2bMXrxm
[7]
S. Lang, Complex Analysis, 4th ed. Springer, 2003, pp. 489–489 [Online]. Available: http://amzn.to/2c7OaR0
[8]
J. B. Conway, Functions of One Complex Variable, 2nd ed. Springer, 1978, pp. 330–330 [Online]. Available: http://amzn.to/2cggbF1
[9]
El. M. Stein and R. Shakarchi, Complex Analysis. Princeton University Press, 2003, pp. 400–400 [Online]. Available: http://amzn.to/2bGOG9c

Introduction to Dynamical Systems and Chaos

Introduction to Dynamical Systems and Chaos (Summer, 2016)

About the Course:

In this course you’ll gain an introduction to the modern study of dynamical systems, the interdisciplinary field of applied mathematics that studies systems that change over time.

Topics to be covered include: phase space, bifurcations, chaos, the butterfly effect, strange attractors, and pattern formation. The course will focus on some of the realizations from the study of dynamical systems that are of particular relevance to complex systems:

  1. Dynamical systems undergo bifurcations, where a small change in a system parameter such as the temperature or the harvest rate in a fishery leads to a large and qualitative change in the system’s
    behavior.
  2. Deterministic dynamical systems can behave randomly. This property, known as sensitive dependence or the butterfly effect, places strong limits on our ability to predict some phenomena.
  3. Disordered behavior can be stable. Non-periodic systems with the butterfly effect can have stable average properties. So the average or statistical properties of a system can be predictable, even if its details are not.
  4. Complex behavior can arise from simple rules. Simple dynamical systems do not necessarily lead to simple results. In particular, we will see that simple rules can produce patterns and structures of surprising complexity.

About the Instructor:

content_headshotDavid Feldman is Professor of Physics and Mathematics at College of the Atlantic. From 2004-2009 he was a faculty member in the Santa Fe Institute’s Complex Systems Summer School in Beijing, China. He served as the school’s co-director from 2006-2009. Dave is the author of Chaos and Fractals: An Elementary Introduction (Oxford University Press, 2012), a textbook on chaos and fractals for students with a background in high school algebra. Dave was a U.S. Fulbright Lecturer in Rwanda in 2011-12.

Course dates:

5 Jul 2016 9am PDT to
20 Sep 2016 3pm PDT

Prerequisites:

A familiarity with basic high school algebra. There will be optional lessons for those with stronger math backgrounds.

Syllabus

  • Introduction I: Iterated Functions
  • Introduction II: Differential Equations
  • Chaos and the Butterfly Effect
  • Bifurcations: Part I (Differential Equations)
  • Bifurcations: Part II (Logistic Map)
  • Universality
  • Phase Space
  • Strange Attractors
  • Pattern Formation
  • Summary and Conclusions

Source: Complexity Explorer

https://www.youtube.com/watch?v=li_YSEvdvvg&feature=youtu.be

Calculating the Middle Ages?

Bookmarked Calculating the Middle Ages? The Project "Complexities and Networks in the Medieval Mediterranean and Near East" (COMMED) [1606.03433] (arxiv.org)
The project "Complexities and networks in the Medieval Mediterranean and Near East" (COMMED) at the Division for Byzantine Research of the Institute for Medieval Research (IMAFO) of the Austrian Academy of Sciences focuses on the adaptation and development of concepts and tools of network theory and complexity sciences for the analysis of societies, polities and regions in the medieval world in a comparative perspective. Key elements of its methodological and technological toolkit are applied, for instance, in the new project "Mapping medieval conflicts: a digital approach towards political dynamics in the pre-modern period" (MEDCON), which analyses political networks and conflict among power elites across medieval Europe with five case studies from the 12th to 15th century. For one of these case studies on 14th century Byzantium, the explanatory value of this approach is presented in greater detail. The presented results are integrated in a wider comparison of five late medieval polities across Afro-Eurasia (Byzantium, China, England, Hungary and Mamluk Egypt) against the background of the {guillemotright}Late Medieval Crisis{guillemotleft} and its political and environmental turmoil. Finally, further perspectives of COMMED are outlined.

Network and Complexity Theory Applied to History

This interesting paper (summary below) appears to apply network and complexity science to history and is sure to be of interest to those working at the intersection of some of these types of interdisciplinary studies. In particular, I’d be curious to see more coming out of this type of area to support theses written by scholars like Francis Fukuyama in the development of societal structures. Those interested in the emerging area of Big History are sure to enjoy this type of treatment. I’m also curious how researchers in economics (like Cesar Hidalgo) might make use of available(?) historical data in such related analyses. I’m curious if Dave Harris might consider such an analysis in his ancient Near East work?

Those interested in a synopsis of the paper might find some benefit from an overview from MIT Technology Review: How the New Science of Computational History Is Changing the Study of the Past.

16w5113: Stochastic and Deterministic Models for Evolutionary Biology | Banff International Research Station

Bookmarked Stochastic and Deterministic Models for Evolutionary Biology (Banff International Research Station)
A BIRS / Casa Matemática Oaxaca Workshop arriving in Oaxaca, Mexico Sunday, July 31 and departing Friday August 5, 2016

Evolutionary biology is a rapidly changing field, confronted to many societal problems of increasing importance: impact of global changes, emerging epidemics, antibiotic resistant bacteria… As a consequence, a number of new problematics have appeared over the last decade, challenging the existing mathematical models. There exists thus a demand in the biology community for new mathematical models allowing a qualitative or quantitative description of complex evolution problems. In particular, in the societal problems mentioned above, evolution is often interacting with phenomena of a different nature: interaction with other organisms, spatial dynamics, age structure, invasion processes, time/space heterogeneous environment… The development of mathematical models able to deal with those complex interactions is an ambitious task. Evolutionary biology is interested in the evolution of species. This process is a combination of several phenomena, some occurring at the individual level (e.g. mutations), others at the level of the entire population (competition for resources), often consisting of a very large number of individuals. the presence of very different scales is indeed at the core of theoretical evolutionary biology, and at the origin of many of the difficulties that biologists are facing. The development of new mathematical models thus requires a joint work of three different communities of researchers: specialists of partial differential equations, specialists of probability theory, and theoretical biologists. The goal of this workshop is to gather researchers from each of these communities, currently working on close problematics. Those communities have usually few interactions, and this meeting would give them the opportunity to discuss and work around a few biological thematics that are especially challenging mathematically, and play a crucial role for biological applications.

The role of a spatial structure in models for evolution: The introduction of a spatial structure in evolutionary biology models is often challenging. It is however well known that local adaptation is frequent in nature: field data show that the phenotypes of a given species change considerably across its range. The spatial dynamics of a population can also have a deep impact on its evolution. Assessing e.g. the impact of global changes on species requires the development of robust mathematical models for spatially structured populations.

The first type of models used by theoretical biologists for this type of problems are IBM (Individual Based Models), which describe the evolution of a finite number of individuals, characterized by their position and a phenotype. The mathematical analysis of IBM in spatially homogeneous situations has provided several methods that have been successful in the theoretical biology community (see the theory of Adaptive Dynamics). On the contrary, very few results exist so far on the qualitative properties of such models for spatially structured populations.

The second class of mathematical approach for this type of problem is based on ”infinite dimensional” reaction-diffusion: the population is structured by a continuous phenotypic trait, that affects its ability to disperse (diffusion), or to reproduce (reaction). This type of model can be obtained as a large population limit of IBM. The main difficulty of these models (in the simpler case of asexual populations) is the term modeling the competition from resources, that appears as a non local competition term. This term prevents the use of classical reaction diffusion tools such as the comparison principle and sliding methods. Recently, promising progress has been made, based on tools from elliptic equations and/or Hamilton-Jacobi equations. The effects of small populations can however not be observed on such models. The extension of these models and methods to include these effects will be discussed during the workshop.

Eco-evolution models for sexual populations:An essential question already stated by Darwin and Fisher and which stays for the moment without answer (although it continues to intrigue the evolutionary biologists) is: ”Why does sexual reproduction maintain?” Indeed this reproduction way is very costly since it implies a large number of gametes, the mating and the choice of a compatible partner. During the meiosis phasis, half of the genetical information is lost. Moreover, the males have to be fed and during the sexual mating, individual are easy preys for predators. A partial answer is that recombination plays a main role by better eliminating the deleterious mutations and by increasing the diversity. Nevertheless, this theory is not completely satisfying and many researches are devoted to understanding evolution of sexual populations and comparison between asexual and sexual reproduction. Several models exist to model the influence of sexual reproduction on evolving species. The difficulty compared to asexual populations is that a detailed description of the genetic basis of phenotypes is required, and in particular include recombinations. For sexual populations, recombination plays a main role and it is essential to understand. All models require strong biological simplifications, the development of relevant mathematical methods for such mechanisms then requires a joint work of mathematicians and biologists. This workshop will be an opportunity to set up such collaborations.

The first type of model considers a small number of diploid loci (typically one locus and two alleles), while the rest of the genome is considered as fixed. One can then define the fitness of every combination of alleles. While allowing the modeling of specific sexual effects (such as dominant/recessive alleles), this approach neglects the rest of the genome (and it is known that phenotypes are typically influenced by a large number of loci). An opposite approach is to consider a large number of loci, each locus having a small and additive impact on the considered phenotype. This approach then neglects many microscopic phenomena (epistasis, dominant/recessive alleles…), but allows the derivation of a deterministic model, called the infinitesimal model, in the case of a large population. The construction of a good mathematical framework for intermediate situation would be an important step forward.

The evolution of recombination and sex is very sensitive to the interaction between several evolutionary forces (selection, migration, genetic drift…). Modeling these interactions is particularly challenging and our understanding of the recombination evolution is often limited by strong assumptions regarding demography, the relative strength of these different evolutionary forces, the lack of spatial structure… The development of a more general theoretical framework based on new mathematical developments would be particularly valuable.

Another problem, that has received little attention so far and is worth addressing, is the modeling of the genetic material exchanges in asexual population. This phenomena is frequent in micro-organisms : horizontal gene transfers in bacteria, reassortment or recombination in viruses. These phenomena share some features with sexual reproduction. It would be interesting to see if the effect of this phenomena can be seen as a perturbation of existing asexual models. This would in particular be interesting in spatially structured populations (e.g. viral epidemics), since the the mathematical analysis of spatially structured asexual populations is improving rapidly.

Modeling in evolutionary epidemiology: Mathematical epidemiology has been developing since more than a century ago. Yet, the integration of population genetics phenomena to epidemiology is relatively recent. Microbial pathogens (bacteria and viruses) are particularly interesting organisms because their short generation times and large mutation rates allow them to adapt relatively fast to changing environments. As a consequence, ecological (demography) and evolutionary (population genetics) processes often occur at the same pace. This raises many interesting problems.

A first challenge is the modeling of the spatial dynamics of an epidemics. The parasites can evolve during the epidemics of a new host population, either to adapt to a heterogeneous environment, or because it will itself modify the environment as it invades. The applications of such studies are numerous: antibiotic management, agriculture… An aspect of this problem for which our workshop can bring a significant contribution (thanks to the diversity of its participants) is the evolution of the pathogen diversity. During the large expansion produced by an epidemics, there is a loss of diversity in the invading parasites, since most pathogens originate from a few parents. The development of mathematical models for those phenomena is challenging: only a small number of pathogens are present ahead of the epidemic front, while the number of parasites rapidly become very large after the infection. The interaction between a stochastic micro scale and a deterministic macro scale is apparent here, and deserves a rigorous mathematical analysis.

Another interesting phenomena is the effect of a sudden change of the environment on a population of pathogens. Examples of such situations are for instance the antibiotic treatment of an infected patients, or the transmission of a parasite to a new host species (transmission of the avian influenza to human beings, for instance). Related experiments are relatively easy to perform, and called evolutionary rescue experiments. So far, this question has received limited attention from the mathematical community. The key is to estimate the probability that a mutant well adapted to the new environment existed in the original population, or will appear soon after the environmental change. Interactions between biologists specialists of those questions and mathematicians should lead to new mathematical problems.

The L-functions and modular forms database

Bookmarked The L-functions and modular forms database by Tim Gowers (Gowers's Weblog)
...I rejoice that a major new database was launched today. It’s not in my area, so I won’t be using it, but I am nevertheless very excited that it exists. It is called the L-functions and modular forms database. The thinking behind the site is that lots of number theorists have privately done lots of difficult calculations concerning L-functions, modular forms, and related objects. Presumably up to now there has been a great deal of duplication, because by no means all these calculations make it into papers, and even if they do it may be hard to find the right paper. But now there is a big database of these objects, with a large amount of information about each one, as well as a great big graph of connections between them. I will be very curious to know whether it speeds up research in number theory: I hope it will become a completely standard tool in the area and inspire people in other areas to create databases of their own.

…I rejoice that a major new database was launched today. It’s not in my area, so I won’t be using it, but I am nevertheless very excited that it exists. It is called the L-functions and modular forms database. The thinking behind the site is that lots of number theorists have privately done lots of difficult calculations concerning L-functions, modular forms, and related objects. Presumably up to now there has been a great deal of duplication, because by no means all these calculations make it into papers, and even if they do it may be hard to find the right paper. But now there is a big database of these objects, with a large amount of information about each one, as well as a great big graph of connections between them. I will be very curious to know whether it speeds up research in number theory: I hope it will become a completely standard tool in the area and inspire people in other areas to create databases of their own.

–Tim Gowers

Tom M. Apostol, 1923–2016

Bookmarked Tom M. Apostol, 1923–2016 (CalTech Division of Physics, Mathematics and Astronomy)
Tom M. Apostol, professor of mathematics, emeritus at California Institute of Technology passed away on May 8, 2016. He was 92.
My proverbial mathematical great-grandfather passed away yesterday.

As many know, for over a decade, I’ve been studying a variety of areas of advanced abstract mathematics with Michael Miller. Mike Miller received his Ph.D. in 1974 (UCLA) under the supervision of Basil Gordon who in turn received his Ph.D. in 1956 (CalTech) under the supervision of Tom M. Apostol.

Incidentally going back directly three generations is Markov and before that Chebyshev and two generations before that Lobachevsky.

Sadly, I never got to have Tom as a teacher directly myself, though I did get to meet him several times in (what mathematicians might call) social situations. I did have the advantage of delving into his two volumes of Calculus as well as referring to his book on Analytic Number Theory. If it’s been a while since you’ve looked at calculus, I highly recommend an evening or two by the fire with a glass of wine while you revel in Calculus, Vol 1 or Calculus, Vol 2.

It’s useful to take a moment to remember our intellectual antecedents, so in honor of Tom’s passing, I recommend the bookmarked very short obituary (I’m sure more will follow), this obituary of Basil, and this issue of the Notices of the AMS celebrating Basil as well. I also came across a copy of Fascinating Mathematical People which has a great section on Tom and incidentally includes some rare younger photos of Sol Golomb who suddenly passed away last Sunday. (It’s obviously been a tough week for me and math in Southern California this week.)

Mathematician Basil Gordon (wearing a brown sweater) sitting near Chris Aldrich
Somehow Basil Gordon (brown sweater) and I were recognized on the same day at an event for Johns Hopkins University. Later we discovered that he had overseen the Ph.D. of one of my long-time mathematics professors and he was a lifelong friend of my friend/mentor Solomon Golomb. — at Loews Santa Monica Beach Hotel on March 13, 2010

Michael Miller making a "handwaving argument" during a lecture on Algebraic Number Theory at UCLA on November 15, 2015. I've taken over a dozen courses from Mike in areas including Group Theory, Field Theory, Galois Theory, Group Representations, Algebraic Number Theory, Complex Analysis, Measure Theory, Functional Analysis, Calculus on Manifolds, Differential Geometry, Lie Groups and Lie Algebras, Set Theory, Differential Geometry, Algebraic Topology, Number Theory, Integer Partitions, and p-Adic Analysis.
Michael Miller making a “handwaving argument” during a lecture on Algebraic Number Theory at UCLA on November 15, 2015. I’ve taken over a dozen courses from Mike in areas including Group Theory, Field Theory, Galois Theory, Group Representations, Algebraic Number Theory, Complex Analysis, Measure Theory, Functional Analysis, Calculus on Manifolds, Differential Geometry, Lie Groups and Lie Algebras, Set Theory, Differential Geometry, Algebraic Topology, Number Theory, Integer Partitions, and p-Adic Analysis.

Devastating News: Sol Golomb has apparently passed away on Sunday

I was getting concerned that I hadn’t heard back from Sol for a while, particularly after emailing him late last week, and then I ran across this notice through ITSOC & the IEEE:

Solomon W. Golomb (May 30, 1932 – May 1, 2016)

Shannon Award winner and long-time ITSOC member Solomon W. Golomb passed away on May 1, 2016.
Solomon W. Golomb was the Andrew Viterbi Chair in Electrical Engineering at the University of Southern California (USC) and was at USC since 1963, rising to the rank of University and Distinguished Professor. He was a member of the National Academies of Engineering and Science, and was awarded the National Medal of Science, the Shannon Award, the Hamming Medal, and numerous other accolades. As USC Dean Yiannis C. Yortsos wrote, “With unparalleled scholarly contributions and distinction to the field of engineering and mathematics, Sol’s impact has been extraordinary, transformative and impossible to measure. His academic and scholarly work on the theory of communications built the pillars upon which our modern technological life rests.”

In addition to his many contributions to coding and information theory, Professor Golomb was one of the great innovators in recreational mathematics, contributing many articles to Scientific American and other publications. More recent Information Theory Society members may be most familiar with his mathematics puzzles that appeared in the Society Newsletter, which will publish a full remembrance later.

A quick search a moment later revealed this sad confirmation along with some great photos from an award Sol received just a week ago:

As is common in academia, I’m sure it will take a few days for the news to drip out, but the world has certainly lost one of its greatest thinkers, and many of us have lost a dear friend, colleague, and mentor.

I’ll try touch base with his family and pass along what information sniff I can. I’ll post forthcoming obituaries as I see them, and will surely post some additional thoughts and reminiscences of my own in the coming days.

Golomb and national medal of science
President Barack Obama presents Solomon Golomb with the National Medal of Science at an awards ceremony held at the White House in 2013.

Online Lectures in Information Theory

Replied to Where can I find good online lectures in information theory? (quora.com)
There aren’t a lot of available online lectures on the subject of information theory, but here are the ones I’m currently aware of:

Introductory

Advanced

Fortunately, most are pretty reasonable, though vary in their coverage of topics. The introductory lectures don’t require as much mathematics and can probably be understood by those at the high school level with just a small amount of basic probability theory and an understanding of the logarithm.

The top three in the advanced section (they generally presume a prior undergraduate level class in probability theory and some amount of mathematical sophistication) are from professors who’ve written some of the most commonly used college textbooks on the subject. If I recall a first edition of the Yeung text was available via download through his course interface. MacKay’s text is available for free download from his site as well.

Feel free to post other video lectures or resources you may be aware of in the comments below.

Editor’s Update: With sadness, I’ll note that David MacKay died just days after this was originally posted.

“ALOHA to the Web”: Dr. Norm Abramson to give 2016 Viterbi Lecture at USC

Bookmarked USC - Viterbi School of Engineering - Dr. Norm Abramson (viterbi.usc.edu)

“ALOHA to the Web”

Dr. Norman Abramson, Professor Emeritus, University of Hawaii

Lecture Information

Thursday, April 14, 2016
Hughes Electrical Engineering Center (EEB)
Reception 3:00pm (EEB Courtyard)
Lecture 4:00pm (EEB 132)

Abstract

Wireless access to the Internet today is provided predominantly by random access ALOHA channels connecting a wide variety of user devices. ALOHA channels were first analyzed, implemented and demonstrated in the ALOHA network at the University of Hawaii in June, 1971. Information Theory has provided a constant guide for the design of more efficient channels and network architectures for ALOHA access to the web.

In this talk we examine the architecture of networks using ALOHA channels and the statistics of traffic within these channels. That traffic is composed of user and app oriented information augmented by protocol information inserted for the benefit of network operation. A simple application of basic Information Theory can provide a surprising guide to the amount of protocol information required for typical web applications.

We contrast this theoretical guide of the amount of protocol information required with measurements of protocol generated information taken on real network traffic. Wireless access to the web is not as efficient as you might guess.

Biography

Norman Abramson received an A.B. in physics from Harvard College in 1953, an M.A. in physics from UCLA in 1955, and a Ph.D. in Electrical Engineering from Stanford in 1958.

He was an assistant professor and associate professor of electrical engineering at Stanford from 1958 to 1965. From 1967 to 1995 he was Professor of Electrical Engineering, Professor of Information and Computer Science, Chairman of the Department of Information and Computer Science, and Director of the ALOHA System at the University of Hawaii in Honolulu. He is now Professor Emeritus of Electrical Engineering at the University of Hawaii. He has held visiting appointments at Berkeley (1965), Harvard (1966) and MIT (1980).

Abramson is the recipient of several major awards for his work on random access channels and the ALOHA Network, the first wireless data network. The ALOHA Network went into operation in Hawaii in June, 1971. Among these awards are the Eduard Rhein Foundation Technology Award (Munich, 2000), the IEEE Alexander Graham Bell Medal (Philadelphia, 2007) and the NEC C&C Foundation Award (Tokyo, 2011).

2016 North-American School of Information Theory, June 21-23

Bookmarked 2016 North-American School of Information Theory, June 21-23, 2016 (itsoc.org)

The 2016 School of information will be hosted at Duke University, June 21-23. It is sponsored by the IEEE Information Theory Society, Duke University, the Center for Science of Information, and the National Science Foundation. The school provides a venue where doctoral and postdoctoral students can learn from distinguished professors in information theory, meet with fellow researchers, and form collaborations.

Program and Lectures

The daily schedule will consist of morning and afternoon lectures separated by a lunch break with poster sessions. Students from all research areas are welcome to attend and present their own research via a poster during the school.  The school will host lectures on core areas of information theory and interdisciplinary topics. The following lecturers are confirmed:

  • Helmut Bölcskei (ETH Zurich): The Mathematics of Deep Learning
  • Natasha Devroye (University of Illinois, Chicago): The Interference Channel
  • René Vidal (Johns Hopkins University): Global Optimality in Deep Learning and Beyond
  • Tsachy Weissman (Stanford University): Information Processing under Logarithmic Loss
  • Aylin Yener (Pennsylvania State University): Information-Theoretic Security

Logistics

Applications will be available on March 15 and will be evaluated starting April 1.  Accepted students must register by May 15, 2016.  The registration fee of $200 will include food and 3 nights accommodation in a single-occupancy room.  We suggest that attendees fly into the Raleigh-Durham (RDU) airport located about 30 minutes from the Duke campus. Housing will be available for check-in on the afternoon of June 20th.  The main part of the program will conclude after lunch on June 23rd so that attendees can fly home that evening.

To Apply: click “register” here (fee will accepted later after acceptance)

Administrative Contact: Kathy Peterson, itschool2016@gmail.com

Organizing Committee

Henry Pfister (chair) (Duke University), Dror Baron (North Carolina State University), Matthieu Bloch (Georgia Tech), Rob Calderbank (Duke University), Galen Reeves (Duke University). Advisors: Gerhard Kramer (Technical University of Munich) and Andrea Goldsmith (Stanford)

Sponsors

Introduction to Information Theory | SFI’s Complexity Explorer

Many readers often ask me for resources for delving into the basics of information theory. I hadn’t posted it before, but the Santa Fe Institute recently had an online course Introduction to Information Theory through their Complexity Explorer, which has some other excellent offerings. It included videos, fora, and other resources and was taught by the esteemed physicist and professor Seth Lloyd. There are a number of currently active students still learning and posting there.

Introduction to Information Theory

About the Tutorial:

This tutorial introduces fundamental concepts in information theory. Information theory has made considerable impact in complex systems, and has in part co-evolved with complexity science. Research areas ranging from ecology and biology to aerospace and information technology have all seen benefits from the growth of information theory.

In this tutorial, students will follow the development of information theory from bits to modern application in computing and communication. Along the way Seth Lloyd introduces valuable topics in information theory such as mutual information, boolean logic, channel capacity, and the natural relationship between information and entropy.

Lloyd coherently covers a substantial amount of material while limiting discussion of the mathematics involved. When formulas or derivations are considered, Lloyd describes the mathematics such that less advanced math students will find the tutorial accessible. Prerequisites for this tutorial are an understanding of logarithms, and at least a year of high-school algebra.

About the Instructor(s):

Professor Seth Lloyd is a principal investigator in the Research Laboratory of Electronics (RLE) at the Massachusetts Institute of Technology (MIT). He received his A.B. from Harvard College in 1982, the Certificate of Advanced Study in Mathematics (Part III) and an M. Phil. in Philosophy of Science from Cambridge University in 1983 and 1984 under a Marshall Fellowship, and a Ph.D. in Physics in 1988 from Rockefeller University under the supervision of Professor Heinz Pagels.

From 1988 to 1991, Professor Lloyd was a postdoctoral fellow in the High Energy Physics Department at the California Institute of Technology, where he worked with Professor Murray Gell-Mann on applications of information to quantum-mechanical systems. From 1991 to 1994, he was a postdoctoral fellow at Los Alamos National Laboratory, where he worked at the Center for Nonlinear Systems on quantum computation. In 1994, he joined the faculty of the Department of Mechanical Engineering at MIT. Since 1988, Professor Lloyd has also been an adjunct faculty member at the Sante Fe Institute.

Professor Lloyd has performed seminal work in the fields of quantum computation and quantum communications, including proposing the first technologically feasible design for a quantum computer, demonstrating the viability of quantum analog computation, proving quantum analogs of Shannon’s noisy channel theorem, and designing novel methods for quantum error correction and noise reduction.

Professor Lloyd is a member of the American Physical Society and the Amercian Society of Mechanical Engineers.

Tutorial Team:

Yoav Kallus is an Omidyar Fellow at the Santa Fe Institute. His research at the boundary of statistical physics and geometry looks at how and when simple interactions lead to the formation of complex order in materials and when preferred local order leads to system-wide disorder. Yoav holds a B.Sc. in physics from Rice University and a Ph.D. in physics from Cornell University. Before joining the Santa Fe Institute, Yoav was a postdoctoral fellow at the Princeton Center for Theoretical Science in Princeton University.

How to use Complexity Explorer: How to use Complexity Explore
Prerequisites: At least one year of high-school algebra
Like this tutorial? 


Syllabus

  1. Introduction
  2. Forms of Information
  3. Information and Probability
  4. Fundamental Formula of Information
  5. Computation and Logic: Information Processing
  6. Mutual Information
  7. Communication Capacity
  8. Shannon’s Coding Theorem
  9. The Manifold Things Information Measures
  10. Homework