Many readers often ask me for resources for delving into the basics of information theory. I hadn’t posted it before, but the Santa Fe Institute recently had an online course Introduction to Information Theory through their Complexity Explorer, which has some other excellent offerings. It included videos, fora, and other resources and was taught by the esteemed physicist and professor Seth Lloyd. There are a number of currently active students still learning and posting there.
Introduction to Information Theory
About the Tutorial:
This tutorial introduces fundamental concepts in information theory. Information theory has made considerable impact in complex systems, and has in part co-evolved with complexity science. Research areas ranging from ecology and biology to aerospace and information technology have all seen benefits from the growth of information theory.
In this tutorial, students will follow the development of information theory from bits to modern application in computing and communication. Along the way Seth Lloyd introduces valuable topics in information theory such as mutual information, boolean logic, channel capacity, and the natural relationship between information and entropy.
Lloyd coherently covers a substantial amount of material while limiting discussion of the mathematics involved. When formulas or derivations are considered, Lloyd describes the mathematics such that less advanced math students will find the tutorial accessible. Prerequisites for this tutorial are an understanding of logarithms, and at least a year of high-school algebra.
About the Instructor(s):
Professor Seth Lloyd is a principal investigator in the Research Laboratory of Electronics (RLE) at the Massachusetts Institute of Technology (MIT). He received his A.B. from Harvard College in 1982, the Certificate of Advanced Study in Mathematics (Part III) and an M. Phil. in Philosophy of Science from Cambridge University in 1983 and 1984 under a Marshall Fellowship, and a Ph.D. in Physics in 1988 from Rockefeller University under the supervision of Professor Heinz Pagels.
From 1988 to 1991, Professor Lloyd was a postdoctoral fellow in the High Energy Physics Department at the California Institute of Technology, where he worked with Professor Murray Gell-Mann on applications of information to quantum-mechanical systems. From 1991 to 1994, he was a postdoctoral fellow at Los Alamos National Laboratory, where he worked at the Center for Nonlinear Systems on quantum computation. In 1994, he joined the faculty of the Department of Mechanical Engineering at MIT. Since 1988, Professor Lloyd has also been an adjunct faculty member at the Sante Fe Institute.
Professor Lloyd has performed seminal work in the fields of quantum computation and quantum communications, including proposing the first technologically feasible design for a quantum computer, demonstrating the viability of quantum analog computation, proving quantum analogs of Shannon’s noisy channel theorem, and designing novel methods for quantum error correction and noise reduction.
Professor Lloyd is a member of the American Physical Society and the Amercian Society of Mechanical Engineers.
Tutorial Team:
Yoav Kallus is an Omidyar Fellow at the Santa Fe Institute. His research at the boundary of statistical physics and geometry looks at how and when simple interactions lead to the formation of complex order in materials and when preferred local order leads to system-wide disorder. Yoav holds a B.Sc. in physics from Rice University and a Ph.D. in physics from Cornell University. Before joining the Santa Fe Institute, Yoav was a postdoctoral fellow at the Princeton Center for Theoretical Science in Princeton University.
I know many people who could identify a fake Louis Vuitton (LVMH) purse, a knock off Christian Louboutin, or a sham Rolex, but who simultaneously are overly religious about their food brands and topics like organic food and couldn’t similarly identify the fakes they’re eating because of fraud in food labeling and misdirection and legerdemain within the food supply chain. Finally there’s a course to help everyone become smarter consumers.
The food industry is one of the most important commercial sectors in the world. Everyone uses it, but how many people abuse it? As we witness the increasing globalisation of the supply chain, a growing challenge is verifying the questionable identity of raw materials in the food we eat.
In this course we will look at topical issues concerning ‘food fraud’ and explore ways in which analytical chemistry can help in its identification and prevention. We’ll share fascinating examples, such as the history of white bread and a surprising ingredient once found in bitter beer.
The University of East Anglia has joined forces with the world-renowned Institute of Food Research (IFR) to bring you this unique course. You’ll be led by Kate Kemsley, a specialist in the use of advanced instrumentation for measuring the chemical composition of food materials. Course content is linked with UEA’s MChem postgraduate programme, which supports final-year students’ practical research projects in this area of science.
Top chefs and Harvard researchers explore how everyday cooking and haute cuisine can illuminate basic principles in physics and engineering, and vice versa.
A most interesting article on The Great Courses (TGC) appeared in the New York Times on Saturday. TGC has been featured in newspaper articles before: scads of articles, in fact, over the last 20-plus years. But those articles (at least the ones I’m aware of and I am aware of most of them) have always focused on the content of TGC offerings: that they are academic courses offered up on audio/video media. This article, written by the Times’ TV critic Neil Genzlinger, is different. It focuses on TGC as a video production company and on TGC courses as slick, professional, high-end television programs.
My goodness, how times have changed.
Long-time readers of this blog will recall my descriptions of TGC in its early days. I would rehash a bit of that if only to highlight the incredible evolution of the company from a startup to the polished gem it is today.
I made my first course back in May of 1993: the first edition of “How to Listen to and Understand Great Music”. We had no “set”; I worked in front of a blue screen (or a “traveling matte”). The halogen lighting created an unbelievable amount of heat and glare. The stage was only about 6 feet deep but about 20 feet wide. With my sheaf of yellow note paper clutched in my left hand, I roamed back-and-forth, in constant motion, teaching exactly the way I did in the classroom. I made no concessions to the medium; to tell the truth, it never occurred to me or my director at the time that we should do anything but reproduce what I did in the classroom. (My constant lateral movement did, however, cause great consternation among the camera people, who were accustomed to filming stationary pundits at CNN and gasbags at C-span. One of our camera-dudes, a bearded stoner who will remain nameless kept telling me “Man . . . I cannot follow you, man. Please, man, please!” He was a good guy though, and offered to “take my edge off” by lighting me up during our breaks. I wisely declined.)
We worked with a studio audience in those days: mostly retirees who were free to attend such recording sessions, many of whom fell asleep in their chairs after lunch or jingled change in their pockets or whose hearing aids started screaming sounds that they could not hear but I most certainly did. Most distracting were the white Styrofoam coffee cups; in the darkened studio their constant (if irregular) up-and-down motion reminded me of the “bouncing ball” from the musical cartoons of the 1930s, ‘40s, and ‘50s.
I could go on (and I will, at some other time), though the point is made: in its earliest days TGC was simply recording more-or-less what you would hear in a classroom or lecture hall. I am reminded of the early days of TV, during which pre-existing modes of entertainment – the variety show, theatrical productions, puppet shows – were simply filmed and broadcast. In its earliest permutation, the video medium did not create a new paradigm so much as record old ones. But this changed soon enough, and the same is true for TGC. Within a few years TGC became a genuine production company, in which style, look, and mode of delivery became as important as the content being delivered. And this is exactly as it should be. Audio and video media demand clarity and precision; the “ahs” and “ums” and garbled pronunciations and mismatched tenses that we tolerate in a live lecture are intolerable in media, because we are aware of the fact that in making media they can (and should) be corrected.
Enough. Read the article. Then buy another TGC course; preferably one of mine. And while watching and/or listening, let us be aware, as best as we can, of the tens-of-thousands of hours that go into making these courses – these productions – the little masterworks that they indeed are.
My response to his post with some thoughts of my own follows:
This is an interesting, but very germane, review. As someone who’s both worked in the entertainment industry and followed the MOOC (massively open online courseware) revolution over the past decade, I very often consider the physical production value of TGCs offerings and have been generally pleased at their steady improvement over time. Not only do they offer some generally excellent content, but they’re entertaining and pleasing to watch. From a multimedia perspective, I’m always amazed at what they offer and that generally the difference between the video versus the audio only versions isn’t as drastic as one might otherwise expect. Though there are times that I think that TGC might include some additional graphics, maps, etc. either in the course itself or in the booklets, I’m impressed that they still function exceptionally well without them.
Within the MOOC revolution, Sue Alcott’s Coursera course Archaeology’s Dirty Little Secrets is still by far the best produced multi-media course I’ve come across. It’s going to take a lot of serious effort for other courses to come up to this level of production however. It’s one of the few courses which I think rivals that of The Teaching Company’s offerings thus far. Unfortunately, the increased competition in the MOOC space is going to eventually encroach on the business model of TGC, and I’m curious to see how that will evolve and how it will benefit students. Will TGC be forced to offer online fora for students to interact with each other the way most MOOCs do? Will MOOCs be forced to drastically increase their production quality to the level of TGC? Will certificates or diplomas be offered for courseware? Will the subsequent models be free (like most MOOCs now), paid like TGC, or some mixture of the two?
One area which neither platform seems to be doing very well at present is offering more advanced coursework. Naturally the primary difficulty is in having enough audience to justify the production effort. The audience for a graduate level topology class is simply far smaller than introductory courses in history or music appreciation, but those types of courses will eventually have to exist to make the enterprises sustainable – in addition to the fact that they add real value to society. Another difficulty is that advanced coursework usually requires some significant work outside of the lecture environment – readings, homework, etc. MOOCs seem to have a slight upper hand here while TGC has generally relied on all of the significant material being offered in a lecture with the suggestion of reading their accompanying booklets and possibly offering supplementary bibliographies. When are we going to start seeing course work at the upper-level undergraduate or graduate level?
The nice part is that with evolving technology and capabilities, there are potentially new pedagogic methods that will allow easier teaching of some material that may not have been possible previously. (For some brief examples, see this post I wrote last week on Latin and the digital humanities.) In particular, I’m sure many of us have been astounded and pleased at how Dr. Greenberg managed the supreme gymnastics of offering of “Understanding the Fundamentals of Music” without delving into traditional music theory and written notation, but will he be able to actually offer that in new and exciting ways to increase our levels of understanding of music and then spawn off another 618 lectures that take us all further and deeper into his exciting world? Perhaps it comes in the form of a multimedia mobile app? We’re all waiting with bated breath, because regardless of how he pulls it off, we know it’s going to be educational, entertaining and truly awe inspiring.
Following my commentary, Scott Ableman, the Chief Marketing Officer for TGC, responded with the following, which I find very interesting:
Chris, all excellent observations (and I agree re Alcott’s course). I hope you’ll be please to learn that the impact of MOOCs, if any, on The Great Courses has been positive, in that there is a rapidly growing awareness and interest in the notion that lifelong learning is possible via digital media. As for differentiating vs. MOOCs, people who know about The Great Courses generally find the differences to be self-evident:
Curation: TGC scours the globe to find the world’s greatest professors;
Formats: The ability to enjoy a course in your car or at home on your TV or on your smartphone, etc.;
Lack of pressure: Having no set schedule and doing things at your own pace with no homework or exams (to be sure, there are some for whom sitting at a keyboard at a scheduled time and taking tests and getting a certificate is quite valuable, but that’s a different audience).
The Great Courses once were the sole claimant to a fourth differentiator, which is depth. Obviously, the proliferation of fairly narrow MOOCs provides as much depth on many topics, and in some cases addresses your desire for higher level courses. Still TGC offers significant depth when compared to the alternatives on TV or audio books. I must say that I was disappointed that Genzlinger chose to focus on this notion that professors these days “don’t know how to lecture.” He suggests that TGC is in the business of teaching bad lecturers how to look good in front of a camera. This of course couldn’t be further from the truth. Anybody familiar with The Great Course knows that among its greatest strengths is its academic recruiting team, which finds professors like Robert Greenberg and introduces them to lifelong learners around the world.
Since the beginning of January, I’ve come back to regularly browsing and using the website Rap Genius. I’m sure that some of the education uses including poetry and annotations of classics had existed the last time I had visited, but I was very interested in seeing some of the scientific journal article uses which I hadn’t seen before. Very quickly browsing around opened up a wealth of ideas for using the platform within the digital humanities as well as for a variety of educational uses.
Overview of Rap Genius
Briefly, the Rap Genius website was originally set up as an innovative lyrics service to allow users to not only upload song lyrics, but to mark them up with annotations as to the meanings of words, phrases, and provide information about the pop-culture references within the lyrics themselves. (It’s not too terribly different from Google’s now-defunct Sidewicki or the impressive Highbrow, textual annotation browser, but has some subtle differences as well as improvements.)
Users can use not only text, but photos, video, and even audio to supplement the listings. Built-in functionality includes the ability to link the works to popular social media audio services SoundCloud, and Spotify as well as YouTube. Alternately one might think of it as VH1’s “Pop-up Video”, but for text on the Internet. Ultimately the site expanded to include the topics of rock, poetry, and news. The rock section is fairly straightforward following the format of the rap section while the poetry section includes not only works of poetry (from The Rime of the Ancient Mariner to the King James version of The Bible), but also plays (the works of William Shakespeare) and complete novels (like F. Scott Fitzgerald’s The Great Gatsby.) News includes articles as well as cultural touchstones like the 2013 White House Correspondents’ Dinner Speech and the recent State of the Union. Ultimately all of the channels within Rap Genius platform share the same types of functionality, but are applied to slightly different categories to help differentiate the content and make things easier to find. Eventually there may be a specific “Education Genius” (or other) landing page(s) to split out the content in the future depending on user needs.
On even its first blush, I can see this type of website functionality being used in a variety of educational settings including Open Access Journals, classroom use, for close readings, for MOOCs, publishing in general, and even for maintaining simple-to-use websites for classes. The best part is that the ecosystem is very actively growing and expanding with a recent release of an iPhone app and an announcement of a major deal with Universal to license music lyrics.
General Education Use
To begin with, Rap Genius’ YouTube channel includes an excellent short video on how Poetry Genius might be used in a classroom setting for facilitating close-readings. In addition to the ability to make annotations, the site can be used to maintain a class specific website (no need to use other blogging platforms like WordPress or Blogger for things like this anymore) along with nice additions like maintaining a class roster built right in. Once material begins to be posted, students and teachers alike are given a broad set of tools to add content, make annotations, ask questions, and provide answers in an almost real-time setting.
MOOC Use Cases
Given the rapid growth of the MOOC-revolution (massively open online courseware) over the past several years, one of the remaining difficulties in administering such a class can hinge not only on being able to easily provide audio visual content to students, but allow them a means of easily interacting with it and each other in the learning process. Poetry Genius (aka Education Genius) has a very interesting view into solving both of these problems, and, in fact, I can easily see the current version of the platform being used to replace competing platforms like Coursera, EdX, Udacity and others in a whole cloth fashion.
Currently most MOOC’s provide some type of simple topic-based threaded fora in which students post comments and questions as well as answers. In many MOOCs this format becomes ungainly because of the size of the class (10,000+ students) and the quality of the content which is being placed into it. Many students simply eschew the fora because the time commitment per amount of knowledge/value gained is simply not worth their while. Within the Poetry Genius platform, students can comment directly on the material or ask questions, or even propose improvements, and the administrators (the professor or teaching assistants in this case) can accept, reject or send feedback request to students to amend their work and add it to the larger annotated work. Fellow classmates can also vote up or down individual comments.
As I was noticing the interesting educational-related functionality of the Rap Genius platform, I ran across what is presumably the first MOOC attempting to integrate the platform into its pedagogical structure. Dr. Laura Nasrallah’s HarvardX course “Early Christianity: The Letters of Paul,” which started in January, asks students to also create Poetry Genius accounts to read and comment on the biblical texts which are a part of the course. The difficult portion of attempting to use Poetry Genius for this course is the thousands of “me-too” posters who are simply making what one might consider to be “throw-away” commentary rather than the intended “close reading” commentary for a more academic environment. (This type of posting is also seen in many of the fora-based online courses.) Not enough students are contributing substantial material, and when they are, it needs to be better and more quickly edited and curated into the main post to provide greater value to students as they’re reading along. Thus when 20,000 students jump into the fray, there’s too much initial chaos and the value that is being extracted out of it upon initial use is fairly limited – particularly if one is browsing through dozens of useless comments. It’s not until after-the-fact – once comments have been accepted/curated – that the real value will emerge. The course staff is going to have to spend more time doing this function in real time to provide greater value to the students in the class, particularly given the high number of people without intense scholarly training just jumping into the system and filling it with generally useless commentary. In internet parlance, the Poetry Genius site is experiencing the “Robert Scoble Effect” which changes the experience on it. (By way of explanation, Robert Scoble is a technology journalist/pundit/early-adopter with a massive follower base. His power-user approach and his large following can drastically change his experience with web-based technology compared to the common everyday user. It can also often bring down new services as was common in the early days of the social media movement.)
Typically with the average poem or rap song, the commentary grows slowly/organically and is edited along the way. In a MOOC setting with potentially hundreds of thousands of students, the commentary is like a massive fire-hose which makes it seemingly useless without immediate real-time editing. Poetry Genius may need a slightly different model for using their platform in larger MOOC-style courses versus the smaller classroom settings seen in high school or college (10-100 students). In the particular case for “The Letters of Paul,” if the course staff had gone into the platform first and seeded some of the readings with their own sample commentary to act as a model of what is expected, then the students would be a bit more accepting of what is expected. I understand Dr. Nasrallah and her teaching assistants are in the system and annotating as well, but it should also be more obvious which annotations are hers (or those of teaching assistants) to help better guide the “discussion” and act as a model. Certainly the materials generated on Poetry Genius will be much more useful for future students who take the course in future iterations. Naturally, Poetry Genius exists for the primary use of annotation, while I’m sure that the creators will be tweaking classroom-specific use as the platform grows and user needs/requirements change.
In my mind, this type of platform can easily and usefully be used for publishing open access journal articles. In fact, one could use the platform to self-publish journal articles and leave them open to ongoing peer review. Sadly at present, there seems to be only a small handful of examples on the site, including a PLOS ONE article, which will give a reasonable example of some of the functionality which is possible. Any author could annotate and footnote their own article as well as include a wealth of photos, graphs, and tables giving a much more multimedia view into their own work. Following this any academic with an account could also annotate the text with questions, problems, suggestions and all of these can be voted up or down as well as be remedied within the text itself. Other articles can also have the ability to directly cross-reference specific sections of previously posted articles.
Individual labs or groups with “journal clubs” could certainly join in the larger public commentary and annotation on a particular article, but higher level administrative accounts within the system can also create a proverbial clean slate on an article and allow members to privately post up their thoughts and commentaries which are then closed to the group and not visible to the broader public. (This type of functionality can be useful for Mrs. Smith’s 10th grade class annotating The Great Gatsby so that they’re not too heavily influenced by the hundreds or possibly thousands of prior comments within a given text as they do their own personal close readings.) One may note that some of this type of functionality can already be seen in competitive services like Mendeley, but the Rap Genius platform seems to take the presentation and annotation functionalities to the next level. For those with an interest in these types of uses, I recommend Mendeley’s own group: Reinventing the Scientific Paper.
A Rap Genius representative indicated they were pursuing potential opportunities with JSTOR that might potentially expand on these types of opportunities.
Publishing
Like many social media related sites including platforms like WordPress, Tumblr, and Twitter, Rap Genius gives it’s users the ability to self-publish almost any type of content. I can see some excellent cross-promotional opportunities with large MOOC-type classes and the site. For example, professors/teachers who have written their own custom textbooks for MOOCs (eg. Keith Devlin’s Introduction to Mathematical Thinking course at Stanford via Coursera) could post up the entire text on the Poetry Genius site and use it not only to correct mistakes/typos and make improvements over time, but they can use it to discover things which aren’t clear to students who can make comments, ask questions, etc. There’s also the possibility that advanced students can actively help make portions clear themselves when there are 10,000+ students and just 1-2 professors along with 1-2 teaching assistants. Certainly either within or without the MOOC movement, this type of annotation set up may work well to allow authors to tentatively publish, edit, and modify their textbooks, novels, articles, journal articles, monographs, or even Ph.D. theses. I’m particularly reminded of Kathleen Fitzpatrick’s open writing/editing of her book Planned Obsolescence via Media Commons. Academics could certainly look at the Rap Genius platform as a simpler more user-friendly version of this type of process.
Other Uses
I’m personally interested in being able to annotate science and math related articles and have passed along some tips for the Rap Genius team to include functionality like mathjax to be able to utilize Tex/LaTeX related functionality for typesetting mathematics via the web in the future.
Naturally, there are a myriad of other functionalities that can be built into this type of platform – I’m personally waiting for a way to annotate episodes of “The Simpsons”, so I can explain all of the film references and in-jokes to friends who laugh at their jokes, but never seem to know why – but I can’t write all of them here myself.
Interested users can easily sign up for a general Rap Genius account and dig right into the interface. Those interested in education-specific functionality can request to be granted an “Educator Account” within the Rap Genius system to play around with the additional functionality available to educators. Every page in the system has an “Education” link at the top for further information and details. There’s also an Educator’s Forum [requires free login] for discussions relating specifically to educational use of the site.
Are there particular (off-label) applications you think you might be able to use the Rap Genius platform for? Please add your comments and thoughts below.
A few weeks ago I had communicated a bit with paleoanthropologist John Hawks. I wanted to take a moment to highlight the fact that he maintains an excellent blog primarily concerning his areas of research which include anthropology, genetics and evolution. Even more specifically, he is one of the few people in these areas with at least a passing interest in the topic of information theory as it relates to his work. I recommend everyone take a look at his information theory specific posts.
I’ve previously written a brief review of John Hawks’ (in collaboration with Anthony Martin) “Major Transitions in Evolution” course from The Learning Company as part of their Great Courses series of lectures. Given my interest in the MOOC revolution in higher education, I’ll also mention that Dr. Hawks has recently begun a free Coursera class entitled “Human Evolution: Past and Future“. I’m sure his current course focuses more on the area of human evolution compared with the prior course which only dedicated a short segment on this time period. Given Hawks’ excellent prior teaching work, I’m sure this will be of general interest to readers interested in information theory as it relates to evolution, biology, and big history.
I’d love to hear from others in the area of anthropology who are interested in information theoretical applications.
