Hypothes.is | Chris Aldrich

Annotation Colors Using Hypothes.is: Problems and Solutions

I frequently see the feature request from Hypothes.is users to have the tool add the ability to choose different colors for highlights.

The first occurrence of the issue is probably documented here: https://github.com/hypothesis/vision/issues/123, by the head of the company.

I suspect it may take a while before such a color feature might be built in, if ever. (Here I’ll note that I don’t work for or speak for the company or any of the other open source developers on the project, but I am one what one would consider a “heavy user”.) If they have the time (I know they’re very busy), perhaps they may chime in with a potential roadmap or other ideas.

Color highlights are a difficult user interface problem

While I’m thinking about it, in an academic context for students, colors may be slightly better indicators of different users’ annotations of a particular text as a means of differentiating one annotator from another more subtly, particularly on texts that are extensively marked up.

Just this difference points out what a mixed bag of functionality colored highlights brings from a usability, design, and user interface perspective. While colored highlights is a seemingly “simple” sounding feature in the analog world where a single document is only annotated by one user, mapping it into a digital shared context is a difficult engineering problem to navigate and solve for. What if your color meanings aren’t the same as those of another?, for example.

While colors can be useful for individuals, do they have the same place in a social annotation product?

I already find it difficult to annotate heavily annotated pages that all use the same color, much less a rainbow of others’ colors. (If this is also you, I’ll note that there’s a handy “eye” icon in the annotation drawer that will allow you to turn them on/off.)

Potential Color Highlight Hacks

While the value of colors may be useful in some contexts, you could potentially use a few other features, functionality, and methods to creatively achieve a similar feature in Hypothes.is for yourself today. Below are a few potential creative “hacks” that some might try.

Use the tagging functionality

You could use the tagging system to create specific tags to stand in for your desired colors: As an example, in some systems I might use the following color designators:

Yellow—general highlights and highlights which don’t fit under another category below
Orange—Vocabulary word; interesting and/or rare word
Green—Reference to read
Blue—Interesting Quote
Gray—Typography Problem
Red—Example to work through

Instead of colors in Hypothesis, for example, one could use the tags “words” or “vocabulary”, “reference” or “citation“, “typo“, “quotes“, or “examples” to stand in for these particular “colors” respectively. I sometimes practice some of these which you can find by clicking on the links, though you may note that in practice I use other tags for them.

In some sense, this is what the software would be doing, particularly with regard to search for these after-the-fact. If you wanted a list of all your “citations” for example, you’d have to search for the color for that and be able to find them all, presuming this search functionality existed with such a color feature. This isn’t really much different than simply tagging all those particular highlights with words like “citation” or “reference” in the first place.

Use the Group Functionality

You could created different “groups” (private or public) to stand in for the colors you wish you had, thus a “yellow group” could be used for one “color” of highlight and a “green group” for another. ( See Annotating with Groups for more details.)

Switching between groups for annotating isn’t going to be drastically different than a user interface for switching colors of highlighter. The one drawback (or perhaps it’s a feature?) here is that you will only be able to see one “color” at a time.

Roll your own solution with open source

As ever, with some work, you could self-host the open source software and modify your copy to add this functionality in for yourself.

Some clever hacks in your browser with CSS might also give you your preferred output. I know some users have done custom work to the Hypothes.is UI in the past: eg. https://tomcritchlow.com/2019/02/12/annotations/, see his gist at the bottom of the post.

Another custom solution which may give you ideas can also be found at https://web.hypothes.is/blog/do-it-yourself-anchoring-and-the-evolution-of-the-hypothesis-toolkit/.

Perhaps adding custom classes on the tags or usernames might allow people the ability to target highlights on a page so that one could define custom CSS rules for each highlight using either usernames of tags as well? Of course, just like the “eye icon” described above, I’m sure there are times that people will appreciate the ability to turn these colors on and off. I personally don’t want the clean interface dressed up in Josephs Amazing Technicolor Annotation Dreamcoat.

Hypothes.is + Obsidian = Hypothesidian for easier note taking and formatting

Anyone who knows me knows that I love Hypothes.is for all my online highlighting, annotating, and general note taking. They also know that if one isn’t actively using their notes to some better end, then it’s likely not worth having taken them at all, so I store mine in markdown in Obsidian for future-proofing and portability.

Hypothes.is + Obsidian

A while back I came across RoamHacker’s work to dovetail Hypothes.is for use in Obsidian and finally managed to get it up and running with my Obsidian vault. I’ve previously outlined a method for pulling in my notes from Hypothes.is using RSS, however this doesn’t give one any formatting capabilities and it also doesn’t provide any of the Hypothes.is tags as RSS has no layer for taxonomies.

RoamHacker’s work, which leverages the Templater Plugin for Obsidian, fixes both of these problems. I suspect that I’ll keep my prior method in place to create the individual notes, but use this additional work to clean up my fleeting notes from Hypothes.is in my actual commonplace book. Since there’s no server involved, it’s harder to automate the entire process so that every time you create notes they’re automatically ported across either in real-time or in batches every few hours.

Formatting your notes

I did spend some time last night to modify some of RoamHacker’s code to re-format the annotations to better suit my current notes format and layout. I’m excerpting the most relevant part below, but the entire Gist can also be downloaded or further modified for easier copy/pasting into one’s own vault for the needed set up.

I’ve only modified the section of the original Gist at the bottom that follows the line:
/* TEMPLATE STARTS HERE */

The changes still keep all the relevant data fields, but reorder them and add a bit of formatting to fit the layout and the way I use my Obsidian notebook. I changed the formatting so that tags in Hypothes.is are turned into [[wikilinks]] rather than #⁠hashtags as in the original. (The original also doesn’t do so well with multi-word tags, which I use quite a lot.)

Hopefully the small changes I’ve made and comparison with the original Gist will allow those who aren’t as code-savvy to better understand the template and potentially let them make changes to suit their own needs.

/* TEMPLATE STARTS HERE */ 
if (tp.file.content.length==0) {
  //likely a new document, insert front matter
  tR += `---\n`;
  tR += `fileType: HypothesisAnnotations\n`;
  tR += `creationDate: ${tp.date.now('YYYY-MM-DD')} \n`;
  tR += `annotationDate: ${articleAnnotations[0].created.substring(0,10)}\n`;
  tR += `uri: ${articleAnnotations[0].uri}\n`;
  tR += `---\n`;
}

tR += `# ${articleAnnotations[0].title}\n`
tR += `URL: ${articleAnnotations[0].uri}\n\n`

for( a of articleAnnotations) {
  let tags = '';
  let user = '';
  if(a.tags.length>0) tags = ' ' + (a.tags.map(t=> '[['+ t + ']]')).join(' ');
  if(insertUser) user = ' _(' + a.user.replace('acct:','').replace('@hypothes.is','') + ')_';
  if(a.text) tR += `${a.text}\n&mdash;[[${user}]]\n\n`;
  tR += `## Source \n`;
  tR += `> ${a.highlight}[^1]\n\n`;
  tR += `[^1]: [${articleAnnotations[0].title}](${articleAnnotations[0].uri}) | [syndication link](tk) \n`;
  tR += `\n---\ntags: \nlinks: ${tags} \n- broader terms (BT):  \n- narrower terms (NT):  \n- related terms (RT):  \n- used for (UF) or aliases:  \nconnected ideas:  \nMOC:  \n\n---\n`;
}
%>

Differentiating online variations of the Commonplace Book: Digital Gardens, Wikis, Zettlekasten, Waste Books, Florilegia, and Second Brains

A fluorescence of note taking tools

Over the past three or so years there has been a fluorescence of digital note taking tools and platforms.

Some of these include:

Open source projects like Org Mode, Logseq, Foam, Jupyter, Trilium, Databyss, Athens, Dendron, Anagora, and Hypothes.is.

Closed sourced projects like: Roam Research, Notion, Knovigator, Amplenote, RemNote, Memex, Nototo, nvUltra, and Are.na.

Some are based on earlier incarnations of note taking and writing tools like OneNote, Evernote, Simple Note, TiddlyWiki, DEVONthink, Scrivener, etc.

This brief list doesn’t take into account a sea of other mobile apps and platforms in addition to a broad array of social media platforms that people use for similar note taking or annotations.

My particular interest in some of this note taking field comes in the growing number of people who are working in public and sharing their notes in online settings with others. This has been happening organically since the rise of the internet and has happened on blogs within the blogosphere and on personal and communal wikis.

As was highlighted (pun intended) at the recent I Annotate 2021 conference, the note taking space seems to have been coming to a new boil. With the expansion of the ideas of keeping a zettelkasten or a digital garden, these versions of notebooks seem to be a significant part of this new note taking craze.

One thing I have noticed, however, is a dramatic lack of continuity in the history of note taking within the longue durée of Western civilization. (Other cultures including oral cultures have similar traditions, but for our purposes here, I won’t go into them except to say that they’re highly valuable, spectacularly rich, and something of which we should all be aware.)

Many of these products are selling themselves based on ideas or philosophies which sound and even feel solid, but they’re completely ignoring their predecessors to the tune of feeling like they’re trying to reinvent the wheel. As a result, some of the pitches for these products sound like they’re selling snake oil rather than tried and true methods that go back over 2,000 years of intellectual history. I can only presume that modern education is failing us all dramatically. People are “taught” (maybe told is the better verb) to take notes in school, but they’re never told why, what to do with them, or how to leverage them for maximum efficiency. Perhaps the idea has been so heavily imbued into our culture we’ve honestly forgotten the basic parts and reasoning behind it?

Even Vannevar Bush’s dream of the Memex as stated in his article As We May Think (The Atlantic, 1945), which many of these note taking applications might point to as an inspiration, ignores this same tradition and background, so perhaps these app creators and users aren’t all to blame?

Delineating Online Forms

I’ve been doing some serious reading and research into these traditions to help uncover our missing shared history. I’ll write something longer and more specific about them at a later date.

In the meanwhile, I want to outline just a bit about the various flavors as they relate to some of the more public online versions that I see in the related internet spaces. I hope to help better delineate what they have in common, how they differ, and what they may still add to the mix to get us to a more robust version of Bush’s dream.

Other’s thoughts and comments about these various incarnations and their forms and functions are both encouraged and appreciated.

Commonplace books

Historically commonplace books are one of the oldest and most influential structures in the note taking, writing, and thinking space. They have generally been physical books written by hand that contain notes which are categorized by headings (or in a modern context categories or tags. Often they’re created with an index to help their creators find and organize their notes.

They originated in ancient Greece and Rome out of the thought of Aristotle and Cicero as a tool for thinking and writing and have generally enjoyed a solid place in history since. A huge variety of commonplaces have been either copied by hand or published in print book form over the centuries.

Most significant thinkers, writers, and creators throughout history have kept something resembling a commonplace book. While many may want to attribute the output of historical figures like Erasmus, Newton, Darwin, Leibnitz, Locke, or Emerson to sheer genius (and many often do), I might suggest that their works were the result of sustained work of creating personal commonplace books—somewhat like a portable Google search engine for their day, but honed to their particular interests. (One naturally can’t ignore their other many privileges like wealth, education, and time to do this work, which were also certainly a significant factor in their success.)

Many people over the past quarter of a century have used a variety of digital forms to keep digital commonplace books including public versions on blogs, wikis, and other software for either public or private consumption.

Florilegium

Florilegia are a subcategory of commonplace book starting around 900 CE but flourishing in the 12th and 13th centuries and primarily kept by theologians and preachers. The first were a series of short excerpted passages often arranged in order of their appearance in a single text, but eventually were arranged systematically under discrete headings. Medieval florilegia where overwhelmingly, and often exclusively, concerned with religious topics from the works of scriptures, the moral dicta of the Doctors of the Church, and—less frequently—the teachings of approved, classical moral philosophers. The idea and form of florilegium generally merged back into the idea of the commonplace book which had renewed interest and wide popularity during the Renaissance.

These didn’t add any new or innovative features over what had come before. Perhaps, if anything, they were a regression because they so heavily focused only on religion as a topic.

Few (if any) examples of florilegia can be found in modern digital contexts. Though I have seen some people talk about using digital note taking tools for religious study, I have yet to see public versions online.

Zettelkästen

Born out of the commonplace tradition with modifications by Conrad Gessner (1516-1565) and descriptions by Johann Jacob Moser (1701–1785), the Zettelkasten, a German word translated as “slip box”, is generally a collection of highly curated atomic notes collected on slips of paper or index cards. Zettelkasten were made simpler to create and maintain with the introduction of the mass manufacture of index cards (and card boxes and furniture) in the early 20th century. Slips of paper which were moveable within books or files and later on index cards were a significant innovation in terms of storing and organizing a commonplace book.

Generally zettels (or cards) are organized by topics and often contain dates and other taxonomies or serialized numbers as a means of linking them to other cards within the system. The cross linking of these cards (and thus ideas) were certainly a historical physical precursor of the internet we have today, simply in digital form.

Almost all the current references I’ve seen online to Zettelkasten mention Niklas Luhmann as their inspiration, but none of them reference any other well-known historical examples despite the fact the idea has been around and evolving for several centuries now.

This productivity system and sets of digital tools around it came to greater attention in Germany in 2013 with the exhibition “Zettelkästen: Machines of Fantasy” at the Museum of Modern Literature, Marbach am Neckar and in 2014 with the launch of the zettelkasten.de website. A subsequent boost in the English speaking world occurred following the publication of Sönke Ahrens’s book How to Take Smart Notes – One Simple Technique to Boost Writing, Learning and Thinking – for Students, Academics and Nonfiction Book Writers in February 2017. The recent ability to use platforms like Roam Research, Obsidian, Notion, et al. has helped to fan the flames of their popularization.

More often than not, most of these digital tools (like their card-based predecessors) are geared toward private personal use rather than an open public model. Roam Research and Obsidian Publish have features which allow public publishing. TiddlyWiki is also an excellent tool for this as its so-called Tiddlers have a card-based appearance and can be placed in custom orders as well as transcluded, but again not many are available to the online public.

Waste books/Sudelbücher

This sub-genre of notebooks comes out of the tradition of double-entry book keeping where accountants often kept a daily diary of all transactions in chronological order. These temporary notes were then later moved into a more permanent accounting ledger and the remaining book was considered “waste”.

In the commonplace book tradition, these books for temporary notes or (fleeting notes in a Zettelkasten framing), might eventually be copied over, expanded, and indexed into one’s permanent commonplace collection.

In modern digital settings, one might consider some of the ephemeral social media stream platforms like Twitter to be a digital version of a waste book, though to my knowledge I may be the first person to suggest this connection. (To be clear, others have certainly mentioned Twitter as being a waste and even a wasteland.)

Wikis

Inspired, in part, by Apple’s HyperCard, Ward Cunningham created the first public wiki on his website on March 25, 1995. Apple had designed a system allowing users to create virtual “card stacks” supporting links among the various cards (sound familiar?). HyperCard was designed as a single user system.

Wikis allowed multiple users to author and edit pages on the web with a basic web browser. They were also able to create meaningful links and associations between pages, whether they existed or not using [[WikiLinks]]. They were meant to allow the average visitor to participate in an ongoing process of creation and collaboration.

Here there is some innovative user interface as well as the ability to collaborate with others in keeping a commonplace book. Transclusion of one page into another is a useful feature here.

Personal wikis have been used (as have many blogs) for information aggregation and dissemination over the years in a manner similar to their historical predecessors.

Second brain

Second brain is a marketing term which stands in for the idea of the original commonplace book. It popped up in the note taking context in early 2017 for promoting the use of commonplace books techniques using Tiago Forte’s expensive online course Building a Second Brain which focused on capturing, organizing, and sharing your knowledge using (digital) notes. It is a platform agnostic method for improving productivity wholly using the commonplace underpinning.

Google searches for this term will be heavily mixed in with results about the gastrointestinal system being the body’s “second brain”, the enteric nervous system, second brain tumors, a debunked theory that dinosaurs had two brains, and other general health-related topics.

Some websites, personal wikis and other online versions will use the phrase second brain, but they generally have no innovative features that are missing from prior efforts. Again, I view the phrase simply as marketing with no additional substance.

Digital Gardens

Informed heavily by their cultural predecessors in commonplace books, zettelkasten, and wikis, digital gardens are digital first note collections which are primarily public by default and encourage the idea of working in public.

Digital Gardens arose more formally in 2019 and 2020 out of the work and influence of Mark Bernstein’s 1998 essay Hypertext Gardens: Delightful Vistas, Ward Cunningham’s Smallest Federated Wiki (which just celebrated it’s 10th anniversary), Mike Caulfield’s essays including The Garden and the Stream: A Technopastoral as well as some influence from the broader IndieWeb Community and their focus on design and user interface.

Digital garden design can often use the gardening metaphor to focus attention on an active tending and care of one’s personal knowledge base and building toward new knowledge or creations. The idea of planting a knowledge “seed” (a note), tending it gradually over time with regular watering and feeding in a progression of 🌱 Seedlings → 🌿 Budding → 🌳 Evergreen is a common feature.

There are a growing number of people with personal digital gardens in public. Many are built on pre-existing wiki software like WikiMedia, the Smallest Federated Wiki, or TiddlyWiki, static site generators like Jekyll, note taking platforms like Obsidian Publish and Roam Research, or even out of common blogging software like WordPress. A growing common feature of these platforms is that they not only link out to resources on the open web, but contain bidirectional links within themselves using either custom code (in a wiki-like manner) or using the W3C Webmention specification.

The Future?

With luck, application and platform designers and users will come to know more about the traditions, uses, and workflows of our rich cultural note taking history. Beyond this there are a few innovations, particularly in the public-facing arena which could be useful, but which aren’t broadly seen or available yet.

Still missing from the overall personal knowledge and note taking space is a more tightly integrated version of both a garden and a stream (in Mike Caulfield’s excellent framing) that easily allows interaction between the two arenas. Some of the more blog-based sites with notes, bookmarks, articles and IndieWeb friendly building blocks like Webmention, feeds (RSS, JSON Feed, h-feed), Micropub, and Microsub integrations may come the closest to this ideal.

One of the most fascinating recent entrants on the scene is Flancian’s Anagora which he uses as a personal commonplace book in a wiki-esque style. Over other incarnations it also has the ability to pull in and aggregate the notes of other digital commonplace books to create a larger marketplace of ideas. It also includes collaborative note taking space using Etherpad, which I’ve seen as a standalone tool, but never integrated into a digital commonplace book.

Ultimately, my dream—similar to that of Bush’s—is for individual commonplace books to be able to communicate not only with their users in the Luhmann-esqe sense, but also communicate with each other.

Niklas Luhmann apparently said:

Ohne zu schreiben, kann man nicht denken; jedenfalls nicht in anspruchsvoller, anschlussfähiger Weise.

(Translation) You cannot think without writing; at least not in a sophisticated, connectable way.

I think his conceptualization of “connectable” was much more limited and limiting than he might have guessed. Vannevar Bush, as the academic advisor of Claude Shannon, the godfather of the modern digital age, was more prepared to envision it.

(Luhmann’s “you” in his quote is obviously only a Western cultural referent which erases the existence of oral based cultures which have other ways to do their sophisticated thinking. His ignorant framing on the topic shouldn’t be a shared one.)

This post has grown out of my own personal commonplace book, portions of which are on housed on my blog, in a wiki, and in a private repository of which I hope to make more public soon. Further thoughts, ideas and expansions of it are more than welcome.

I’ve slowly been updating pieces of the history along with examples on shared commonplaces in both the IndieWeb Wiki and Wikipedia under the appropriate headings. Feel free to browse those or contribute to them as you would, at least until our digital commonplace books can communicate with each other.

I’d also invite those who are interested in this topic and who have or want online spaces to do this sort of thing to join us at the proposed upcoming Gardens and Streams II IndieWebCamp Pop up session which is being planned for later this Summer or early Fall. Comment below, stop by the page or chat to indicate your interest in attending.