Ideas for IndieWeb-ifying Hypothes.is

I use Hypothes.is regularly as part of my daily workflow. I’m also very interested in being able to “own” the data I generate with the tool and being able to keep it on my own digital commonplace book (aka website). As part of this, I’d like to be able to receive notifications from people publicly annotating, highlighting, and replying to my content and potentially display those directly on either my website in the comments section or as marginalia.

I’d promised to do a quick outline for the kind gang at Hypothes.is to outline how to make their product could be a bit more open and support some additional web standards to make it more IndieWeb friendly as well as to work toward supporting the Webmention protocol to send notifications of annotations on a page. A few weeks ago at IndieWebCamp New Haven I decided to finally sketch out some of the pieces which should be relatively easy for them to implement into the product. Below are some of the recommendations and some examples of what needs to be done to implement them into their platform to allow it to better interact with other content on the web. This post is in reply to a few prior conversations about Webmention, but primarily pertains to Microformats which will help in creating those. [1] [2] [3] [4]

Overview

To my knowledge Hypothes.is generates a hash for each annotation it has in the system and generates two separate, but related URLs for them. As an example, here are the two URLs for a response Jon Udell made on my website recently:

The first URL is where a stand-alone copy of the annotation lives on the web, separate from the content it is related to. screen capture of the content at URL https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow The second URL resolves to the page on which the annotation was made and both will automatically open up Hypothesis’ side drawer UI to the annotation in question and will–on most browsers–auto-scroll down the page to show the point at which the annotation was made. Essentially this second URL shows the annotation in-situ in conjunction with the Hypothes.is user interface. I’ll note that they can also have some human readable trailing data in the URL that indicates the site on which the annotation was made like so: https://hyp.is/_tLJyA-cEemE-qPndyfQow/boffosocko.com/?p=55708991. However, in practice, one could remove or replace the boffosocko.com and trailing portion with any other URL and the correct page will still resolve.

It is great that they make the first URL available with the relevant data. This in itself is very IndieWeb friendly to have each annotation in the system have its own stand-alone URL. Sadly all the data on this particular page seems to be rendered using JavaScript rather than being raw HTML. (See also js;dr.) This makes the page human readable, but makes it much more difficult for machines to read or parse these pages. I’d recommend three simple things to make Hypothes.is more (Indie)Web friendly:

  1. Render the annotation on the first URL example in full HTML instead of JavaScript;
  2. Add the appropriate microformats classes on those pages;
  3. Add the canonical URL for the page on which the annotation is in reference to either instead of or in addition to the Hypothes.is prefixed URL which already appears on these pages. Webmention functioning properly will require this canonical URL to exist on the page to be able to send notifications and have them be received properly.

These things would make these pages more easily and usefully parseable on the open web. If/when Hypothes.is may support Webmention (aka web notifications) then all of these prerequisite pieces will already be in place. In the erstwhile, even without Hypothes.is running code to support sending Webmentions, users could force manual Webmentions using services like Telegraph, mention-tech.appspot, or even personal endpoints generated on individual posts (see the one below) or on custom endpoint pages like mine on WordPress. Aaron Parecki’s article Sending your First Webmention from Scratch is a useful tutorial for those with little experience with Microformats or Webmention.

Types of Annotations and Microformats Markup

To my knowledge there are three distinct types of annotations that might occur which may need slightly different microformats mark up depending on the type. These are:

  1. Unassigned page notes (or sometimes orphaned page notes): For all intents and purposes are the equivalent of bookmarks (and are used this way by many) though they go by a different name within the service.
  2. Highlights of particular passages: In IndieWeb parlance, these are roughly equivalent to quotations of content.
  3. Highlights and annotations of particular passages: In IndieWeb terms these again are quotes of content which also have what might be considered a reply or comment to that segment of quoted text. Alternately the annotation itself might be considered a note related to what was highlighted, but I suspect from a UI and semantic viewpoint, treating these as replies is probably more apropos in the majority of cases.

Each of these can obviously have one or more potential tags as well. Some of the examples below include the p-category microformats for how these would logically appear. Using the example URL above and several others for the other cases, I’ll provide some example HTML with proper microformats classes to make doing the mark up easier. I’ve created some minimal versions of text and mark up, though Hypothes.is obviously includes much more HTML (and a variety of divs for CSS purposes. While some of the mark up is a bit wonky, particularly with respect to adding the hyp.is and the original posts’ canonical URLs, it could be somewhat better with some additional reworking of the presentation, but I wanted to change as little as possible of their present UI. For the minimal examples, I’ve stripped out the native Hypothes.is classes and only included the semantic microformats. Because microformats are only meant for semantic mark up, the developers should keep in mind it is good practice NOT to use these classes for CSS styling.

Page note with no annotations (bookmarks)

Example from https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow (but without the annotation portion)


<div class="h-entry">
    <a class="p-author h-card" href="https://hypothes.is/users/judell">judell</a>
    Public on <https://hyp.is/_tLJyA-cEemE-qPndyfQow/boffosocko.com/?p=55708991>"Chris Aldrich on the IndieWeb"</a> (<a class="u-bookmark-of" href="https://boffosocko.com/?p=55708991">boffosocko.com</a>)
    <time class="dt-published" datetime="2019-01-11 18:052:00" title="Friday, Jan 11, 2019, 6:52 PM"><a href="https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow" target="_blank" rel="noopener">Jan 11</a></time>
    
<div class="p-category">tag-name1</div>
 
<div class="p-category">tag-name2</div>
 
<div class="p-category">tag-name3</div>
</div>

Page note with an annotation

(aka a reply, but could alternately be marked up as above as a bookmark) Example from https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow


<div class="h-entry">
    <a class="p-author h-card" href="https://hypothes.is/users/judell">judell</a>
    Public on <https://hyp.is/_tLJyA-cEemE-qPndyfQow/boffosocko.com/?p=55708991>"Chris Aldrich on the IndieWeb"</a> (<a class="u-in-reply-to" href="https://boffosocko.com/?p=55708991">boffosocko.com</a>)
    <time class="dt-published" datetime="2019-01-11 18:052:00" title="Friday, Jan 11, 2019, 6:52 PM"><a href="https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow" target="_blank" rel="noopener">Jan 11</a></time>
    
<div class="e-content">
        
<p>This is web thinking in action.</p>
<p>https://blog.jonudell.net/2011/01/24/seven-ways-to-think-like-the-web/</p>
<p>Well done!</p>

    </div>

    
<div class="p-category">tag-name</div>
</div>

Highlights (aka quotes)

Example from https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow


<div class="h-entry">
    <a class="p-author h-card" href="https://hypothes.is/users/judell">judell</a>
    Public on <a href="https://hyp.is/gBZPQucmEeaPBQvYzSRo-Q/www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/">"As We May Think"</a> (<a class="u-quotation-of h-cite" href="https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/">www.theatlantic.com</a>)
    <time class="dt-published" datetime="2017-04-30 08:40:00" title="Sunday, Apr 30, 2017, 08:40 AM"><a href="https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow" target="_blank" rel="noopener">Apr 30, 2017</a></time>
    
<blockquote>First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together.</blockquote>

    
<div class="p-category">IAnnotate2017</div>
</div>

Annotations (replies)

Example from https://hypothes.is/a/9JrX5lf9RraeLKKn9WwmMQ


<div class="h-entry">
    <a class="p-author h-card" href="https://hypothes.is/users/jeremydean">jeremydean</a>
    Public on <a href="https://hyp.is/9JrX5lf9RraeLKKn9WwmMQ/www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/">"As We May Think"</a> (<a class="u-in-reply-to" href="https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/">www.theatlantic.com</a>)
    <time class="dt-published" datetime="2015-09-02 15:11:00" title="Wednesday, Sep 2, 2015, 03:11 PM"><a href="https://hypothes.is/a/_tLJyA-cEemE-qPndyfQow" target="_blank" rel="noopener">Sep 2, 2015</a></time>
    
<blockquote class="p-in-reply-to h-cite">This has not been a scientist's war; it has been a war in which all have had a part.
<blockquote>
    
   
<div class="e-content">
        
<p>It kind of blows me mind that the end of WWII is the context for these early dreams of the Internet. Is it the hope experienced in patriotic collaboration toward technological innovation? That's what Bush seems to acknowledge explicitly. It's a techno-militaristic union that haunts us to this day (#prism). But I wonder too if it's the precarious of knowledge, or perhaps the destructiveness of knowledge, that also inspires Bush…</p>

    </div>

    
<div class="p-category">tag-name</div>
</div>

I’ll also note that there’s the potential of a reply on Hypothes.is to a prior reply to a canonical URL source. In that case it could be either marked up as a reply to the “parent” on Hypothesis and/or a reply to the canonical source URL, or even both so that webmentions could be sent further upstream. (My experience in this is more limited, not having dealt with it personally in the past.) Once these pieces are implemented, they can be tested against a variety of microformats parsers to ensure they’re outputting the correct (and properly nested) information. I often find that pin13 is a pretty solid modern and up-to-date choice for this.

Additional resources with examples

I’ll also leave the caveat here, that while I’ve got a stronger grasp of Microformats than the average bear, that the above examples may have some subtle quirks that others may catch or which could be improved upon. I find that the Microformats web chat can be a good source for helps from some of the world’s best experts in the area. (Other methods for engaging in chat via IRC, Slack, etc. can be utilized as well.)

If Dan, Jon, or any of the gang has questions about any of this, I’m happy to chat via phone, video conference, or other to help get them going.

Thoughts on open notebooks, research, and social media

I remember thinking over a decade ago how valuable it would be if researchers kept open notebooks (aka digital commonplace books) like the one Kimberly Hirsh outlines in her article Dissertating in the Open: Keeping a Public Research Notebook. I’d give my right arm to have a dozen people in research areas I’m interested in doing this very thing!

The best I could hope for back in 2008, and part of why I created the @JohnsHopkins Twitter handle, was that researchers would discover Twitter and be doing the types of things that some of the Johns Hopkins professors outlined in this recent article The Promise and Peril of Academia Wading into Twitter are now finally doing. It seems sad that it has taken over a decade and this article is really only highlighting the bleeding edge of the broader academic scene now. While what they’re doing is a great start, I think they really aren’t going far enough. They aren’t doing their audiences as much service  as they could because there’s only so much that Twitter allows in terms of depth of ideas and expressiveness. It would be far better if they were doing this sort of work from their own websites and more directly interacting with their colleagues on the open web. The only value that Twitter is giving them is a veneer of reach to a broader audience, but they’re also opening themselves up to bigger attacks as is described in the article.

In addition to Kimberly’s example, another related area of potential innovation would be moving the journal clubs run by many research groups and labs online and opening them up. Want to open up science?  Then let’s really do it!  By bookmarking a variety of articles on their own websites, various members could be aggregated to contribute to a larger group, which could then use their own websites with protocols like Webmention or even simple tools like Hypothes.is to guide and participate in larger online conversations to move science communication along at an even faster pace. Greg McVerry and I have experimented in taking some of these tools into the classroom in the past.

If you think about it, arXiv and other preprint servers are really just journal clubs writ large. The problem is that they’re only communicating in one direction by aggregating the initial content, but they’re dramatically failing their audiences in that they aren’t facilitating or aggregating any open discussion around that content. As a result, the largest portion of their true value is still locked away in the individual brains of their readers rather than as commentary or even sentence level highlights and annotations on particular pieces out in the open. Often is the time that I’ll tweet about an interesting article only to receive a (lucky) reply that the results have been debunked, yet that information is almost never disclosed in or around the journal article (especially online) where it certainly belongs. Academic publishers are not only gouging us financially by siloing their content, they’re failing us far worse than most realize.

Another idea: Can’t get a journal of negative results to publish your latest research failure? Why not post a note or article on your own website to help out future researchers? (or even demonstrate to your students that not everything always works out?)

Naturally having aggregation services like indieweb.xyz, building planets, using OPML subscriptions, or the coming wave of feed readers could make a lot of these things easier, but we’re already right on the cusp for people who are willing to take a shot for doing this type of research online on their own websites and out in the open.

Want to try out some of the above? I’m happy to help (gratis) researchers who’d like to experiment in the area to get themselves set up. Just send me a note or give me a call.

🔖 CROWDLAAERS

Bookmarked CROWDLAAERS (crowdlaaers.org)
Explore any URL featuring Hypothesis annotation. CROWDLAAERS provides learning analytics about active participants, temporal activity (active days), collaborative discourse (threads), and also Hypothesis tags. Groups of individual annotations may be sorted by date, contributor, annotation, tags, and level (or the position of an annotation reply in a thread). Select any annotation to read the full content within CROWDLAAERS or in context of the source document. Or explore how CROWDLAAERS has been applied to curated sets of online texts by selecting from Projects.
RSVPed Attending AnnotatED at OLC Innovate 2019
Join us 1–4pm MT Tuesday, 2 April 2019 at AnnotatED, a free annotation summit brought to you by Hypothesis in conjunction with OLC Innovate 2019. RSVP NOW to reserve your spot at this free event. Can't make it to Colorado? RSVP as a virtual participant. Learn more about all the annotation activities happening at #OLCInnovate! Attendees will include leaders from the annotation community like Marginal Syllabus co-founder Remi Kalir, Francisco Perez from CROWDLAAERS, Director of Education Jeremy Dean and Nate Angell from Hypothesis, and folks from institutions now piloting annotation like CSU Channel Islands and MSU Denver. The summit will feature a mini-keynote from Manuel Espinoza, Associate professor of Educational Foundations at CU Denver. Connect with your peers at other institutions working with annotation. Learn about and share annotation use cases. Explore existing and new research on the impact of annotation in education. Find out how you, your colleagues, and your institution can get started or expand annotation in partnership with Hypothesis. Help shape future annotation summits. Refreshments provided.
Attending remotely
It’s not exactly an implementation of Webmention, but I was interested to find that there’s a tool from Hypothes.is that will show you (all?) the annotations (and replies) on your website.

Just visit https://jonudell.info/h/facet/ and then enter the appropriate domain name followed by /* as a wildcard to search.

Examples for:

Now wouldn’t it be cool if this were available in the main UI? Perhaps if there were a button for “Site notes” or highlights? This may be unwieldy for the New York Times, but could be reasonable and very useful for smaller personal and/or academic based websites.

The user interface for the side bar of Hypothesis with a "Site Notes" element added in red next to "Annotations" and "Page Notes"

Other than following the RSS feeds of specific people’s public highlights and annotations, is there an easier way of following people on Hypothes.is? Is there a social layer or reader side I’m missing?

Who should I be following? How can I discover interesting annotators besides besides slowly and organically? Who out there is using Hypothes.is in unique and interesting ways?

And of course, there’s also following feeds of interesting tags, but how can one find the largest and most interesting subsets? Many of the tags I’m interested in following are only being annotated and followed by me.

Is there a master list of public tags ranked in order of prevalence? Academic based tags? 

I feel like there’s far more interesting material being unearthed by this tool, just based on how I’m using it, but that the discovery portion is largely missing, or hidden away in the dark corners of Jon Udell’s web or only via API access.

I find myself wondering what’s at the bleeding edge that I’m not seeing (without following the GitHub repo on a regular basis).

 

📑 Exploring the UX of web-annotations | Tom Critchlow

Annotated Exploring the UX of web-annotations by Tom CritchlowTom Critchlow (tomcritchlow.com)
There’s also a robust ecosystem of tools to follow users, monitor site annotations etc.  
Wait? What!? I’ve been wanting to be able to follow users annotations and I’d love the ability to monitor site annotations!! (I’ve even suggested that they added Webmention before to do direct notifications for site annotations.)

Where have you seen these things hiding Tom?

📑 Exploring the UX of web-annotations | Tom Critchlow

Annotated Exploring the UX of web-annotations by Tom CritchlowTom Critchlow (tomcritchlow.com)
Especially on mobile.  
I’ve found in the past that highlighting on Chrome for Android was nearly impossible. I’ve switched to using Firefox when I need to use hypothes.is on mobile.

👓 Augmenting Human Intellect: A Conceptual Framework | framework.thoughtvectors.net

Read Augmenting Human Intellect: A Conceptual Framework (framework.thoughtvectors.net)

This February 2019, join us as we collaboratively read and collectively annotate three crucial parts of Doug Engelbart’s 1962 research report and manifesto, Augmenting Human Intellect: A Conceptual Framework.

Doug Engelbart’s 1962 manifesto  offers a unique, multidisciplinary perspective on how human ingenuity, in symbiosis with networked digital computing technologies, might enlarge human capability and help address humanity’s most urgent problems.

This looks like a very cool annotation project!

📑 Welcome to my online sandbox. | Joyce Garcia

Annotated Welcome to my online sandbox. by Joyce GarciaJoyce Garcia (Gratuitous Web Presence)
Then I learned about the IndieWeb movement and Micro.blog, and I fell in love with the Internet as I once hoped it would be: a place where people could congregate, converse, and learn from one another with somewhat minimal rancor — and without an overtly overarching need to make a buck with their “content.”  

📑 Collaborative resource curation | Hypothes.is

Replied to Collaborative resource curation by Jon Udell (Hypothesis)
Recently we decided to keep better track of tweets, blog posts, and other web resources that mention and discuss our product. There are two common ways to do that: send links to a list maintainer, or co-edit a shared list of links. Here’s a third way, less common but arguably more powerful and flexible: tag the web resources in situ.
It isn’t rocket science, but as Jon indicates, it’s *incredibly *powerful.

I use my personal website with several levels of taxonomy for tagging and categorizing a variety of things for later search and research.

Much like the example of the Public Radio International producer, I’ve created what I call a “faux-cast” because I tag everything I listen to online and save it to my website including the appropriate <audio> link to the.mp3 file so that anyone who wants to follow the feed of my listens can have a playlist of all the podcast and internet-related audio I’m listening to.

A visual version of my “listened to” tags can be found at https://boffosocko.com/kind/listen/ with the RSS feed at https://boffosocko.com/kind/listen/feed/

👓 Collaborative resource curation | Hypothesis

Read Collaborative resource curation by Jon Udell (Hypothesis)
Recently we decided to keep better track of tweets, blog posts, and other web resources that mention and discuss our product. There are two common ways to do that: send links to a list maintainer, or co-edit a shared list of links. Here’s a third way, less common but arguably more powerful and flexible: tag the web resources in situ.