👓 Open web annotation of audio and video | Jon Udell

Read Open web annotation of audio and video by Jon UdellJon Udell (Jon Udell)
Text, as the Hypothesis annotation client understands it, is HTML, or PDF transformed to HTML. In either case, it’s what you read in a browser, and what you select when you make an annotation. What’s the equivalent for audio and video? It’s complicated because although browsers enable us to select passages of text, the standard media players built into browsers don’t enable us to select segments of audio and video. It’s trivial to isolate a quote in a written document. Click to set your cursor to the beginning, then sweep to the end. Now annotation can happen. The browser fires a selection event; the annotation client springs into action; the user attaches stuff to the selection; the annotation server saves that stuff; the annotation client later recalls it and anchors it to the selection. But selection in audio and video isn’t like selection in text. Nor is it like selection in images, which we easily and naturally crop. Selection of audio and video happens in the temporal domain. If you’ve ever edited audio or video you’ll appreciate what that means. Setting a cursor and sweeping a selection isn’t enough. You can’t know that you got the right intro and outro by looking at the selection. You have to play the selection to make sure it captures what you intended. And since it probably isn’t exactly right, you’ll need to make adjustments that you’ll then want to check, ideally without replaying the whole clip.

Jon Udell has been playing around with media fragments to create some new functionality in Hypothes.is. The nice part is that he’s created an awesome little web service for quickly and easily editing media fragments online for audio and video (including YouTube videos) which he’s also open sourced on GitHub.

I suspect that media fragments experimenters like Aaron Parecki, Marty McGuire, Kevin Marks, and Tantek Çelik will appreciate what he’s doing and will want to play as well as possibly extend it. I’ve already added some of the outline to the IndieWeb wiki page for media fragments (and a link to fragmentions) which has some of their prior work.

I too look forward to a day where web browsers have some of this standardized and built in as core functionality.

Highlights, Quotes, & Marginalia

Open web annotation of audio and video

This selection tool has nothing intrinsically to do with annotation. It’s job is to make your job easier when you are constructing a link to an audio or video segment.

I’m reminded of a JavaScript tool written by Aaron Parecki that automatically adds a start fragment to the URL of his page when the audio on the page is paused. He’s documented it here: https://indieweb.org/media_fragment


(If I were Virginia Eubanks I might want to capture the pull quote myself, and display it on my book page for visitors who aren’t seeing it through the Hypothesis lens.)

Of course, how would she know that the annotation exists? Here’s another example of where adding webmentions to Hypothesis for notifications could be useful, particularly when they’re more widely supported. I’ve outlined some of the details here in the past: http://boffosocko.com/2016/04/07/webmentions-for-improving-annotation-and-preventing-bullying-on-the-web/

Syndicated copies to:

Reply to Annotating Web Audio by Jon Udell

Replied to Annotating Web Audio by Jon UdellJon Udell (Jon Udell)
On a recent walk I listened to Unmasking Misogyny on Radio Open Source. One of the guests, Danielle McGuire, told the story of Rosa Parks’ activism in a way I hadn’t heard before. I wanted to capture that segment of the show, save a link to it, and bookmark the link for future reference.

Jon, this is certainly an awesome and interesting way to target audio on the web, which can be tremendously useful.

Given what you’ve got here, I suspect that you may be unaware of the W3C spec for media fragments which may make portions of what you’re attempting to do a bit easier (and also much more standardized). The spec is relatively broadly supported by most browsers, so it immediately makes things a tad easier from a plumbing perspective.

Some people will be somewhat familiar with the targeting technique as it’s similar to the one used by YouTube which lets users hot link to specific portions of video on their platform.

To summarize the concept, on most audio and video files one can add a #t=XXX the the end of a URL where XXX is the number in seconds into the file where one wants to start. One can target stretches of audio similarly with the pattern #t=XXX,YYY where XXX is the start and YYY is the stop time for the fragment, again in seconds.

As an example I can use it to specifically target the audio  on a particular standalone audio file like so:

https://media.martymcgui.re/70/d5/f1/77/975194c74454dc7a3ef71586bf98612a94bcb5685f7e7d3ca60dc183.mp3#t=269

With some clever JavaScript, one can go a step further and implement this at the level of targeting audio/video as embedded on a particular page which may contain a wealth of additional (potentially necessary) context. As an example of this, we can look at the audio above in its original context as part of a podcast using the same type of time fragment notation:

https://martymcgui.re/2017/10/29/163907/#t=269

As an added bonus, on this particular page with audio, you’ll notice that you can play the audio and if you pause it, the page URL in your browser should automatically refresh to indicate the particular audio timestamp for that particular position! Thus in your particular early example it makes things far easier to bookmark, save, or even share!

For use within Hypothesis, I suspect that one could use this same type of system to directly annotate the original audio file on the original page by using this scheme, potentially by using such JavaScript within the browser plugin for Hypothesis.

It would be nice if the user could queue up the particular audio segment and press pause, and then annotate the audio portion of the page using such a targeting segment. Then one could potentially share a specific URL for their annotation (in typical Hypothesis fashion) that not only targets the original page with the embedded audio, but it could also have that audio queued up to the correct portion (potentially with a page refresh to reset the audio depending on the annotation.)

The nice part is that the audio can be annotated within the page on which it originally lived rather than on some alternate page on the web that requires removing the context and causing potential context collapse. It also means one doesn’t have to host an intermediate page to have the whole thing work.

For more information on the idea, take a peek at the IndieWeb’s page on audio fragments which includes a few examples of people using it in the wild as well as a link to the JavaScript sample for doing the targeting within the page itself.

I’m curious if the scheme may make putting all the smaller loose pieces together even easier, particularly for use within Hypothesis? and while keeping more of the original context in which the audio was found?

I also suspect that these types of standards could be used to annotate audio in much the same way that the SoundCloud service handles their audio annotations, though in a much more open way. One would simply need to add on some additional UI to make the annotations on such audio present differently.

Just for fun, this type of sub-targeting on web pages also works visually for text as well with the concept of fragmention. As an example of this, I can target this specific paragraph with this link http://boffosocko.com/2018/01/07/reply-to-annotating-web-audio-by-jon-udell/#Just+for+fun, and a snippet of JavaScript on the page creates a yellow highlighting effect as well.

Syndicated copies to:

This smartphone microscope lets you play games with microbes.@richcampbell @CarlFranklin –> Full Episode:

Syndicated copies to:

Introducing Shortcut | This American Life

Bookmarked Introducing Shortcut (This American Life)
Have you ever heard a moment on the show that you wish you could share with your friends? Well, now you can! Shortcut is a new app we created that allows you to turn your favorite podcast moments into videos that you can post to social media. It’s kind of like making a gif, but for audio. Here’s how to use it.

I love the functionality that they’ve built into their podcast to clip and share audio snippets. It’s only a step or two further to highlight and annotate within an audio file. This is somewhat reminiscent of bits of SoundCloud’s user interface.

h/t Jon Udell via Twitter

Syndicated copies to:

Hypothes.is and the IndieWeb

A new plugin helps to improve annotations on the internet

Last night I saw two great little articles about Hypothes.is, a web-based annotation engine, written by a proponent of the IndieWeb:

Hypothes.is as a public research notebook

Hypothes.is Aggregator ― a WordPress plugin

As a researcher, I fully appreciate the pro-commonplace book conceptualization of the first post, and the second takes things amazingly further with a plugin that allows one to easily display one’s hypothes.is annotations on one’s own WordPress-based site in a dead-simple fashion.

This functionality is a great first step, though honestly, in keeping with IndieWeb principles of owning one’s own data, I think it would be easier/better if Hypothes.is both accepted and sent webmentions. This would potentially allow me to physically own the data on my own site while still participating in the larger annotation community as well as give me notifications when someone either comments or augments on one of my annotations or even annotates one of my own pages (bits of which I’ve written about before.)

Either way, kudos to Kris Shaffer for moving the ball forward!

Examples

My Hypothes.is Notebook

The plugin mentioned in the second article allows me to keep a running online “notebook” of all of my Hypothes.is annotations on my own site.

My IndieWeb annotations

I can also easily embed my recent annotations about the IndieWeb below:

[ hypothesis user = 'chrisaldrich' tags = 'indieweb']

Syndicated copies to:

Boffo Socko Now Supports Hypothes.is Annotations

You can now highlight and annotate most of the pages here on Boffo Socko as well as other web pages.

I’d played around with many of them in the past, but a recent conversation with Matt Gross about News Genius and their issues in the last week reminded me about internet annotation platforms. Since some of what I write here is academic in nature, I thought I would add native Hypothes.is Annotation support to the site.

hypothesisIf you haven’t heard about it before, you might find the ability to highlight and annotate web pages very useful. Hypothesis allows for public or private highlights and notes and it can be a very useful extension of one’s commonplace book.

At the moment, I’m not sure where it all fits into the IndieWeb infrastructure I’m building here, but, at least for the moment, I’d hope that those making public annotations and notes will also enter their commentary into the comments either here on the blog or by way of syndicated versions on Facebook or Twitter so that they’re archived here for posterity. (Keep in mind site-deaths are prevalent and even Hypothes.is acknowledges in a video on their homepage that there have been many incarnations of web annotations that have come and gone in the life of the internet.) Perhaps one day there will be a federated and cross-linked version of highlights and annotations in the IndieWeb universe with webmentions included?!

Educators and researchers interested in using web annotation are encouraged to visit the wealth of information provided by providers like Hypothes.is and Genius.com.  In particular, the Hypothes.is blog has some great material and examples over the past year, and they have a special section for educators as well.

As it’s similar in functionality to highlighting on the web, I’ll remind users that we also still support Kevin Marks’s fragmentions as well.

If anyone is aware of people or groups working on the potential integration of the IndieWeb movement (webmentions) and web annotation/highlighting, please include them in the comments below–I’d really appreciate it.

 

Syndicated copies to: