A note taking problem and a proposed solution

tl;dr

It’s too painful to quickly get frequent notes into note taking and related platforms. Hypothes.is has an open API and a great UI that can be leveraged to simplify note taking processes.

Note taking tools

I’ve been keeping notes in systems like OneNote and Evernote for ages, but for my memory-related research and work in combination with my commonplace book for the last year, I’ve been alternately using TiddlyWiki (with TiddlyBlink) and WordPress (it’s way more than a blog.)

I’ve also dabbled significantly enough with related systems like Roam Research, Obsidian, Org mode/Org Roam, MediaWiki, DocuWiki, and many others to know what I’m looking for.

Many of these, particularly those that can be used alternately as commonplace books and zettelkasten appeal to me greatly when they include the idea of backlinks. (I’ve been using Webmention to leverage that functionality in WordPress settings, and MediaWiki gives it grudgingly with the “what links to this page” basic functionality that can be leveraged into better transclusion if necessary.)

The major problem with most note taking tools

The final remaining problem I’ve found with almost all of these platforms is being able to quickly and easily get data into them so that I can work with or manipulate it. For me the worst part of note taking is the actual taking of notes. Once I’ve got them, I can do some generally useful things with them—it’s literally the physical method of getting data from a web page, book, or other platform into the actual digital notebook that is the most painful, mindless, and useless thing for me.

Evernote and OneNote

Older note taking services like Evernote and OneNote come with browser bookmarklets or mobile share functionality that make taking notes and extracting data from web sources simple and straightforward. Then once the data is in your notebook you can actually do some work with it. Sadly neither of these services has the backlinking functionality that I find has become de rigueur for my note taking or knowledge wrangling needs.

WordPress

My WordPress solutions are pretty well set since that workflow is entirely web-based and because WordPress has both bookmarklet and Micropub support. There I’m primarily using a variety of feeds and services to format data into a usable form that I can use to ping my Micropub endpoint. The Micropub plugin handles the post and most of the meta data I care about.

It would be great if other web services had support for Micropub this way too, as I could see some massive benefits to MediaWiki, Roam Research, and TiddlyWiki if they had this sort of support. The idea of Micropub has such great potential for great user interfaces. I could also see many of these services modifying projects like Omnibear to extend themselves to create highlighting (quoting) and annotating functionality with a browser extension.

With this said, I’m finding that the user interface piece that I’m missing for almost all of these note taking tools is raw data collection.

I’m not the sort of person whose learning style (or memory) is benefited by writing or typing out notes into my notebooks. I’d far rather just have it magically happen. Even copying and pasting data from a web browser into my digital notebook is a painful and annoying process, especially when you’re reading and collecting/curating as many notes as I tend to. I’d rather be able to highlight, type some thoughts and have it appear in my notebook. This would prevent the flow of my reading, thinking, and short annotations from being subverted by the note collection process.

Different modalities for content consumption and note taking 

Based on my general experience there are only a handful of different spaces where I’m typically making notes.

Reading online

A large portion of my reading these days is done in online settings. From newspapers, magazines, journal articles and more, I’m usually reading them online and taking notes from them there.

.pdf texts

Some texts I want to read (often books and journal articles) only live in .pdf form. While reading them in an app-specific setting has previously been my preference, I’ve taken to reading them from within browsers. I’ll explain why in just a moment, but it has to do with a tool that treats this method the same as the general online modality. I’ll note that most of the .pdf  specific apps have dreadful data export—if any.

Reading e-books (Kindle, e-readers, etc.)

If it’s not online or in .pdf format, I’m usually reading books within a Kindle or other e-reading device. These are usually fairly easy to add highlights, annotations, and notes to. While there are some paid apps that can extract these notes, I don’t find it too difficult to find the raw file and cut and paste the data into my notebook of choice. Once there, going through my notes, reformatting them (if necessary), tagging them and expanding on them is not only relatively straightforward, but it also serves as a simple method for doing a first pass of spaced repetition and review for better long term recall.

Lectures

Naturally taking notes from live lectures, audiobooks, and other spoken events occurs, but more often in these cases, I’m typically able to type them directly into my notebook of preference or I’m using something like my digital Livescribe pen for notes which get converted by OCR and are easy enough to convert in bulk into a digital notebook. I won’t belabor this part further, though if others have quick methods, I’d love to hear them.

Physical books

While I love a physical book 10x more than the next 100 people, I’ve been trying to stay away from them because I find that though they’re easy to highlight, underline, and annotate the margins, it takes too much time and effort (generally useless for memory purposes for me) to transfer these notes into a digital notebook setting. And after all, it’s the time saving piece I’m after here, so my preference is to read in some digital format if at all possible.

A potential solution for most of these modalities

For several years now, I’ve been enamored of the online Hypothes.is annotation tool. It’s open source, allows me reasonable access to my data from the (free) hosted version, and has a simple, beautiful, and fast process for bookmarking, highlighting, and annotating online texts on desktop and mobile. It works exceptionally well for both web pages and when reading .pdf texts within a browser window.

I’ve used it daily to make several thousand annotations on 800+ online web pages and documents. I’m not sure how I managed without it before. It’s the note taking tool I wished I’d always had. It’s a fun and welcome part of my daily life. It does exactly what I want it to and generally stays out of the way otherwise. I love it and recommend it unreservedly. It’s helped me to think more deeply and interact more directly with countless texts.

When reading on desktop or mobile platforms, it’s very simple to tap a browser extension and have all their functionality immediately available. I can quickly highlight a section of a text and their UI pops open to allow me to annotate, tag it, and publish. I feel like it’s even faster than posting something to Twitter. It is fantastically elegant.

The one problem I have with it is that while it’s great for collecting and aggregating my note data into my Hypothes.is account, there’s not much I can do with it once it’s there. It’s missing the notebook functionality some of these other services provide. I wish I could plug all my annotation and highlight content into spaced repetition systems or move it around and modify it within a notebook where it might be more interactive and cross linked for the long term. Sadly I don’t think that any of this sort of functionality is on Hypothes.is’ roadmap any time soon.

There is some great news however! Hypothes.is is open source and has a reasonable API. This portends some exciting things! This means that any of these wiki, zettelkasten, note taking, or spaced repetition services could leverage the UI for collecting data and pipe it into their interfaces for direct use.

As an example, what if I could quickly tell Obsidian to import all my pre-existing and future Hypothes.is data directly into my Obsidian vault for manipulating as notes? (And wouldn’t you know, the small atomic notes I get by highlighting and annotating are just the sort that one would like in a zettelkasten!) What if I could pick and choose specific course-related data from my reading and note taking in Hypothes.is (perhaps by tag or group) for import into Anki to quickly create some flash cards for spaced repetition review? For me, this combination would be my dream application!

These small pieces, loosely joined can provide some awesome opportunities for knowledge workers, students, researchers, and others. The education focused direction that Hypothes.is, many of these note taking platforms, and spaced repetition systems are all facing positions them to make a super-product that we all want and need.

An experiment

So today, as a somewhat limited experiment, I played around with my Hypothes.is atom feed (https://hypothes.is/stream.atom?user=chrisaldrich, because you know you want to subscribe to this) and piped it into IFTTT. Each post creates a new document in a OneDrive file which I can convert to a markdown .md file that can be picked up by my Obsidian client. While I can’t easily get the tags the way I’d like (because they’re not included in the feed) and the formatting is incredibly close, but not quite there, the result is actually quite nice.

Since I can “drop” all my new notes into a particular folder, I can easily process them all at a later date/time if necessary. In fact, I find that the fact that I might want to revisit all my notes to do quick tweaks or adding links or additional thoughts provides the added benefit of a first round of spaced repetition for the notes I took.

Some notes may end up being deleted or reshuffled, but one thing is clear: I’ve never been able to so simply highlight, annotate, and take notes on documents online and get them into my notebook so quickly. And when I want to do something with them, there they are, already sitting in my notebook for manipulation, cross-linking, spaced repetition, and review.

So if the developers of any of these platforms are paying attention, I (and I’m sure others) really can’t wait for plugin integrations using the full power of the Hypothes.is API that allow us to all leverage Hypothes.is’ user interface to make our workflows seamlessly simple.

A short post mortem, video and note links, and challenge from The Garden and the Stream IndieWebCamp Pop-up session

Thank you everyone!

For those who attended yesterday’s The Garden and the Stream IndieWebCamp session, thank you for participating! I honestly only expected 4 or 5 wiki fans to show up, so I was overwhelmed with the crowd that magically appeared from across multiple countries and timezones.

I’ve heard from many–both during the session and privately after–that it was a fantastic and wide-ranging conversation. (I never suspected memory palaces or my favorite 13th century Franciscan tertiary to be topics of discussion.) Several have suggested we host not only a continuation of the session, but that they’d be interested in other pop-up IndieWebCamp sessions. If you’re interested in future follow ups or sessions shoot me a quick email (you can find it on my home page) and I’ll be sure you get an invite. You can also follow future events via events.indieweb.org or find them in the IndieWeb’s weekly newsletter that is emailed out every Friday afternoon.

If you’re interested in hosting or suggesting other topics for future sessions, there’s a stub page on the IndieWeb wiki for doing so.

Our session went on far longer than I ever could have anticipated and I suspect we could have easily gone all day and still not touched on a fraction of all the topics we all outlined. Special thanks to the larger majority of those who were interested enough and had the free time to stay well past the hour mark and on to the end. I will say it’s nice to be able to cover so much ground and so many ideas without the threat of 5 more sessions following you.

Video and Notes

For those who missed it and are interested or those who have inquired, the video link and the notes from the session have been posted to the IndieWeb wiki. 

If you write up any notes or posts about the session, do add a link to them in the IndieWebCamp Pop-Ups page under blog posts/articles or photos. If you can’t log into the wiki (with your own website), feel free to ping me with the URL and I’ll add them for you.

I’ll  try to write up an organizer’s post-mortem with a few ideas about doing future sessions for others to consider. I hope to rewatch the session myself and add to the growing list of notes and thoughts about it.

Creators Challenge

Because this was just a single IndieWebCamp-style discussion session and we hadn’t specifically planned a traditional creator’s day or hack day, I did want to throw out a small challenge to those who either attended or who are interested in participating. 

For most, the IndieWeb is more about creating something than just talking about it. So in that spirit, I’ll challenge everyone to spend a few hours today/tomorrow or sometime this week and create something on your website or wiki related to the session. It can be a summary of ideas, a blog post about wikis (or anything you like really), a small change you’ve always wanted on your site (a CSS improvement, adding bi-directional links to your wiki, Webmention support, etc.), or anything else you might have found interesting from the conversation. The best part is that you can choose what you create on your own site! Make something you’ll use or appreciate. Have fun!

My personal plan for the challenge is to continue some work to my TiddlyWiki to support bi-directional links using TiddlyBlink. I might also take a crack at doing some design and building work to show some incoming webmentions on my TiddlyWiki. (If anyone is interested in test-driving Mike Caulfield’s implementation of Wikity on WordPress in conjunction with Webmention, I could be game for that too!)

Once you’ve made your creation, post a link to your article or notes or make a quick 2-3 minute demo video of the new feature or write up a post about it and add them to the IndieWeb wiki page for Pop-up Session Demos. Again if you can’t log into the wiki with your own website yet, drop me a note and I’ll add them for you or you can ask for help on how to do it in the IndieWeb chat.

Thanks again everyone! I look forward to seeing what you come up with.


cc: Kailyn Nelson (t), Phil Jones (t), Brian Sholis, Jack Baty

Gardens and Streams: Wikis, Blogs, and UI—a pop up IndieWebCamp session

There has been some sporadic conversation about doing impromptu IndieWebCamp sessions and thus far we’ve yet to organize one. Given our physical distancing and the dearth of bigger IndieWebCamps, I thought I would propose this single topic stand alone camp session to get something rolling. I’d invite others to propose and schedule others in the future.

April 25, 2020
Sat 10:00 – 11:00am (America/Los_Angeles)
Meeting ID: 950-1243-4695
Meeting Password: 021089
This is an online only event. We will provide a Zoom video conference link 30 minutes before the session here and in the IndieWeb chat.

Session Topic

We’ll be discussing and brainstorming ideas related to wikis and the IndieWeb, user interfaces, functionalities, examples of wikis and how they differ from blogs and other social media interfaces, and everyones’ ideas surrounding these. Bring your ideas and let’s discuss.

This is just a single one hour IndieWebCamp-like session (though we have the option to go over a bit since there isn’t a session following us) where we’ll brainstorm and discuss a particular topic. Hopefully the weekend time will be convenient for a wide range of people in Europe and North America who have previously shown interest in the topic. Everyone is welcome to attend.

Resources

To prepare for the session we’ll be using the following:

See also: https://indieweb.org/IndieWebCamps/Attending#Technology
This event is covered by the IndieWeb Code of Conduct. By participating, you’re acknowledging your acceptance of this code.

Questions? Concerns?

Feel free to ask in the IndieWeb chat: https://chat.indieweb.org/indieweb/

RSVP (optional)

If your website supports it, post an indie RSVP. Or, log in to indieweb.org and click “I’m Going”. (And if none of that means anything to you, don’t worry about it; just show up!)

I ran across Roam Research in late 2019, and in exploring TiddlyWiki lately I’m seeing a lot about people adding Roam-based functionality to TW5. What I’m not seeing is any conversation about how the same sort of backlinking could be done in MediaWiki. 

Of course I did mention something related the other day, but the functionality could be better surfaced:

One of my favorite “hidden” features of the IndieWeb wiki is the sidebar link for “What links here” that often provides some deeper and richer information than is found in the See Also sections.

I’ve been working on making some significant back-end tweaks to my wiki today. Among other things I’ve fixed, it should now be set up to accept incoming webmentions. Hooray! I’ve always wanted an IndieWeb wiki to go with my IndieWeb website; it looks like I’m well on my way.

Special thanks to Aaron Parecki for leaving some breadcrumbs to follow to make it possible.

Small progress in my wiki explorations and a fix to my MediaWiki administrative user email address

I’d looking into maintaining a wiki a while back and have recently been determined to get back to it. As a result, I’ve been looking at TiddlyWiki since that’s what some of Kicks Condor‘s group has been using. (Yep, I’ve still got that tab opened and am tinkering away slowly on the ideas–but mostly the technology.)

I’ve been having some issues in self-hosting a TiddlyWiki the way I’d like to. If anyone has any clear cut documentation on how to host a TiddlyWiki on one’s own domain name, I’d appreciate it. The documentation doesn’t seem as clear as I would expect (or perhaps more likely my server is having issues propagating/connecting?). If anything it’s muddled by the fact that they can seemingly be hosted in dozens of places one might not otherwise expect. My primary reservation is that it looks to me like they’re designed as single user instances, so I’m not exactly sure how Kicks et al. are effectuating their hyperconversations. Part of my issue is my mental model of some of the wikis involved in addition to the busy-ness of the sites’ themes, not to mention some of the non-standard conversational style on some. (I’ll get there eventually.)

I’ve also been using the IndieWeb’s MediaWiki for several years, so I’ve become much better at how it works as well as the ins-and-outs of the markup and how to do some slightly more advanced things using it. I’d set one up nearly a year ago this month and used it sporadically at best.

One of the bigger problems with my MediaWiki install was that somehow I wasn’t able to log into the primary account to do some of the necessary administrative functions. Today I got fed up with being hampered a bit and went spelunking into my install to see where things went wrong, suspecting that it was a one button install issue.

After digging through some documentation, I dug into the mySQL database and found a daunting looking [Blob] in the user_email field. Why couldn’t it be an easy-to-edit field? I not knowing anything better to do, I downloaded it, opened it up in my text editor, and discovered that I’d managed to leave a letter out of my own name in the email address! No wonder it wouldn’t work and the system wouldn’t let me reset my email address or password. A quick text edit later, the email was fixed, I uploaded the (now less intimidating) [Blob], and did a reset of the password in the admin interface, and we’re back in business! I’m always glad not to have borked the entire database and site.

If nothing else, it’ll help me in my explorations. Onward.

Replied to Starting an IndieWeb Homebrew Website Club by Jeremy Felt (jeremyfelt.com)
Starting things is fun. Narrating things as you go is… funner. Just about a month ago I joined the IndieWeb chat via Slack, which is connected to IRC and a web chat as well. I haven’t actually participated, but I’ve been getting the feel of conversations and checking out a bunch of the materia...
Congrats this is awesome!
Even doing the cutting/pasting from the wiki page to set up an event can sometimes be harrowing, so kudos for sticking with that part.

The part I got hung up on the most here was actually adding my name in the RSVP. The code seemed to suggest that adding would work, but it kept showing me “Template:Jeremyfelt.com” instead. I then poked around and saw that others had redirects setup, so I created a page titled “jeremyfelt” and added a wiki redirect to my user page and changed the code to , but it then said “Template:jeremyfelt” and I knew I was going nowhere. Finally, I updated it with standard URL syntax: [[jeremyfelt|Jeremy Felt]] and my name appeared as expected. No cool picture next to it or anything, but I’ll figure that at some point. This is all wiki stuff I probably used to know but have completely forgotten.

Some of this is relatively arcane and custom templated MediaWiki business. Here’s a link that explains most of it: https://indieweb.org/wikifying#How_to_Join_the_IndieWeb_Wiki

Feel free to hop into the helpful chat and most are ready and happy to try to help you out when you get stuck or provide pointers.
— Annotated December 19, 2019 at 01:31PM

Watched "American Idol" 202 (Auditions) from ABC
With Alejandro Aranda, Jake Durkin, Caleb Lee Hutchinson, Eddie Island. Aspiring singers from Coeur d'Alene, Idaho, Louisville, Ky., and Los Angeles see if they have what it takes to earn a coveted golden ticket to Hollywood.
Watching passively while I’m setting up a few new websites, including a wiki and a Wallabag instance.

👓 Highly integrate WordPress with Mediawiki | hawkinqian.com

Read Highly integrate WordPress with Mediawiki by Hawkin QianHawkin Qian (hawkinqian.com)
WordPress is the most popular blog software which can provide us opportunities to create beautiful and powerful websites. Mediawiki is also a very famous wiki software which can be applied to build our own knowledge pool like Wikipedia. Although some people may use WordPress to set up a small wiki system using plugins while others may use Mediawiki to blog their personal experiences, it cannot be more professional to use each of them to do things they are good at. Then the question comes as, is it possible to integrate them at the same time to make your website more functional? The answer is absolutely YES! Just take hawkinqian.com as an example. I use WordPress and Mediawiki to serve as my personal blog system and wiki system respectively, and they both function pretty well and also integrates pretty well visually. That’s what I would like to talk about, the way to highly integrate WordPress and Mediawiki.

👓 WordPress MediaWiki integration | StackOverflow

Read WordPress MediaWiki integration (Stack Overflow)
On the other end of the spectrum, I would be happy if I could install a wiki and share the login credentials between WordPress and the wiki. I hacked MediaWiki a while ago to share logins with anot...