Creating a commonplace book or zettelkasten index from Hypothes.is tags

I thought it might be useful to have a relatively complete list of cross linked topical headings in my digital notebook (currently Obsidian) which is a mélange of wiki, zettelkasten, journal, project management tool, notebook, and productivity tool. First, let’s be honest that mélange is too poetic based on what I see of how others use Obsidian and similar tools. My version is structured to have very clear delineations between these forms even though I’m using the same tool for various functions.

I find that indexed subject headings can be useful for creating links between my wiki-like pages as well as links between atomic ideas in my digital zettelkasten. Gradually as one’s zettelkasten becomes larger and one works with it more, it becomes easier to recall individual ideas and cross link them. Until this happens or for smaller zettelkasten it can be useful to cross reference subject headings from one zettel to see what those link to and use those as a way to potential create links to other zettels. This method can also be used as a search/discovery aide for connecting edge ideas in new areas to pre-existing portions of one’s zettelkasten as well. Of course at massive scale with decades of work, I suspect this index will have increased value as well.

I don’t hear people talking about these types of indices for their zettelkasten in any of the influencer spaces or on social media. Are people simply skipping this valuable tool? For those enamored of Niklas Luhmann, we should mention that having and maintaining a subject index was a powerful portion of his system, even if the digitized version of his zettelkasten hasn’t yet been fully digitized. I haven’t seen the whole collection myself, but based on the condition of some of the cards in his index, Luhmann heavily used his subject index. (Note to self: I wonder what his whole system would look like in Obsidian?) Having a general key word/subject heading/topic heading index of all the material in one’s system can be very useful for general search and discovery as well. This is one of the reasons that John Locke wrote about a system for indexing one’s commonplace book in 1685. His work here is likely the distal reason Luhmann had one in his system.

Systems that have graphical knowledge graphs may make this process easier as one can look from one zettel out one or two levels to see where those link to.

Since such a large swath of my note taking practice starts by using Hypothes.is as my tool of choice, I’m able to leverage several years of using it to my benefit. Within it I’ve got 9,314 annotations, highlights, and bookmarks tagged with over 3,326 subject headings as of this writing.

To get all my subject heading tags, I used Jon Udell’s excellent facet tool to go to the tag editing interface. There I entered a “max” number larger than my total number of annotations and left the “tag” field empty to have it return the entire list of my tags. I was then able to edit a few of them to concatenate duplicates, fix misspellings, and remove some spurious tags.

An alternate way of doing this is to use a method described in this GitHub issue which shows how to get the tags out of local storage in your web browser. Your mileage may vary though if you use Hypothes.is in multiple browsers, which I do.

I moved this list from the tag editor into a spreadsheet software to massage the list a bit, clean up any character encodings, and then spit out a list of [[wikilinked]] index keywords. I then cut and pasted it into my notebook and threw in some alphabetical headings so that I could more easily jump around the list.

Now I’ve got an excellent tool and interface for more easily searching and browsing the various areas of my multi-purpose digital notebook.

I’m sure there are other methods within various tools of doing this, including searching all files and cutting and pasting those into a page, though in my case this doesn’t capture non-existing files. One might also try a search for a regex phrase like: /(?:(?:(?:<([^ ]+)(?:.*)>)\[\[(?:<\/\1>))|(?:\[\[))(?:(?:(?:<([^ ]+)(?:.*)>)(.+?)(?:<\/\2>))|(.+?))(?:(?:(?:<([^ ]+)(?:.*)>)\]\](?:<\/\5>))|(?:\]\]))/ (found here) or something as simple as /\[\[.*\]\]/ though in my case they don’t quite return what I really want or need.

I’ll likely keep using more local search and discovery, but perhaps having a centralized store of subject headings will offer some more interesting affordances for search and browsing?

Have you created an index for your system? How did you do it?

Watched Synoptic Obsidian Book Club by Dan Allosso from YouTube
Announcing our next Obsidian Book Club, beginning next week, in which we will synoptically read two books: Too Much to Know and The Extended Mind. Everybody is welcome, whether or not you have been in a book club before. It's a really good group and I think these books will spark some very interesting conversations. If you're interested, drop me a line at the email in the video and I'll send you the details.
A new session of Dan Allosso’s synoptic Obsidian-based book club is starting April 2 for five weeks with some fascinating selections:

  • The Extended Mind: The Power of Thinking Outside the Brain by Annie Murphy Paul 
  • Too Much to Know: Managing Scholarly Information before the Modern Age by Ann M. Blair

The last two clubs were incredibly scintillating, so I can’t wait to see what this incarnation holds. Everyone interested in the topics and/or the process is welcome to join us. Details in the video.

In addition to the fun of the two particular texts, those interested in note taking, information management, personal knowledge management, zettelkasten and using tools like Obsidian and Hypothes.is in group settings will appreciate the experience. If you’re an educator interested in using these tools in a classroom-like setting for active reading and academic writing, I think there’s something to be learned in the process of what we’re all doing here.

Obsidian Book Club

Tentative Schedule beginning on Saturday, March 26, 2022 Saturday, April 2, 2022

Week 1
Paul: Introduction and Part 1
Blair: Chapter 1

Week 2
Paul: Part 2
Blair: Chapter 2

Week 3
Paul: Part 3
Blair: Chapter 3

Week 4
Paul: Conclusion
Blair: Chapter 4

Week 5
Paul: Any overflow from before??
Blair: Chapter 5

Replied to a tweet by Bill Seitz (Twitter)
@BillSeitz @bmann @flancian @gordonbrander I’ll eventually bring back the TW when I can sync it with my (currently private) Obsidian. I’m slowly working at making things compatible with/consumable by @an_agora.

Time lapse of The Dawn of Everything Book Club (End)

The final (?) time lapse of the contributions to the Obsidian vault for Dan Allosso’s The Dawn of Everything Book Club. I know I’ve still got hundreds of notes to process and add myself and I’m sure that there’s more that people may/might work on, but the club is officially over, so I thought I’d do one last video before the vault is “frozen”.

Zettelkasten Overreach

The zettelkasten is just that, it isn’t a calendar, a rolodex, a to do list or a hammer, saw, or even a jackhammer.

The basic zettelkasten note taking method is very simple and clear cut as originally described by Konrad Gessner in Pandectarum sive Partitionum Universalium (Zurich: Christoph Froschauer. Fol. 19-20, 1548) to Sönke Ahrens’s book How to Take Smart Notes: One Simple Technique to Boost Writing, Learning and Thinking – for Students, Academics and Nonfiction Book Writers. Just a handful of bullet points can outline the elegance and simplicity of the system. This dramatic simplicity leads to some tremendous value and complexity.

However, in modern use as seen online since roughly 2018 on, the idea and the digital tools surrounding it, has seen some severe mission creep. Zettlekasten has moved to the fad stage and we’re “zettlecasting” everything under the sun. While it can be used as a productivity tool specifically for writing, some are adapting and using it (and tools built for it) for productivity use writ-large. This includes project management or GTD (Getting Things Done) functions. Some are using it as a wiki, digital garden, or personal knowledge management system for aggregating ideas and cross linking them over time. Others are using it as a journal or diary with scheduling and calendaring functions tacked on. Still others are using it to collect facts and force the system to do spaced repetition. These additional functionalities can be great and even incredibly useful, but they’re going far beyond the purpose-fit functionality of what a zettelkasten system was originally designed to do.

Ahrens highlights the zettelkasten method as being simply and specifically designed to do its particular workflow well—no more, no less. He cleverly analogizes slip boxes to their larger box cousins, the shipping container, and the way that that they revolutionized the shipping industry.

In hindsight, we know why they failed: The ship owners tried to integrate the container into their usual way of working without changing the infrastructure and their routines. They tried to benefit from the obvious simplicity of loading containers onto ships without letting go of what they were used to.

Following this analogy, many people are currently trying to not only revolutionize shipping, but sourcing, manufacturing, distribution, and marketing as well. While this may be interesting and the digital tools might accommodate some of these functions, are they really custom built from start to finish to really excel at these functionalities? Can they really do all of them at once? While some may come close and do well enough, the added complexity and overreach of all these functionalities may be diluting the base power of what the zettelkasten is capable.

People conflate the idea of note taking and the zettelkasten with tools like Obsidian, Logseq, and Roam Research. This is not necessarily a good thing. If they expect it to do everything and it’s not capable of that or well designed to do what they expect, they’re more likely to get confused, frustrated, and eventually give up. I’ve seen it happening more and more.

As an example, in a book club related to Ahrens’ text in which many highly educated and talented people have been using these tools and have even previously read the book, many are still far too confused about what these tools are and the value that can come from them.

For those who are just coming to the idea of a zettelkasten, I recommend you limit yourselves to just that basic functionality. Don’t muddy the waters with other productivity functions, to do lists, journals, diaries, kitchen sinks, or the latest wiz-bang plugin. Don’t throw in buzz words like GTD and MOC. Stick to the simplest script for a few months and focus on finely honing a small handful of questions and ideas each day from your reading to see what happens. Write, link, repeat. Don’t get caught up in the collector’s fallacy by keeping and saving every single fleeting note (thought) you’ve got (or if you must, put them into a folder off to the side). Focus on the core idea.

Once you’ve got that part down and it’s working for you, then consider adding on those other functionalities. Experiment with them; see what works. But don’t be surprised if those other portions aren’t the magic bullet that is going to revolutionize your life. We’re likely to need new tools, functionality, and a system built from start to finish, to make those other things a useful reality.

Featured image: zettelkasten flickr photo by x28x28de shared under a Creative Commons (BY-SA) license

I’m excited to join Dan Allosso‘s book club on How to Take Smart Notes as a means of turning my active reading, annotating, and note taking into papers, articles and books using Obsidian.md and Hypothes.is

Details: 

cc: Ian O’Byrne, Remi Kalir

Replied to The Dawn of Everything – Part 1 by Miriam Ronzoni (Crooked Timber)
I recently finished reading* The Dawn of Everything by David Graeber and David Wengrow; I enjoyed it very much indeed. I thought I’d write a two parts review for CT, and here’s the first – I will p…
I’ve only begun reading the text for a book club being run by historian Dan Allosso who is also doing an experiment in a communally shared wiki/notebook platform Obsidian, but I’m quite curious about the Neolithic pieces relating to the inhabitants at Stonehenge. In particular, I’ve recently finished Lynne Kelly’s research in Knowledge and Power in Prehistoric Societies: Orality, Memory and the Transmission of Culture (Cambridge University Press, 2015) in which she touches on the primary orality of those peoples and the profound impact that settling into sedentary lifeways may have had on their culture. If she’s correct, then that settlement was dramatically “expensive” and more complex than we’ve been led to believe. This may have had confounding issues within their society as it grew and flourished. I would suspect that Graeber and Wengrow don’t touch on this portion of the complexity, but it may support their general thesis. I’ll try to report back as I get deeper into the topic.

Incidentally, if folks want to join this Obsidian book club on this text, it’s just starting and is comprised of a number of academics and researchers in a vein similar to CT. A quick web search should uncover the details to join.

Time Lapse of The Dawn of Everything Book Club (24 hours)

Just for fun, here’s roughly the first 24 hours of content creation for Dan Allosso‘s The Dawn of Everything book club‘s shared Obsidian vault. Can’t wait to see what it looks like at the end of our five weeks together.

Replied to a thread by Flancian and Silicon Jungle (Twitter)
@flancian @JungleSilicon Expert? 🙈I may be the only one posting to my website using pen and paper or to my vault via [[micropub]], but you’ll find many experts, ideas, and help at https://chat.indieweb.org/dev if you need it.
@wiobyrne, @AllossoDan has also been using it in his teaching. If you’re curious to see a use case applicable to the classroom, you might appreciate joining/watching an upcoming “book club” he’s doing w/ Obsidian over the holiday break: https://danallosso.substack.com/p/obsidian-book-club-the-dawn-of-everything.

Some notes about the semantic change of “interlink” and “backlink”

I’m reasonably certain that he’s raised the question or issue about the definition of “interlink” or “backlink” before, but it’s come up again today with some discussion and notes which I wanted to capture permanently here with few modifications for myself:

doubleloop[m] APP 12:30 PM
I have some notes I’ve taken on interlinking wikis here – https://commonplace.doubleloop.net/interlinking-wikis

tantek 12:39 PM
doubleloop[m], what’s the difference between “just” a link and an “interlink” from a user perspective?
genuine question (feel free to also answer if you have an idea @chrisaldrich) because Wikipedia seems to consider “interlink” as a common noun to be a synonym for “hyperlink” https://en.wikipedia.org/wiki/Interlink

Chris Aldrich 20:45 PM
I think that the definition for interlinking is expanding based on actual use cases. Historically Tim Berners Lee tried to create hyperlinks as bi-directional and then scrapped the idea as not easily implementable. As a result we’ve all come to expect that links are uni-directional.

In the digital gardens, wiki spaces and now, even with Webmention, there’s an expectation (I would suggest) by a growing number of people that some links in practice will be bi-directional.

If Neil puts a link to something within his own wiki/digital garden, he’s expecting that to be picked up in a space like the Agora and it will interlink his content with that of others.

Many who are practicing POSSE/PESOS are programatically (or manually) placing backlinks between their content and the copies that live on silos creating a round trip set of links that typically hasn’t been seen on the web historically.

Because we’ve mostly grown up with a grammar of single directional links and no expectation of visible reverse links (except perhaps in the spammy framing of SEO linkfarms), the word “interlink” has taken on the connotation seen in Wikipedia. I think that definition is starting to change.

Among a class of users in the note taking/personal knowledge management space (Roam Research, Obsidian, Logseq, TiddlyWiki, et al) most users are expecting tools to automatically interlink (in my definition with the sense of an expected bi-directional link) pages. Further, they’re expecting that if you change the word(s) that appear within a [[wikilink]] that it will globally change all instances of that word/phrase that are so linked within one’s system.

In many of those systems you can also do a manual /redirect the way we do on the IndieWeb wiki, but they expect the system to actively rename their bi-directional links without any additional manual work.

tantek 1:08 PM
ok, the bidirectionality as expectation is interesting

Chris Aldrich 1:08 PM
By analogy, many in the general public have a general sense of what /syndication is within social media, but you (Tantek) and others in the IndieWeb space have created words/phrases/acronyms that specify a “target” and “source” to indicate in which direction the syndication is being done and between sites of differing ownership (POSSE, PESOS, PASTA, PESETAS, POOSNOW,… not to mention a linear philosophical value proposition of which are more valuable to the end user). There is a group of people who are re-claiming a definition of the words “interlink” and perhaps “backlink” to a more logical position based on new capabilities in technology. Perhaps it may be better if they created neologisms for these, but linguistically that isn’t the path being taken as there are words that would seem to have an expandable meaning for what they want. I’d classify it as a semantic change/shift/drift in the words meanings: https://en.wikipedia.org/wiki/Semantic_change

I suspect that if Roam Research, or any of the other apps that have this bi-directionality built in, were to remove it as a feature, they’d loose all of their userbase.

tantek 1:11 PM
yes, such a semantic shift in the meaning of “interlink” seems reasonable, and a useful distinction from the now ubiquitously expected unidirectionality of “hyperlink”

Chris Aldrich 1:12 PM
I’m expecting that sometime within the next year or so that major corporate apps like Evernote and OneNote will make this bi-directional linking a default as well.

tantek 1:12 PM
in sci-fi metaphor terms, one-way vs two-way wormholes (per other uses of “hyper”)

Chris Aldrich 1:14 PM
I can only imagine what a dramatically different version of the web we’d be living in if the idea of Webmention had existed in the early 90s. Particularly as there’s the ability to notify the other end in changes/updates/deletions of a page. Would the word “linkrot” exist in that world?

Joe Crawford 1:22 PM
Or in a world with Xanaduian transclusions, for that matter.
Alas

Chris Aldrich 1:25 PM
Related to this and going into the world of the history of information is the suggestion by Markus Krajewski in “Paper Machines: About Cards & Catalogs, 1548-1929” that early card catalog and index card systems are really an early paper/manual form of a Turing Machine: https://mitpress.mit.edu/books/paper-machines.

One might imagine the extended analogy libraries:books:index cards :: Internet:websites:links with different modes and speeds of transmission.

I’ve downloaded my copy of The Dawn of Everything by David Graeber and David Wengrow for Dan Allosso’s forthcoming Obsidian-based book club. https://danallosso.substack.com/p/obsidian-book-club-the-dawn-of-everything

Curious to see how these tools can be communally used for collaborative note taking, knowledge creation, and discussion.

Replied to Twitter thread by CatoMinor & Flancian (Twitter)
Me too:

  • https://forum.obsidian.md/t/using-hypothes-is-to-quickly-place-notes-into-my-digital-notebook/5010
  • https://boffosocko.com/2021/07/08/hypothes-is-obsidian-hypothesidian-for-easier-note-taking-and-formatting/; or
  • https://electricarchaeology.ca/2021/02/14/from-hypothesis-annotation-to-obsidian-note/