Manual Backfeed in the Blogosphere

Forcing webmentions for conversations on websites that don’t support Webmention

Within the IndieWeb community there is a process called backfeed which is the process of syndicating interactions on your syndicated (POSSE) copies back (AKA reverse syndicating) to your original posts. As it’s commonly practiced, often with the ever helpful Brid.gy service, it is almost exclusively done with social media silos like Twitter, Instagram, Flickr, Github, and Mastodon. This is what allows replies to my content that I’ve syndicated to Twitter, for example, to come back and live here on my website.

Why not practice this with other personal websites? This may become increasingly important in an ever growing and revitalizing blogosphere as people increasingly eschew corporate social sites and their dark patterns of tracking, manipulative algorithmic feeds, and surveillance capitalism. It’s also useful for sites whose owners may not have the inclination, time, effort, energy or expertise to support the requisite technology.

I’ve done the following general reply pattern using what one might call manual backfeed quite a few times now (and I’m sure a few others likely have too), but I don’t think I’ve seen it documented anywhere as a common IndieWeb practice. As a point of fact, my method outlined below is really only half-manual because I’m cleverly leveraging incoming webmentions to reduce some of the work.

Manually syndicating my replies

Sometimes when using my own website to reply to another that doesn’t support the W3C’s Webmention spec, I’ll manually syndicate (a fancy way of saying cut-and-paste) my response to the website I’m responding to. In these cases I’ll either put the URL of my response into the body of my reply, or in sites like WordPress that ask for my website URL, I’ll use that field instead. Either way, my response appears on their site with my reply URL in it (sometimes I may have to wait for my comment to be moderated if the receiving site does that).

Here’s the important part: Because my URL appears on the receiving site (sometimes wrapped as a link on either my name or the date/time stamp depending on the site’s user interface choices), I can now use it to force future replies on that site back to my original via webmention! My site will look for a URL pointing back to it to verify an incoming webmention on my site.

Replies from a site that doesn’t support sending Webmentions

Once my comment appears on the receiving site, and anyone responds to it, I can take the URL (with fragment) for those responses, and manually input them into my original post’s URL reply box. This will allow me to manually force a webmention to my post that will show up at minimum as a vanilla mention on my website. 

The manual webmention box and button that appear on all my posts.

(Note, if your site doesn’t have a native box like this for forcing manual webmentions, you might try external tools like Aaron Parecki’s Telegraph or Kevin Mark’s Mention.Tech, which are almost as easy. For those who are more technical, cURL is an option as well.)

Depending on the microformats mark up of the external site, the mention may or may not have an appropriate portion for the response and/or an avatar/name. I can then massage those on my own site (one of the many benefits of ownership!) so that the appropriate data shows, and I can change the response type from a “mention” to a “reply” (or other sub-types as appropriate). Et voilà, with minimal effort, I’ve got a native looking reply back on my site from a site that does not support Webmention! This is one of the beautiful things of even the smallest building-blocks within the independent web or as a refrain some may wish to sing–“small pieces, loosely joined”!

This method works incredibly well with WordPress websites in particular. In almost all cases the comments on them will have permalink URLs (with fragments) to target the individual pieces, often they’ve got reasonable microformats for specifying the correct h-card details, and, best of all, they have functionality that will send me an email notification when others reply to my portion of the conversation, so I’m actually reminded to force the webmentions manually.

An Illustrative Example

As an example, I posted on my website that I’d read an article on Matt Maldre’s site along with a short comment. Since Matt (currently) doesn’t support either incoming or outgoing webmentions, I manually cut-and-pasted my reply to the comment section on his post. I did the same thing again later with an additional comment on my site to his (after all, why start a new separate conversation thread when I can send webmentions from my comments section and keep the context?).

Matt later approved my comments and posted his replies on his own website. Because his site is built on WordPresss, I got email notifications about his replies, and I was able to use the following URLs with the appropriate fragments of his comments in my manual webmention box:

https://www.spudart.org/blog/xeroxing-your-face/#comment-43843
https://www.spudart.org/blog/xeroxing-your-face/#comment-43844

After a quick “massage” to change them from “mentions” into “replies” and add his gravatar, they now live on my site where I expect them and in just the way I’d expect them to look if he had Webmention support on his website.

I’ll mention that, all of this could be done in a very manual cut-and-paste manner–even for two sites, neither of which have webmention support.  But having support for incoming webmentions on one’s site cuts back significantly on that manual pain.

For those who’d like to give it a spin, I’ll also mention that I’ve similarly used the incredibly old refbacks concept in the past as a means of notification from other websites (this can take a while) and then forced manual webmentions to get better data out of them than the refback method allows.

Liked a tweet by Vincent PickeringVincent Pickering (Twitter)
Interesting diagram of set up for an IndieWeb site.
Spent a few minute to finally set up my website with Brid.gy so that it’s now pulling responses back from Mastodon. It’s so nice to see all the interactions that were once “lost” to me coming back to live with their proper contexts on my website.

For those looking to tinker with their websites as it relates to interacting with Mastodon, the IndieWeb has a reasonable number of potential options in addition to your ability to roll your own.

Liked Backfeed without code by Ryan BarrettRyan Barrett (snarfed.org)
I’ve spent most of my time in the IndieWeb on backfeed: sending interactions from social networks back to your web site. Bridgy, the service we built, has served that need wel...
I’ve obviously read this before, but somehow I’ve now got some new respect for the basis of the broader underlying idea.
My last post to Facebook was almost a year ago on July 31, 2018, a day before Facebook turned off their API and prevented my website from interacting with their service. Other moral and ethical concerns with Facebook aside, I’ve got what I hope to be a useful method for people’s interactions with my Facebook account to come back to my site. This will let me better own and control my data while still interacting with people “stuck” on this problematic service.

This return post will serve as a test to see if I might return to and occasionally post there again.

Replied to a tweet by Rachel CherryRachel Cherry (Twitter)

I’m definitely curious what you come up with! There are so many syndication options and I’m always on the look out for better/more standardized methods. (Of course, nothing beats the feed directly from the source…)

Once you have posting out done, are you going to work on backfeed to have the responses to your posts on Twitter come back to aggregate the conversation on the original site? Perhaps using Webmention and Brid.gy?

👓 Backfeed, Doing It Without Code | Interdependent Thoughts

Read Backfeed, Doing It Without Code by Ton Zijlstra (zylstra.org)
Backfeed is an important element in breaking out of silos like Twitter, Facebook, and others. Backfeed means if I post something from my site to e.g. Twitter, that the responses to it (likes, answers) also become visible on my site. So that when those silos inevitably go away and get replaced, my in...

👓 facebook backfeed via email notifications · Issue #854 · snarfed/bridgy | GitHub

Read facebook backfeed via email notifications · Issue #854 · snarfed/bridgy (GitHub)
this is a kinda crazy, somewhat dangerous, generally inadvisable idea that we almost certainly shouldn't do...at least, not as a public facing service for everyone. @chrisaldrich had the ...

👓 Backfeed without code | Ryan Barrett

Read Backfeed without code by Ryan BarrettRyan Barrett (snarfed.org)
I’ve spent most of my time in the IndieWeb on backfeed: sending interactions from social networks back to your web site. Bridgy, the service we built, has served that need wel...
This sounds like a lovely and simple idea. Definitely worth attempting as a simple method.

Manually reconstructed Bridgy URLs redirect to silos

Filed an Issue snarfed/bridgy (GitHub)
Bridgy pulls comments and likes from social networks back to your web site. You can also use it to publish your posts to those networks.
It’s mentioned in the documentation that one can reconstruct URLs to allow manually resending webmentions for missed backfeed. However, it appears this may no longer work(?) as these reconstructed URLs, which used to be static are now automatically redirecting to their siloed instances.

Example: https://brid.gy/post/twitter/schnarfed/476408043819659264
redirects to https://twitter.com/schnarfed/status/476408043819659264

Separately, though related, the example in the documentation for Instagram no longer seems to exist and could be replaced and the example for Google+ could be removed as the service no longer exists.

Replied to facebook backfeed via email notifications (unlikely) · Issue #854 · snarfed/bridgy by Ryan Barrett (GitHub)

thanks @grantcodes!

i could come up with gmail, sieve, etc email filters for people to install that would forward all facebook emails except password resets. or maybe only notification emails. still seems iffy security wise, and awful UX, but...meh?

quick poll time. hey @aaronpk @chrisaldrich @jgmac1106 @grantcodes @dshanske @tantek and others: if you could get facebook backfeed again by setting up an email filter like this, but it'd be somewhat sketchy security wise (discussion above), and might be inconsistent and incomplete, and the bridgy UI might be weak or nonexistent...would you do it? feel free to emoji respond +1 for yes, -1 for no. also feel free to @-mention anyone else who might vote.

I should have outlined this originally… Likely safer (for Bridgy and the end user) would be to follow the model of posting to WordPress via email (or services like Reading.am which allow posting to it via custom email addresses). Bridgy could provide users with a private hashed email address like 123xyzABC@brid.gy which could be linked to their particular account to which they could manually (or automatically) forward the relevant Facebook notification emails. Upon receipt, Bridgy would know which account sent it and could also match it to the user’s post URL as a check before sending the appropriate webmentions.

This would leave Bridgy free from being the potential source for security leaks and put the onus on the end user. You’d naturally need to have the ability to reset/change the user’s hash in the case that they accidentally allowed their custom email address to leak, although generally this isn’t a huge issue as emails which don’t match the user’s account/endpoints would be dropped and not send webmentions in any case. (In some sense it’s roughly equivalent to my being able to visit https://brid.gy/twitter/schnarfed and clicking on the Poll now or Crawl now buttons. It’s doable, but doesn’t give a bad actor much. You’d probably want to rate limit incoming emails to prevent against mass spam or DDoS sort of attacks against Bridgy.)

A side benefit of all of this is that those who have kept their old email notifications could relatively easily get much of their past missing back feed as well. Or if they’re missing back feed for some reason, they could easily get it by re-sending the relevant emails instead of some of the current manual methods. Perhaps allowing preformatted emails with those same manual methods could be used to do back feed for Facebook or other providers as well?

We could also put together some forwarding filters for common platforms like gmail to help people set up autoforwarders with appropriate keywords/data to cut down on the amount of false positive or password containing emails being sent to Bridgy.

The one potential privacy issue to consider(?) is that this set up may mean that Bridgy could be sending webmentions for private messages since users get both private and public message notifications whereas the API distinguished these in the past. To remedy this, the comment URL could be tested to see if/how it renders as a test for public/private prior to sending. Separately, since Bridgy doesn’t need to store or show these messages (for long?), private messages could be sent, but potentially with a payload that allows the receiving end to mark them as private (or to be moderated to use WordPress terminology). This would allow the user’s website to receive the notifications and give them the decision to show or not show them, though this may be a potential moral gray area as they could choose to show responses that the originator meant to be private communication. The API would have prevented this in the past, but this email method could potentially route around that.

❤️ Bridgy stats update | snarfed.org

Liked Bridgy stats update by Ryan BarrettRyan Barrett (snarfed.org)
It’s that time of year again! No, not awards season…Bridgy stats time!
Looking at the graphs, the elephant in the room is clearly the Facebook shutdown. It was Bridgy’s second largest silo, numbering 1477 users when we wer...

👓 Some IndieWeb WordPress tuning | EdTech Factotum

Replied to Some IndieWeb WordPress tuning by Clint LalondeClint Lalonde (EdTech Factotum)
Been spending a bit of time in the past 2 days adding some new functionality to the blog. I am making more of an effort to write more, thanks in no small part to the 9x9x25 blog challenge I am doin…

Right now, I just want to write.  

You might find that the micropub plugin is a worthwhile piece for this. It will give your site an endpoint you can use to post to your site with a variety of third party applications including Quill or Micropublish.net.
October 14, 2018 at 01:01AM

My hope is that it will somehow bring comments on Facebook back to the blog and display them as comments here.  

Sadly, Aaron Davis is right that Facebook turned off their API access for this on August 1st, so there currently aren’t any services, including Brid.gy, anywhere that allow this. Even WordPress and JetPack got cut off from posting from WordPress to Facebook, much less the larger challenge of pulling responses back.
October 14, 2018 at 01:03AM

Grant Potter  

Seeing the commentary from Greg McVerry and Aaron Davis, it’s probably worthwhile to point you to the IndieWeb for Education wiki page which has some useful resources, pointers, and references. As you have time, feel free to add yourself to the list along with any brainstorming ideas you might have for using some of this technology within your work realm. Many hands make light work. Welcome to the new revolution!
October 14, 2018 at 01:08AM

the autoposts from Twitter to Facebook were  

a hanging thought? I feel like I do this on my site all too often…
October 14, 2018 at 01:09AM

I am giving this one a go as it seems to be the most widely used.  

It is widely used, and I had it for a while myself. I will note that the developer said he was going to deprecate it in favor of some work he’d been doing with another Mastodon/WordPress developer though.
October 14, 2018 at 01:19AM

👓 Turning off Facebook for Bridgy | snarfed.org

Read Turning off Facebook for Bridgy by Ryan BarrettRyan Barrett (snarfed.org)
I announced recently that Bridgy Publish for Facebook would shut down soon. Facebook’s moves to restrict its API to improve privacy and security are laudable, and arguably ...
This is so disappointing. Facebook is literally killing itself for me. So much for their “connecting everyone” philosophy.

Brid.gy was the last thing really keeping me connected to Facebook at all. Now that Facebook is shutting down its most useful functionality from my perspective, perhaps it’s time to deactivate it and move toward shutting it all down?

Enabling two way communication with WordPress and GitHub for Issues

This week, using the magic of open web standards, I was able to write an issue post on my own website, automatically syndicate a copy of it to GitHub, and later automatically receive a reply to the copy on GitHub back to my original post as a comment there. This gives my personal website a means of doing two way communication with GitHub.

This functionality is another in a long line of content types my website is able to support so that I’m able to own my own content, yet still be able to interact with people on other websites and social media services. Given the number of social sites I’ve seen disappear over the years (often taking my content with them), this functionality gives me a tremendously larger amount of control and ownership over my web presence and identity while still allowing me to easily communicate with others.

In this post I wanted to briefly sketch what I’ve done to enable this functionality, so others who are so inclined can follow along to do the same thing.

Setting up WordPress to syndicate to GitHub

I’ll presume as a first step that one has both a GitHub account and a self-hosted WordPress website, though the details will also broadly apply to just about any content management system out there that supports the web standards mentioned.

Register your GitHub account and your website with Bridgy

Ryan Barrett runs a fantastic free open sourced service called Bridgy. To use it you’ll need the microformat rel=​​​“me” links on both your GitHub account and your website’s homepage that point at each other.  GitHub will do most of the work on its side for you simply by adding the URL of your website to the URL field for your GitHub account at https://github.com/settings/profile. Next on your website’s homepage, you’ll want to add a corresponding rel=​​​​​“me” link from your website to your GitHub account.

In my case, I have a simple widget on my homepage with roughly the following link:
<a href="https://github.com/username">GitHub</a>
in which I’ve replaced ‘username’ with my own GitHub username. There are a variety of other ways to add a rel=​​​​​“me” link to your webpage, some of which are documented on the IndieWeb wiki.

Now you can go to Brid.gy and under “Connect your accounts” click on the GitHub button. This will prompt you to sign into GitHub via oAuth if you’re not already logged into the site. If you are already signed in, Brid.gy will check that the rel=​​​​​“me” links on both your site and your GitHub account reciprocally point at each other and allow you to begin using the service to pull replies to your posts on GitHub back to your website.

To allow Brid.gy to publish to GitHub on your behalf (via webmention, which we’ll set up shortly), click on the “Publish” button.

Install the Webmention Plugin

The underlying technology that allows the Bridgy service to both publish on one’s behalf as well as for the replies from GitHub to come back to one’s site is an open web standard known as Webmention. WordPress can quickly and easily support this standard with the simple Webmention plugin that can be downloaded and activated on one’s site without any additional configuration.

For replies coming back from GitHub to one’s site it’s also recommended that one also install and activate the Semantic Linkbacks Plugin which also doesn’t require any configuration. This plugin provides better integration and UI features in the comments section of one’s website.

Install Post Kinds Plugin

The Post Kinds Plugin is somewhat similar to WordPress’s Post Formats core functionality, it just goes the extra mile to support a broader array of post types with the appropriate meta data and semantic markup for interacting with Bridgy, other web parsers, and readers.

Download the plugin, activate it, and in the plugin’s settings page enable the “Issue” kind. For more details on using it, I’ve written about this plugin in relative detail in the past.

Install Bridgy Publish Plugin

One can just as easily install the Bridgy Publish Plugin for WordPress and activate it. This will add a meta box to one’s publishing dashboard that, after a quick configuration of which social media silos one wishes to support, will allow one to click a quick checkbox to automatically syndicate their posts.

Install the Syndication Links Plugin

The Syndication Links plugin is also a quick install and activate process. You can modify the settings to allow a variety of ways to display your syndication links (or not) on your website if you wish.

This plugin will provide the Bridgy Publish Plugin a place to indicate the permalink of where your syndicated content lives on GitHub. The Bridgy service will use this permalink to match up the original content on your website and the copy on GitHub so that when there are replies, it will know which post to send those replies to as comments which will then live on your own website.

Post away

You should now be ready to write your first issue on your website, cross post it to GitHub (a process known in IndieWeb parlance as POSSE), and receive any replies to your GitHub issue as comments back to your own website.

Create a new post.

In the “Kinds” meta box, choose the “Issue” option.

Screen capture of the Kinds meta box with "Issue" option chosen.
Kinds meta box with “Issue” option chosen.

Type in a title for the issue in the “Title” field.

In the “Response Properties” meta box, put the permalink URL of the Github repopository for which you’re creating an issue. The plugin should automatically process the URL and import the repository name and details.

The “Response Properties” meta box.

In the primary editor, type up any details for the issue as you would on GitHub in their comment box. You can include a relatively wide variety of custom symbols and raw html including

and  with code samples which will cross-post and render properly.

In the GitHub meta box, select the GitHub option. You can optionally select other boxes if you’re also syndicating your content to other services as well. See the documentation for Bridgy and the plugin for how to do this.

Screen capture of the Bridgy Publish meta box with GitHub chosen
Bridgy Publish meta box with GitHub chosen.

Optionally set any additional metadata for your post (tags, categories, etc.) as necessary.

Publish your post.

On publication, your issue should be automatically filed to the issue queue of the appropriate GitHub repo and include a link back to your original (if selected). Your post should receive the syndicated permalink of the issue on GitHub and be displayed (depending on your settings) at the bottom of your post.

Syndication Links Plugin will display the location of your syndicated copies at the bottom of your post.

When Bridgy detects future interactions with the copy of your post on GitHub, it will copy them and send them to your original post as a webmention so that they can be displayed as comments there.

An example of a comment sent via webmention from GitHub via Brid.gy. It includes a permalink to the comment as well as a link to the GitHub user’s profile and their avatar.

If you frequently create issues on GitHub like this you might want a slightly faster way of posting. Toward that end, I’ve previously sketched out how to create browser bookmarklets that will allow you one click post creation from a particular GitHub repo to speed things along. Be sure to change the base URL of your website and include the correct bookmarklet type of “issue” in the code.

The Post Kinds plugin will also conveniently provide you with an archive of all your past Issue posts at the URL http://example.com/kind/issue/, where you can replace example.com with your own website. Adding feed/ to the end of that URL provides an RSS feed link as well. Post Kinds will also let you choose the “Reply” option instead of “Issue” to create and own your own replies to GitHub issues while still syndicating them in a similar manner and receive replies back.

Other options

Given the general set up of the variety of IndieWeb-based tools, there are a multitude of other ways one can also accomplish this workflow (both on WordPress as well as with an infinity of other CMSes). The outline I’ve provided here is one of the quickest methods for beginners that will allow a relatively high level of automation and almost no manual work.

One doesn’t necessarily need to use the Post Kinds Plugin, but could manually insert all the requisite HTML into their post editor to accomplish the post side of things via webmention. (One also has the option to manually syndicate the content to GitHub by cutting and pasting it as well.) If doing things manually this way is desired, then one will need to also manually provide a link to the syndicated post on GitHub into their original so that Bridgy can match up the copy and the original to send the replies via webmention.

More details on how to use Bridgy with Github manually in conjunction with WordPress or other CMSes can be found here: https://brid.gy/about#github-issue-comment

Further steps

If you’ve followed many of these broad steps, you’ve given already given yourself an incredibly strong IndieWeb-based WordPress installation. With a minimal amount of small modifications you can also use it to dovetail your website with other social services like Twitter, Facebook, Flickr, Instagram, Google+ and many others. Why not take a quick look around on the IndieWeb wiki to see what other magic you can perform with your website!

I’ve documented many of my experiments, including this one, in a collection of posts for reference.

Help

If you have questions or problems, feel free to comment below or via webmention using your own website. You can also find a broad array of help with these plugins, services, and many other pieces of IndieWeb technology in their online chat rooms.​​​​​​​​