@kfitz I’m not sure that the straightforward functionality you’re looking for exists within the ActivityPub plugin (yet), but it’s certainly something you could potentially file as a feature request.
Since you have other Fediverse accounts you’re using, you might be able to follow the same general pattern I’d documented with Twitter for threading comments between my site and Twitter: https://boffosocko.com/2018/07/02/threaded-conversations-between-wordpress-and-twitter/
Generally, you’d post on your site where it’s seen in the Fediverse via the ActivityPub plugin and/or optionally boosted by your native Mastodon account. Replies to your post (on Mastodon) show up on your site as comments and you reply to them there in your site’s comments section. Then you manually copy/paste the text of your reply from your website into your native Mastodon account and include the comment/reply permalink in that reply. If you’ve got Webmention set up with Brid.gy for Mastodon, replies to your replies on Mastodon should then make their way back to the proper threaded spot in your website’s comments section.
An example of this at work can be seen on my earlier mistake:

The original post on my site;
My own reply on my site;
My manually copied reply on Mastodon;
My Mastodon reply shows up on my site via Brid.gy;
A like of that Mastodon reply shows up on my site (also courtesy of Brid.gy via Webmention).

Related, I’ve been playing around with mirroring my WP site as an instance with the ActivityPub plugin and have boosted posts with my more broadly followed mastodon.social account the same way you mentioned that you were doing with yours. Somehow I’m anecdotally finding that I get more responses/reactions with native posts that with these boosts. I’m curious what your experience has been with this strategy so far? I’m still just starting my experimentation here, but I do like the fact that I’m able to include richer presentation of wrapped links in my WordPress native posts which are seen in the Fediverse while Mastodon seems to strip them out or not allow them (see an example of this in the post above this reply).