Crawling and indexing lemmy inter-instance would be an incredible boon to discoverability on the platform.

Maybe we need a discover tab. Something different from ‘All’. It would request top posts, or highly active posts, or whatever, from as many instances as it can find by crawling instance-to-instance trying to explore the entire network periodically. Does federation take place after the first post request from a new instance, or from the first subscribe request?

Follow up question: After the first contact, will all new posts, comments, votes always be federated between the two instances?

Thanks for the insight.

This is a major hurdle for discovery. My workflow to find new communities is basically to search for them on several different instances, visit their instance domain directly, search for their url through my home instance, and even then it’s occasionally useless because the only posts available to view in a feed from my home instance have no votes, no comments, and are pretty random. All the interesting posts that have had a moment to be voted on and commented on are a couple days old, and thus the only way to find things to interact with is to use the instance through it’s native domain. I suppose I could manually search for each URL of posts and comments as I browse in the native domain back into the search engine of my home domain, but this is insanity. Is this really the way we want to do things?

What if there was a way for one instance to request not the entire backlog of posts at once in these situations, but a series of posts long enough to fill one page of a feed at a time that match some search criterion when they attempt to directly explore a new community/instance, even if they are posts created pre-federation. Then the ‘most commented’ or ‘top of the week’ or especially ‘Top of All Time’ and old pinned posts, basically the ones people would want to see, that define a community, would over time be federated as users browsed, accumulating a select subset of pre-federated posts on-demand. Also, it seems like having all the votes and comments on a posts that you just requested would be important. Right now if I search for a specific pre-federated post, I get a bare bones version of it with a tiny fraction of the votes, and usually 0 comments. Something like that seems like it would be a huge step in usability and discoverability, especially for users on newer, smaller instances. I don’t know much about how the system works though, so I don’t know it this is the right idea.