# What is Lemmy? Welcome everyone! Lemmy is a reddit like platform that is
broken up into multiple websites rather than one website owned by a single
company. Structuring Lemmy as multiple websites gives users more power and
control over their content. For example, lemmy.ml [http://lemmy.ml] and
beehaw.org [http://beehaw.org] are both Lemmy websites. Any single Lemmy website
acts much like reddit would: you can create an account on the website, leave
comments and subscribe to subreddits (Lemmy calls subreddits communities). The
power of Lemmy is that two different websites are capable of allowing users to
interact with users on the other. Each unique Lemmy website is called an
instance. For example lemmy.ml [http://lemmy.ml] and beehaw.org
[http://beehaw.org] are two different Lemmy instances. The approach of enabling
websites to talk to each other is called federation. Federation is the opposite
of centralization. For example, reddit is centralized in the sense that it is
owned and operated by a single company who can force their will onto their
users. On the other hand, if a single Lemmy website started acting outside the
interest of their users, those users could start using a separate Lemmy website
and still access all the same content. # Your Lemmy feed Like reddit, when you
visit a Lemmy instance, you’ll be presented with a feed of posts. Unlike reddit,
these posts can come from any Lemmy instance that is federated (talks with) with
your instance. For all the control that a federated platform gives you, the cost
is that you need to do extra work to get each of these separate Lemmy instances
to talk to each other. Lemmy instances discover each other and start federating
when a user from one instance subscribes to a community from another instance.
That means when you’re looking at a specific Lemmy instance, the feed is
comprised of posts from communities that all the users from that Lemmy instance
have subscribed to. You may filter this feed by looking at “Local" vs “All”, but
“All” still only means all the communities that your Lemmy instance knows about.
To teach your Lemmy instance about new communities, a user from your Lemmy
instance must subscribe to that community. # How to use lemmy.directory? Our
goal is to try to provide a feed of the widest range of Lemmy communities
possible, by subscribing our Lemmy instance to nearly every community on other
Lemmy instances. This aggregation of communities can be seen by browsing our
“All” feed. You can imagine us trying to be like reddit.com/r/all. So what we
offer you is simple: hang out on our “All” feed and see what the whole Lemmy
fediverse has to offer. And if you find any new communities you’re interested
in, bring them home to your Lemmy instance. By searching for a community on your
home Lemmy instance and subscribing to it, you’ll add that community to your
home instance’s feed. The best part is, doing so enables everyone else you share
an instance with to also start seeing content from this new community.
Subscribing your home instance to new communities helps share them with more
people & helps us to build community with each other. If you don’t have a home
instance, you don’t need one. You can just hang out here and browse “All” in the
same way you could browse reddit. If you’d like to make an account to upvote or
leave comments, take a look at join-lemmy.org [http://join-lemmy.org] to find an
instance for you to make an account on. Then help grow that instance by
subscribing to new communities you find here. # More community building In
addition to offering a robust feed, this instance also provides a few
communities of our own. Please feel free to check out and post on
!suggest@lemmy.directory [https://lemmy.directory/c/suggest], a place to ask for
where to find communities matching your interests. We’d love support from others
to answer people’s questions and point them to the right place! We also offer
!promo@lemmy.directory [https://lemmy.directory/c/promo] as a place to share out
new communities you’re trying to build or advertise new Lemmy instances. Lastly,
feel free to create a post in !chat@lemmy.directory
[https://lemmy.directory/c/chat] if you have any questions or feedback for us.
Please especially leave a comment if you find any communities that we’re not
tracking and that you think we should subscribe to. Or if you find any
inappropriate communities that you recommend we unfollow or defederate from. #
Can I make lemmy.directory my home instance? For now, lemmy.directory user
registration is closed. In preparation for Reddit’s blackout
[https://lemmy.ml/post/1148152] and in anticipation of an increase in the number
of new Lemmy users, all of lemmy.directory’s resources are going towards serving
an reddit.com/r/all style feed. In the future we may consider opening
registration.
I recently setup a new Lemmy instance and was surprised when my feed was mostly empty. I’ve since learned that a key part of Lemmy’s federation is based on a user from your instance subscribing to communities on other instances. Only then, will your instance pull in posts from the subscribed community to your “All” feed.
This means that subscribing to new communities is especially important if you’re on a young Lemmy instance since it helps to build out everyone’s feed on that instance.
I’ve found discovering new communities to subscribe to on other instances can be difficult. To help me search for new communities I may be interested in, I tried aggregating as much of the Lemmy fediverse together into a single feed by subscribing to the widest range of Lemmy communities possible. This offers a Lemmy feed that’s kind of like reddit.com/r/all. If it’s interesting to anyone else, you can find the instance here: https://lemmy.directory.
Hopefully this offers another way to find new communities to subscribe to on other instances.
Here’s a better description of my understanding on how Lemmy federates communities and why you might be interested in checking out lemmy.directory: https://lemmy.directory/post/34207.
Hope this helps ease the orientation to how Lemmy federation and communities work.
So based on your answer, I am still misunderstanding something. What is the purpose of all the duplication then? Is it just for local caching purposes?
Pretty much everything in your summary was wrong. I can’t reasonably type out a complete primer on federation here, but in short… When the first user on an instance subscribes to a remote community, the subscribing server tells the community-hosting server “send me future updates about posts, comments, and votes and such for this community”. The subscribing server then stores them locally.
Why do this actual replication rather than just an API gateway?
That’s what federation is. I don’t mean to be glib, but there’s more than one way to potentially do this stuff, and some systems designers think federation is a useful approach for a variety of complex reasons. Federation involves federated replication, API proxies aren’t federation.
If the community-hosting server goes down temporarily, content replicated via federation remains available on the subscribing nodes. This enabled me to read communities homed on lemmy.ml recently when it was struggling and frequently down due to overload.
It spreads the browse workload, which is generally bigger than the replication workload. For example, lemmy.world has commonly has like 4k active users this week. But it only takes one batch of federated replication messages for the community’s instance to serve the browse traffic of all those users… then the reads come out of lemmy.world’s db. This spreads the browse workload around the federated network.
There are many other reasons to work this way as well, but as noted… at some point you’ll just want to Google primers on federated apps/networks. They definitely don’t operate according to your current intuition though, and the differences aren’t accidental.
Can federated networks have problems where the replication traffic becomes “too much”? Yes, they must be carefully designed to avoid that problem. And for some apps, it makes single-user instances sometimes anti-efficient as the federation workload for the instance can exceed the browse workload from the user, but for multi-user instances the federation messages are a rounding error compared to the browse work. But replication overload is a problem that federated networks generally weather, and the replication offers benefits that on-balance outweigh the costs.
Pretty much everything in your summary was wrong. I can’t reasonably type out a complete primer on federation here, but in short… When the first user on an instance subscribes to a remote community, the subscribing server tells the community-hosting server “send me future updates about posts, comments, and votes and such for this community”. The subscribing server then stores them locally.
Why do this actual replication rather than just an API gateway?
lemmy.mlrecently when it was struggling and frequently down due to overload.lemmy.worldhas commonly has like 4k active users this week. But it only takes one batch of federated replication messages for the community’s instance to serve the browse traffic of all those users… then the reads come out oflemmy.world’s db. This spreads the browse workload around the federated network.Can federated networks have problems where the replication traffic becomes “too much”? Yes, they must be carefully designed to avoid that problem. And for some apps, it makes single-user instances sometimes anti-efficient as the federation workload for the instance can exceed the browse workload from the user, but for multi-user instances the federation messages are a rounding error compared to the browse work. But replication overload is a problem that federated networks generally weather, and the replication offers benefits that on-balance outweigh the costs.