I’ve recently added anubis to lemmy.ml, and it seems to be working well.
I have a PR to add anubis to lemmy-ansible (our main installation method), and I could use some help tweaking / optimizing its botPolicy.yaml config, for federated services.
Anyone with experience running anubis, this would be much appreciated.



Well behaved bots should see the origin in the headers and only crawl those.