Defederation is simultaneously very useful and very dangerous.
Large instances have the power to kill smaller ones by defederating them, since they control an overwhelming share of Fediverse content. That is a lot of power in the hands of a small group of people, each potentially with their own views and agendas.
I think defederation should be reserved for openly malicious servers and used as an absolute last resort. Think of the Internet and how horrible it would be if countries decided to just disconnect from other countries based on conflicting ideologies.
The vast majority of users on legitimate instances just want to explore interesting things, share some thoughts and have a good time. Defederating hurts legitimate users the most, trolls can easily hop instances and find another way to troll.
This is a Lemmy-ui bug, not related to CDN/cache.
I had submitted a bug report for it some time ago but it’s not that urgent in the great scheme of things so they didn’t get around to it yet: https://github.com/LemmyNet/lemmy-ui/issues/1865
(the report speaks about this happening after you log in, but it’s the same thing).