I downvote non-English results I see on my All page as a punishment for not correctly setting the language on the post, which takes 1 second to do and would ensure it doesn’t spam the feed for non-speakers. I can’t imagine I’m alone - was the post in question not correctly tagged as Polish content, like your post is correctly tagged as English content?
If it was correctly tagged, then those downvotes were all from people who speak Polish.
This stance has nothing to do with anglocentrism and everything to do with making Lemmy usable. You set your languages in your profile so you’ll only see posts and comments in those languages. No one likes seeing lots of posts in languages they don’t understand, and that that only happens when people are too lazy to set the language indicator. I’d fully expect and encourage non-English speakers to downvote improperly tagged English posts in their feed as well.