Hi! I am a researcher at the CISPA-Helmholtz-Institute for information security (https://cispa.de/en/people/kuennemann). I would like to get some community feedback on a project we are working on. I am a Lemmy-user myself, but of course I use my other account for lurking; this account is for my research business.
My question taps into the age-verification debate, which I know is very controversial, and people are sceptical for very good reasons; I just ask everyone to stay polite and engage the topic.
In short, I am worried about the effect bots have on public discourse, which to a large part takes place in the Internet. I would like to propose cryptographic methods that allow an instance to attest that a poster or commenter are a real human being. I think the European Unified ID (euid.eu) provides good infrastructure to do that, at the cost of relying on the member countries to provide them with an ID. Crypto-magic can (potentially - I haven’t worked out the details yet) ensure that:
- users from an instance that opted in are sure to belong to someone with an id, other instances may use this knowledge in their moderation tools
- users from that instance can retain several pseudonyms that even this instance cannot link. Yet (configurable) policy can ensure that no users uses two pseudonyms in the same thread
- we could cap how many posts or comments a user makes per day/week/month, across their pseudonyms
- users who post on this instance but are not displayed (in the web or to other instances) can provide evidence that they are being censored, which the public may review to discredit that instance and chose a new one.
- the infrastructure also happens to allow for age verification (EUID can attest to statements like “this anonymous person is > 18”), and it could be part of an instance’s policy. Thanks to lemmy’s federated infrastructure, multiple verifying instances with different policies can exist. It’s not a focus of mine, though – I think social networks should not exploit children or adults.
Some of these ideas we have worked out (and tested on) reddit in this article, which was before Lemmy became popular and EUID was on the horizon.
I am interested in practical advice about who to talk to about engineering and how to best involve the community – we’ve got a prototype fork of lemmy including end-user verifiable authentication. I am also interested in how the community feels about the burden of moderation and this as a potential remedy. When discussing age verification, some users say they will never accept any ID-based authentication. Are there some amongst those who would change their minds provided they had full pseudonymity and censorship could be detected and made public with cryptographically signed evidence?
this is a bad idea, just seems like opening a whole can of worms for state censorship
i don’t trust
Crypto-magic
just seems like now there’s going to be even more personal information floating around out there to get stolen or subpoenaed
I think proof of personhood, no matter how it is done, is problematic either way. So many things can go wrong. The issue stems from the question where/how to obtain said proof besides a government / institution backed document. Institution will always be problematic one way or another. Look at how TLS certificate mafia operate. Only a few “trusted” companies can make it to be the root of trust by default on billions of devices. If you want to use a selfie then there is an issue of ease of forgery. Let’s face it, even many ID verification services out there struggle with it. Not to mention forging a photo realistic image nowadays is quite easy with generative AI. Overcome the issue of how said proof can be obtained then I am interested.
do everything in your power to sabotage the creation of this garbage
Don’t you think sabotage goes a bit far? I don’t have any project funding that requires me to work on this, I do this on my own accord and with the goal of fighting astro turfing, especially in political forums. (The article details this motivation in the introduction.) You may disagree with this approach, and I can see that ID authentication is really bothering a lot of people. But I am taking in feedback at an early stage. Isn’t that worth something? I might even drop the project!
sabotage is absolutely called for because they’re trying to build a panopticon and that will be used for purposes far more nefarious than what you think it will stop.
I don’t think tying online identities to real ones is a good idea, especially with all the state repression going on especially in the EU.
You’re free to fork lemmy of course, but I doubt people are going to want to give a server their real identities.
In your personal view, does it change matters if the instance does not obtain the real identity, just the mathematical assurance that you have an ID and are not using your pseudonyms in the same thread?
A type of repression that this particular approach cannot protect against is that the ID provider might refuse to attest somebody’s identity – just like how a government could refuse to issue an ID. But we could provide accountability: tangible proof that somebody was refused to login. I think this would be a big deal, like government refusing somebody their physical ID, but it relies on the rule of law and a democratic society producing pressure.
What’s your main concern, privacy or censorship (or something else)?
I don’t know how that’d work without you giving some personal identifying info to some 3rd party of some kind to verify, and people aren’t going to want to do that.
Anyways if bots are the concern there’s better ways of keeping them out. Lemmy is mostly free of bots due to registration applications, and user reports which can find and report them quickly.
I used Reddit when the only thing they asked of me was a username and password. When they told me I could clear my account’s name for ‘suspicious activity’ by providing an email address, I stopped using Reddit altogether. If there is a class of ‘bots’, I am of it.
The difference to reddit would be that in a federated network, this could be decided on a per-instance basis. I assume this is not mainly a convenience problem for you, is it? But more distrust towards Reddit (and what they would do with your data) or the ID issuer (and if they would use this to lock out dissent)?
For the record, I am starting from the premise “we need to limit the amount of astro turfing” because it distorts political discourse. There’s also other ways to recognize bots, but they are not very reliable. The goal is that the issuer does not learn what you do in lemmy and that lemmy (including that magic instance) does not learn who you are. Would it be a bigger problem for you that such a complicated system could be flawed, or that the issuers gets too much power over who can speak (or something else entirely)?
The issue is that I and most other people are not technical enough to audit such a system and be satisfied that identifying info was not retained or intercepted, or the code furtively changed or disguised etc. etc. If what you’re suggesting can be done, that’s neat, but it makes no difference to me because I cannot know when it is and isn’t being applied or if it’s being comprehensively applied, and I will go on assuming that if I show my ID, it is leaked or intercepted instantly. And knowing I have shown my ID would distort my political discourse for sure.
If some people are not certain whether I’m an astroturf, that’s a tradeoff I can live with.
Absolutely fucking not.
Make a upstream PR as an optional feature for instance admins to enable.
I doubt many will do, due to not having an EU exclusive membership and overall privacy concerns, but it doesn’t hurt to have as an option.
That’s the goal. Our analysis assumes a heterogenous network with instances that have this feature and other’s that are unmodified.
No one is going to tie an ID to their account
Ideally, you’d want to get feedback from admins rather than general users. The moderation burden is basically hidden from us.
Given I am not European, any server that required EUID would be off-limits for me. But unless you plan on building it into activitypub itself, anyone who wants to use a non-EUID account can just spin up their own instance and use that completely unattested.
Post/comment limits per account would either be too low, and annoying for some users, or too high, and completely ineffective for combatting spam posts. Spammers will typically only send 1-3 posts per day.
I’m not sure what your bringing to the table for censorship either, the modlog is public, and federated, how does one get silently censored?
Good point – where do I find the admins? Preferably an open forum, like a community.
You are on point regarding spinning up an instance. Other countries / blocks will have different ideas about whether to do this and how, so this would be a demonstration that this is possible. I assume such an instance has less spam (tbh that’s just a guess), so admins may be happier federating with such instances.
Admittedly, once big countries or blocks provide attested identities and should it be true that they have less spam / bots, other instance might be under pressure. So this could still be seen as dangerous to folks that do not like to be forced to derive digital identities from centralized authorities and lead to fragmentation in the network.
Post/comment limits per account would either be too low, and annoying for some users, or too high, and completely ineffective for combatting spam posts. Spammers will typically only send 1-3 posts per day.
In the linked trollthrottle paper we ran some analysis on reddit data, and it turns out that football / cricket posters have incredible commenting frequency and the cap might be a big problem for them. But does it actually fight spam / astroturfing? Do you know a good source about spammer behavior, or are the 1-3 posts per days from experience?
I’m not sure what your bringing to the table for censorship either, the modlog is public, and federated, how does one get silently censored?
Your instance just silently drops your post after you have written it.
I need to have a closer look at what the modlog actually covers, but afaik it does not cover every deletion (permanent purges are mentioned? not sure). Moreover, in our super paranoid threat models, we usually consider the whole instance untrusted, not only some moderators, meaning that a malicious instance ought not to be able to alter the mod log (more precisely, the view of third parties) without this being detectable. In trollthtrolle, we achieved this with an append-only log and a cryptographic receipt, but there are other, more modern ways to achieve this.
All the admins are listed on the instances themselves.
You realise most (all?) fediverse instances aren’t operated by countries? They are just random groups of people. There are no legal agreements or blocs tying servers together.
I’ve seen and reported plenty of spammers, most don’t post that often. They create a few posts and the go silent or get banned. Blocking after N posts still leaves N spam posts to cleanup.
Every post is federated to every other instance. My home instance could purge my posts, but all other instances would still have a copy that can’t be deleted by my admins. So the modlog is resilient by virtue of being distributed copies. If the post was dropped upon submission, then nothing is seen, but thats hardly a silent censorship either, I can immediately see that it wasn’t created.
I dunno, but I think you need a much stronger selling point to get people on board with this, it just seems at odds with reality.
Yeah, I know that instances are run by volunteers. Our prototype instance has EUID as an additional OAuth provider and embeds cryptographic evidence that a poster has a real identity (but not which it is). This information is shared via ActivityPub, even to instances that don’t know what to do with it. The instance can be run by anyone but publishes a policy, e.g., “all users on this instance are pseudonyms of a real identity, real identities cannot appear under different pseudonyms in the same thread, and post at most N per week”. The ID issuer must be trusted by end users who use the modified software to see if the instance adheres to its own policy. End users with the normal software don’t, they just see where the user is from and might trust that instance to uphold its policy, or not care at all.
I need to think (and read) more about how to validate the behavior of spammers. It is well possible that spammers don’t spam that often, but they might as well create new accounts after some N and operate in parallel with multiple accounts. I think the cap needs to be seen in combination with pseudonyms that are tied to identities.
Every post is federated to every other instance. My home instance could purge my posts, but all other instances would still have a copy that can’t be deleted by my admins. So the modlog is resilient by virtue of being distributed copies.
Ah, if your instance is honest. And if it is dishonest, you … switch to a different instance and live with the fact that you lose your “reputation”, i.e., tags at other user’s accounts, post history. Maybe it would be cool to be able to prove that you are the same person?
If the post was dropped upon submission, then nothing is seen, but thats hardly a silent censorship either, I can immediately see that it wasn’t created.
I was about to write "but you cannot prove that’, but if I get your drift, you’d just switch and not bother with making this public.
I dunno, but I think you need a much stronger selling point to get people on board with this, it just seems at odds with reality.
That’s why I am talking with you guys. ;) Thanks for your input btw. Despite using lemmy for a couple of years now, I’ve never seen the moderation panel … and then my perspective is very protocol-centric, although much about moderation seems to take place out-of-band.
I’ve never seen the moderation panel
That’s part of your problem. I commend you for putting so much thought into this, but it seems like there’s a lot of areas you’re missing. I would’ve expected a person pushing something like this to have A LOT of experience with the very thing they’re trying to change so drastically.
I’d say go be an admin or moderator for one of the bigger instances. I’m sure, with more experience, it’ll help better shape your perspective or even change it entirely.
