(Pollen) Ideas to counter discord cheating

[therealmo]

So, after today’s pollen distribution turned into a hot mess, I have taken it upon myself to come up with a list of changes that can be applied to Discord’s distribution system in order to make it fairer, more transparent and more difficult to game by bad actors.
I am going to split those suggestions into 2 categories, Technical changes (changes done to the system itself) and Organisatorial changes (changes done to the process of spotting and punishing the bad actors who manage to game the system)
In order to come up with solutions, we must first take a look and see where the problem lies, more specifically, how did the bad actors manage to cheat the system?

I.Technical Changes

Problem 1: Multiple accounts

Problem description: One pattern that was pretty noticeable among most cheaters, was the use of alternate accounts, or “alts” . Although BrightID is supposed to solve this problem, I think we can all agree getting a few friends or family members to verify a few accounts isn’t that hard. And given the fact there is a pretty substantial financial reward at stake, I think the cheaters won’t lack the motivation to do just that.

Solution: Implement a system similar to the merit system on Bitcointalk. On Bitcointalk, there is a merit system in place where an user has to accumulate a certain amount of merit in order to improve his trust level and unlock new features. They have a few members that have the ability to give merit ( merit source) who are mostly established members of the community and they are the only ones allowed to give it away. So how can we implement that on discord?

I propose that all new verified users on discord start without the ability to mint cred. After they receive an arbitrary amount of cred from one of the verified cred sources ( we can give Fauna and the Seeds this role) they can start minting cred on their own. In parallel to that we can also cap the cred level new users can mint on a time-based system.

So, until verified by a cred source, no cred minted. One week after being verified, a cap of 40 cred per week. After two weeks a cap of 80 and then after 3 weeks they become fully verified users and the cap is removed. The numbers are of course arbitrary and used only as an example, details can be worked upon later.

Problem 2: The Holy League of reactions trading

Problem description: “Gangs” of bad Bees have formed with the sole purpose of reacting to eachother’s post, essentially a like4like agreement between them.

Solution: Implement an asymptotic system similar to the one implemented for multiple reactions on a post ( A reacts to B one time: 1x cred. , second time 0.95x, third time 0.90x…)
Also, have it visible on the SourceCred ledger, to whom did that user react the most and who reacted to him the most, similar to the feature on discourse ( Most liked by…). This could make it more transparent and easier to identify people trying to cheat.

II.Organisatorial changes

Problem: No way to punish the Bad Hombres

Problem description: Bad actors are identified, but since there is no “official” way to report them, this turns into a mess of singling out users, witch hunts and mobs demanding justice across all 1Hive platforms.

Solution: Create a framework for reporting bad behaviour to the DAO members and make the process as transparent as possible. The users doing the reporting must prove beyond resonable doubt that someone is cheating and the DAO members must take the decision to deactivate that account unanimously and with no one being granted a Veto. I agree with Luke here and see no point in making the community vote on issues like that.

I would like to hear your thoughts on this and hopefully we can work this out together to avoid the same mess happening every week.

[Felix]

Thanks for posting this on the forum, which make more sense to put the ideas together. Everything seems well tought. I think you could add polls so we can vote to determine the value for the cred cap per week for differents roles.

I also think that DAO member should be able to deactivate the Bad Hombres without the needs for everyone to get involved.

[therealmo]

@Felix

Thanks for posting this on the forum, which make more sense to put the ideas together. Everything seems well tought. I think you could add polls so we can vote to determine the value for the cred cap per week for differents roles.

Thanks for your insight, Felix!
What values would you suggest putting into the poll? Or is there a way to let the voters write in their own suggestions? I never did a poll before on discourse.

@Felix

I also think that DAO member should be able to deactivate the Bad Hombres without the needs for everyone to get involved.

I agree, but only if they make it in a transparent way and the community is able to verify the whole process.

[FreshPrince]

Good idea, only thing I’d suggest for the solution to problem 2 is that the new asymptotic system gets reset after a certain amount of time (week or month). This way long-term members can still mint cred accordingly to long-term recurring contributing members.

I hope this makes sense, can’t seem to rephrase it

[therealmo]

@FreshPrince

Good idea, only thing I’d suggest for the solution to problem 2 is that the new asymptotic system gets reset after a certain amount of time (week or month). This way long-term members can still mint cred accordingly to long-term recurring contributing members.

I hope this makes sense, can’t seem to rephrase it

Thank you!
This is a good idea and I think we can implement the merit system into it, too.
For example, for a new user the system resets in 2 weeks. For fully verified users it resets in one week.

[Paddington]

Great ideas. I personally I think that using the 1hive trust system is the way forward. Higher the level, more cred you can issue. Found cheating, back to level 0.

[decentralion]

@therealmo

Solution: Implement a system similar to the merit system on Bitcointalk.

This makes sense. From an implementation perspective, what is needed is a bot that every week checks Discord users’ Cred scores, and assigns roles based on how much Cred the user has. We need a bit of work on the SourceCred side to make this convenient (make it easier to cross-reference users with their Discord ids; see here), but it’s do-able even now with a bit of hacking.

@therealmo

So, until verified by a cred source, no cred minted. One week after being verified, a cap of 40 cred per week. After two weeks a cap of 80 and then after 3 weeks they become fully verified users and the cap is removed. The numbers are of course arbitrary and used only as an example, details can be worked upon later.

Rather than make it time based, I’d propose something based just on Cred level, e.g.

Cred Score	Reaction Weight
0-30	0x
30-100	0.5x
100+	1x

@therealmo

Problem description: “Gangs” of bad Bees have formed with the sole purpose of reacting to eachother’s post, essentially a like4like agreement between them.

Solution: Implement an asymptotic system similar to the one implemented for multiple reactions on a post ( A reacts to B one time: 1x cred. , second time 0.95x, third time 0.90x…)

I worry that this will just encourage the bots to spam reactions on a lot of different messages, instead of just spamming a lot of reactions on individual messages. IMO, the root issue is that an individual account can mint unlimited amounts of Cred by just spamming reactions all over the place.

Instead, I propose we limit the total Cred that any account can mint to a fixed rate over time, e.g. 10 Cred per week. Each account’s reactions would mint the full amount up until they hit the limit. Then, once they hit the limit, the minted Cred gets divided evenly between all of the reactions. If a user has a positive weight, that weighting also applies to their overall Mint limit. If they put reactions in channels that have magnified Cred minting, it will also take a larger chunk of their weekly Cred budget.

This solves the same problem (attacker spamming many reactions on one post) but also covers the case where the attacker spams 1 reaction on hundreds of posts.

---

Cred Budgeting

I also want to mention a solution which @befitsandpiper has been advocating for, which is having Cred budgetting on a per-plugin basis. Then we could make a rule that (for example) there’s a maximum of 300 Cred per week from each plugin. This would limit the proportional share going to Discord, which helps ensure that contributions on the forum and on GitHub continue to be meaningful.

[Jasper]

@decentralion

This makes sense. From an implementation perspective, what is needed is a bot that every week checks Discord users’ Cred scores, and assigns roles based on how much Cred the user has. We need a bit of work on the SourceCred side to make this convenient (make it easier to cross-reference users with their Discord ids; see here), but it’s do-able even now with a bit of hacking.

We actually already have a function that returns people’s cred scores from the livepage ledger. Implementing this is easy on the discord side, the issue generally is that the livepage ledger is behind ‘local’ version releases vs the official sourcecred version release causing inaccurate scores by about a week or a few days. Topocount invited me to talk about the github issue you linked but I doubt I’ll be able to contribute much.

Happy to see you and some other sourcecred people showing up in 1hive to share your thoughts.

[therealmo]

First of all, thank you for taking the time to respond!

@decentralion

This makes sense. From an implementation perspective, what is needed is a bot that every week checks Discord users’ Cred scores, and assigns roles based on how much Cred the user has. We need a bit of work on the SourceCred side to make this convenient (make it easier to cross-reference users with their Discord ids; see here), but it’s do-able even now with a bit of hacking.

In addition to that, how would we go around changing the weight for cred given by Cred sources ?
Because the whole point of that system is that you can’t be fully verified and start minting cred until one of the Cred sources (Fauna, Seeds) grants you some. If we assign roles based on the amount of Cred one has, regardless of the source, we can still face the same issues.

@decentralion

Rather than make it time based, I’d propose something based just on Cred level, e.g.

Cred Score	Reaction Weight
0-30	0x
30-100	0.5x
100+	1x

I agree with that, when combined with the first proposal, this will make the system quite hard to beat.

@decentralion

I worry that this will just encourage the bots to spam reactions on a lot of different messages, instead of just spamming a lot of reactions on individual messages. IMO, the root issue is that an individual account can mint unlimited amounts of Cred by just spamming reactions all over the place.

I think I didn’t express myself well enough there. We already have an asymptotic system in place, so when you react multiple times to a post, the reaction weight goes down with each additional reaction.
I proposed we implement that so that everytime you react to a specific user not post , your reaction weight goes down. And have that system reset on a weekly or bi-weekly basis, so it doesn’t affect long-term members.

@decentralion

I also want to mention a solution which @befitsandpiper has been advocating for, which is having Cred budgetting on a per-plugin basis. Then we could make a rule that (for example) there’s a maximum of 300 Cred per week from each plugin. This would limit the proportional share going to Discord, which helps ensure that contributions on the forum and on GitHub continue to be meaningful.

I also agree with that and would like to go even further and say that we should also budget individual channels on discord.

One more idea that I saw being mentioned on discord was bringing the #memes channel weight to 0 and then just doing a meme of the week competition to reward the best 1… or 3 memes.
I think having the meme channel weight at anything other than 0 will encourage full-time meme spammers whereas having the channel weight at 0 will increase the quality of the posts because people will be inclined to post only when they have a really good meme.
Regarding the individuals who are spamming the meme channel with low effort posts, I am absolutely positive they will stop doing it once there is no financial gain.

[befitsandpiper]

we can change the cred minting weight based on cred scores using discord roles

[befitsandpiper]

Okay so there’s lots of discussion over this last week or so, I would say the general consensus is:
pollen is broken. Something needs to change in how the distribution works.

We have some options here:

• we can lower distributions in general, at least temporarily while kinks are worked out, and to reduce both the incentives for system abuse, and the harms done by the abuse
• make the system involve trust levels, maybe making minting cred dependent on the cred a member has already, or some other time-based throttling, or directly assigning roles that represent higher levels of trust
• direct moderation: this involves heavier hands of deactivating members who appear to be making sybils, colluding etc. I think this is one of the trickiest solutions to manage, and we still don’t have a good process for this if we were to go down this path.

Main immediate technical solutions are:

• creating a cred budget per plugin. This makes it so for example discord cannot completely overtake the cred produced by github
• changing the ratio of immediate (last week) payouts vs balanced (all time) to favor all time contributions. This reduces the risk of double paying brand new members.
• assigning roles based on cred scores in discord
• maybe reducing maximum daily likes on discourse
• limiting the amount of cred each member can mint per day/week

None of these solve the problem immediately, this is a long process of improving this tool, and likely pollen will never be an accurate measure of value. We can only strive to improve it.

[DogeKing]

Well to start like @therealmo said def we need some system like bitcointalk which is uniqe in some way(more cred you have / you are slowly becoming a trusted user in the community ).

Something like that maybe some changes but its nice template.

@decentralion

Cred Score	Reaction Weight
0-30	0x
30-100	0.5x
100+	1x

Next thing what I have in my mind is that maybe payouts need to be smaller and maybe this week which is coming for next payout to be on hold until something has changed in pollen system.

[oliver_30]

There’s a lot of good suggestion on how the community should move toward regarding pollen distribution, we must all know that this system is still new and are prone to some abuse and loopholes. I think the best course of action rn is to to lower the the amount of payout everyweek or suspend it until there’s a good system to prevent this gamer and cheater. I prefer the latter one so that we can have a breather and can have a better assessment and better judgement about the issue. I am new here so I still dont know what it’s like to get a payout and I don’t mind waiting other more week until the system is more optimal.

[Rayne]

Hi guys! One of the counter-measurements we have implemented is to include something that is known as an asymptotic dropoff for multireacts. The idea is that the significance of each follow-up react to the same message is halved. Meaning that if I give a emoji , and a emoji, and another emoji, that the amount of cred given equals to 2x + 1x + 0.5x = 3.5x instead of 2x + 2x + 2x = 6x This, however, means that a user can multireact once again to another message of a user, meaning that they can get the same effect of a single non-asymptotic 3-react message by simply reacting to 2 different messages with the same react emojis. (3.5x + 3.5x = 7x). Which is why I think we should get possibly rid of this approach and improve upon it.

The solution I do think is best, and has been mentioned already before, by @therealmo and @decentralion is to have an asymptotic dropoff on the amount of cred given through multiple reacts per unique user, rather than per message. This makes sense, in my eyes, because users do not multi react to every message another user posts. It wouldn’t make sense for me to go and multi react to multiple messages of luigy lemon in a course of 24 hours. I think this approach helps in two ways:

• Users cannot spam multi react multiple messages, to achieve the same amount of cred as if there was no asymptotic dropoff. This especially counters gamers as they cannot spam multi react multiple messages.
• Users are capped on the amount of cred they can give to each unique user per specific time period.

There are two ways of handling the restoration of the amount of cred user A can give to user B:

Option 1: An exponential decaying curve which kicks in as soon as the first react is placed. This means that if User A reacts to User B, the first react has a weight of 1x, while the next react’s weight is halved (=0.5). Over time the weight slowly recovers to 1x.
Here is a simple example of a graph I quickly plotted. In the case that user A reacts to user B, the weight immediately drops to 0.5x. (T = 12). After the initial first react, user A’s cred mint allowance slowly creeps back to 1, over a period of 12 hours. If User A reacts during the 12-hour time-frame to user B, say at T = 5, a weight of 0.74x is taken, the mint cred weight resets to 0.25x and the timer starts again.

These are just numbers that can be tweaked, I just gave an example of 12 hours as it meant that the total mint allowance is reset after 24 hours.

Option 2:
Have a hard cap without any recovery of the asymptotic dropoff over time. Meaning that in say every 24 hours, the weight just automatically resets to 1x. If User A reacts 3 times to user B within 24 hours, the total weight will just be 1x + 0.5x + 0.25 = 1.75x.

Once again, the emojis are sorted by their weight. Meaning that if user A’s first react is a lower weight emoji such as , which is then followed by a emoji, that there is an automatic ordering based on weight.
Lastly, you might say, what if a user makes multiple accounts to just react to each one of them, to counter all of this? I think personally it’s very hard to just 1.2.3 get 10 verified BrightID accounts, so there is a barrier of some sort.

[cryptoclip]

This must have been merged from your other thread because my reply has been deleted in summary I like both ideas, I think we should decide how much cred is appropriate for 1 user to give another user per week total and then can work back from there. E.g. if max 24 per week then your option 2 fits well at 2 per day.

If we then combine that with how much cred total a user account should be able to mint per week, again if we say 24 that’s 2 per day sounds fair to me.

Let’s say 2000 active users minting cred, 24 per week, 48,000 per week total under those rules - how does that compare to how much cred is being minted total per week now - seems like a huge amount?

[METADREAMER]

Another strategy to mitigate the impact of people “reaction trading” could be pairwise bonding (used in Gitcoin grants): https://ethresear.ch/t/pairwise-coordination-subsidies-a-new-quadratic-funding-design/5553

[youre_hot]

Vitalik approved. Seems promising