CoC - new contribution system

Platform Evolution
#42

The main problem is that the number of reviewers willing to (almost) blindly approve a clash is much greater than the number of those who will really look at the problem and give feedback or reject the contribution if necessary. I have never seen a good contribution get rejected under the current system, but bad contributions get approved regularly. The process is clearly skewed one way.

The freeze period is a good idea to give the contributor more time to react to feedback before the contribution gets approved regardless of the issues. However, it does not solve the problem when the author does not respond to feedback. After the freeze period ends it is very likely even a flawed contribution will get 3 approvals faster than 3 rejections.

To balance this I propose a requirement of 5 approvals but only 3 rejections to give the moderators who pay attention more of a chance. I often find myself rejecting a flawed contribution (that might be salvageable with changes) that has already received 1–2 approvals out of fear it will get through and never get fixed. If the requirement was 5 approvals instead of 3 it would be easier to justify leaving a comment instead of immediately rejecting the contribution, which should improve the experience for new contributors as well.

3 Likes
#43

It’s also a good idea, and shouldn’t be difficult to implement.

#44

:raised_hand: Hi to all :slight_smile: !

Last week, after reading your suggestions, i remember that there was already a clash rank in the system …
I don’t known if it would be easy or difficult to implement, but i think we can consider that the fist 10 / 100 / 1000 ranked clashers have a good minimal experience with the fundamental rules about clash to distinguish wish one is more or less good (acceptable) and those are bad or worst ones (may be rejected or updated) …
→ i suggest that at least one of the 3/4 or 5 approvals may have a good rank (or rank score) to validate a contribution !
→ in addition, a rejection from of one of these can block or delay the automatic acceptance process …

I think many of us are familiar with tools like Github or Gitlab, in witch an admin comment can block, for example, merge requests … until the author replies and satisfies worries of reviewers :thinking: …
→ maybe apply some similar functionalities ?

#45

Well, I’ve stopped playing clashes, and fallen from top 100 to top 1000. Probably I’ll fall out of top 1000 at some point too :laughing: Anyway, how do you draw the line as the ranking keeps changing?

The following is what the top leaderboard of CoC currently looks like. Level 4, huh…

top clashers
top clashers2356×1236 141 KB

#46

Why not have it be that all contributions are pending for at least 30 days.

And then use some formula at the end of 30 days to determine if it gets published or not. Maybe it has to have three times as many approvals as rejections or something like that.

#47

Linking the number of approvals and rejections is an interesting idea! But I’ll suggest “three plus” instead of “three times”, and a limit to that, e.g.

No. of existing rejections | No. of approvals needed
          0                |           3
          1                |           4
          2                |           5
          3                |   Automatic rejection
2 Likes
#48

I think it’s a good idea, but does CG want to do something ? Nonofr saw our request on the discord, he agrees with the freeze period, but he didn’t say if something will be done.

#49

I got a new idea yesterday, after a new contribution (this time a puzzle) was rush approved in few seconds only.
The freeze period after a contribution becomes ready is not enough, because it can be rush approved even if there still are things to fix.
I though about a freeze period between two approvals.

So I’d like to summarise the proposals that have been made here with my new idea, and if you agree with that maybe we can suggest it to CG’s team (I don’t know if they will do something, but we can put all the odds on our side).

  • A contribution can’t pass ready if it has not been visible “WIP” for a period (12h? 24h? 48h?)
    (if a contribution change from ready to wip, it can be ready again without waiting one more time)
  • When a contribution passes ready, it can’t be approved for a freeze period (12h? 24h? 48h?)
  • When someone approve a contrubition, nodoby else can approve for another freeze period (4h? 8h? 12h? 24h?)
  • To be fully approved, a contribution needs 3 approvals more than rejections
No. of existing rejections | No. of approvals needed
          0                |           3
          1                |           4
          2                |           5
          3                |   Automatic rejection

By this way, I think it would be impossible to rush approve a bad contribution with friends, without seeing it and reject it.
Does anyone want to add or modify something ?

1 Like
#50

  1. I would suggest a WIP period of at least 48h (I’d much more prefer 1 week, though) so that there is sufficient time for:
    • the initial reviewers to review the contribution,
    • the author to act on the comments,
    • the reviewers to review the contribution a second time.

    Also, we live in different parts of the world with different time zones, so 12h is too short and 24h is barely sufficient for any actions.

  2. The freeze period described in your second bullet point is interesting. I didn’t like the idea initially and thought it was redundant. But the more I think about it, the more I like the idea. It offers kind of a final chance to review and fix things up before formal approval, because there may have been substantial changes to the contribution between a reviewer’s initial review during WIP period and when the contribution becomes ready for approval.

  3. Freeze periods between approvals is also an interesting idea. But if that is implemented, does it mean that freeze periods also apply to rejections? Or we don’t impose any restrictions on rejections?

  4. To make things less complicated, we can combine the second and third bullet points to say x hours (x = 4/8/12/24/some other number) must pass before the first approval / another approval.

#51

In my idea, this doesn’t apply to rejection, so that we can reject a “rushing contribution”.
It gives more power to those who reject, but we haven’t yet seen a “rush reject” for the purpose of harm.

1 Like
#52

New summary :

  • A contribution can’t pass ready if it has not been visible “WIP” for a freeze period (48h? a week?)
    (if a contribution change from ready to wip, it can be ready again without waiting one more time)
  • When a contribution passes ready, and when someone approves a contrubition, nodoby else can approve for another freeze period (4h? 8h? 12h? 24h?). The freeze period doesn’t apply for rejections.
  • To be fully approved, a contribution needs 3 approvals more than rejections
No. of existing rejections | No. of approvals needed
          0                |           3
          1                |           4
          2                |           5
          3                |   Automatic rejection
1 Like