Ok. Let’s take the guideline one by one. (my remarks will be in italic)
General:
1 - Contributions must be written in English. : ok
2- The description must be clear and unambiguous. : of course. Just take the reverse think about it and then remove this sentence (the door is obviously open).
- Keep it clear and consise : _cf point 2 (duplicate + absolutly no interest)_
- Avoid flavour text : _ok_
- Don't address the reader directly : _Why the hell ? But if you want to why not._
- Avoid controversial topics : _Obvious (if someone can define "controversial" properly)_.
_ Don't overlook the protocol : _ok_
3- The default code must be working for all languages. : ok
4- Test cases must be properly defined.(for all games but multiplayer games)
- Test cases should cover all specifications : _ok_
- Test cases must have explicit names : _why ? It might help the reader for certain propositions but not for all._
- Each validator must differ from the corresponding test : _ok_
- Each validator should check the same case as the corresponding test : _ok_
- The first test case must be a simple one : _ok_
5- Contributions must be original.(for all games but Clash of Code) : ok but why the hell not on CoC
…
CoC guidelines:
1- CoC puzzles should be solvable in less than 5 minutes. : By who ? My 14 years daughter is probably not able to solve https://www.codingame.com/contribute/view/4508618d24711919402a7811f21247d27f1d in 15 minutes. Should I remove it for that ? Or should I take into account take SHE HAS NEVER WRITE A LINE OF CODE ?
The main goal of a CoC battles is to be short (except in the "Shortest" mode). CoC players don't expect to learn something from solving a CoC puzzle; they'll choose practice puzzles for that. : _Who told them that ?_
CoC close duplicates are allowed. : _once again why ?_
If the puzzles' themes differ, it's ok to accept multiple versions of the same programming problem. : _cf previous remark_
No CoC puzzle is too easy. : _let me laugh please._
No CoC puzzle should be rejected because it's too simple. : _Redundant with previous sentence._
I just give an example of judging criteria: if 80% of players are able to complete a clash within 2 minutes in the fastest or reverse engineering mode, it is deemed to be “too easy”. The threshold can be changed based on real statistics collected on the platform.
Although this is mostly a post-moderation critieria, the pre-moderation workflow might also be changed to accommodate it: Everyone who votes on a clash is required to provide a score of its difficulty level. If the clash is rated to be too easy, the rater is required to provide a solution to it (like in the fastest mode). The time taken to solve the clash can be a relatively objective measure of the difficulty level.
Maybe the difficulty rating could be asked for anyone who accesses the clash. And the author’s solution would only be revealed after the rating is provided.
Guideline 2. Indeed, the description guideline doesn’t apply to reverse mode.
Guideline 4. Having a meaningful name for a test can help players understand what the test is about. Maybe it does not, but still I think it’s a good practice. Probably it helps reviewers to understand the different cases that are checked too.
Guideline 5. The more CoCs we have, the better for players. Why should we allow close duplicates (same or really close solution but different statement)?
Because of the modes and the limited amount of COCs puzzles, any player will eventually play again the same Clash, which is a shame. Allowing close duplicates of existing puzzles could be less frustrating for a player to solve them than solving again the same ones.
Having more CoCs also reduces the frustration of new players encountering those who play a lot and know all CoCs by heart.
Creators don’t always have access to all the Clash pool and don’t know a similar Clash already existed. Feels bad to reject their contribution because of it.
Of course, we want to avoid having 15 close duplicates of the same Clash. Maybe we could put a limit to 3.
Yes, players could “farm” XP by creating close duplicates of existing CoCs. I don’t see this happening so much though. (most of the times, they don’t know their CoC already exist). We could also think about changing the amount of XP each contribution rewards.
Specific CoC guidelines: CoC is speed game. The light difficulty of CoC puzzles is actually the core of CoC. If you don’t find it fun to solve easy puzzles, just don’t play or moderate CoC. I understand that really good developers like you would also enjoy a speed game with challenging puzzles but that simply doesn’t work with the current CoC setup (no difficulty level or leagues) and players.
What do you think? Does this help to understand our philosophy behind Clash of Code?
Am I wrong if I write you just
1- removed the last guideline No CoC puzzle is too easy. (the restriction at the end is so unclear that one can always argue " overly-simple puzzles"
2- put a maximum duplicate of 5 which IMHO is just worst than allowing duplicates since no one will agree on almost duplicate things. Let’s take an example : are Add0 et LeftConcatenateEmptyString duplicate ? One can say yes since the code to solve both are the same. One can say no since the problem are clearly not the same.
Following discussions in the chat with @Stilgart, I’ve updated the guidelines to set the guideline “test cases must have explicit names” as just a best practice. I’ve also mentioned in the “Defining Test Cases” page that sometimes, to cover edge cases, it’s necessary to have the same validator as the test
I didn’t answer @Boulet’s comments earlier because I didn’t know what to do next. (I’ve also been busy with other things). The thing is that I’m not sure we will ever reach an agreement on the difficulty and duplicate guidelines. I see two options right now (which I don’t like at all):
Enforce the rule and remove moderator rights to everyone who don’t respect it (I would probably have to ban you from CodinGame if I want to avoid another “Add0” episode ). Despite our disagreements, I don’t believe any moderation can work by force, and I certainly don’t want to exclude people who think differently.
Accept the status quo and live with a double-standard moderation system. I don’t think the current situation makes for a welcoming community when it comes to contributing to the platform (and I know we, CodinGame, share a great deal of responsibility in this situation). Even if CoC is the ugly duckling for many people in the core community, a lot of new players still enjoy playing it so I’d like to ensure they continue enjoying it.
So I’ll brainstorm internally and do some research (perhaps also a survey on CoC players) to find a new option everyone could agree on. Any constructive idea is welcomed.
I hope this is the right place to leave some thoughts from the player’s side (as I don’t find any CoC specific feedback topic). Just fyi, I also read contribution guidelines as well as this entire topic including links here and current pending contributions.
TL;DR there are too many hard/long/confusing clashes and it’s not what you expecting
I solve about 50 clashes in the last few weeks on my way to complete corresponding achievement and I would say at least 30% of them don’t match the letter or the spirit of the guidelines. There was some with overly artistic descriptions, descriptions uses kinda specific terminology (which may be hard to understand to non native english speaker), puzzles that takes 10-12 minutes minimum to solve (not only by me but also other clashers). Can you imagine reverse CoC which take 5 sec to understand what to do and not have time to implement? Despite I’d always try to leave feedback on such clashes with proper explaining, I hardly believe them being read and/or ever taken some actions on it.
CoC puzzles should be solvable in less than 5 minutes.
CoC homepage offers 5 min clashes which is usally false. Speed is an essence of CoC.
About 10-20% of my clashes was solved by 15-45 seconds and know what? They was the best I ever solved and I never was dissapointed or bored.
420 clashes ahead, so help me God.
This guideline is so subjective and thus unusable that I can hardly seen any interest. As already written by myself
Is this a reason to reject it because it is too hard ?
Can you explain us where is the interest of fastest typing CoC. Not a fastest finding but typing. In 15-45 seconds you just have time to type 20 chars and no time to read the statements ? CoC are supposed to be programming challenges not pavlov’s reflex challenges
There is at least one viable strategy already to determine puzzle complexity:
Regarding fastest-typing-challenges… with my non-blind 4-fingers typing I’m able to write ~3 LOC for 30sec if I know what to do. I’m pretty sure that 30-45 sec is enough to understand easy puzzle and write 1-2 LOC of simple math using default code. Writing 20 chars in 30sec is extremely slow, isn’t it?
Not a real interest to write code fast, but faster than others.
Agreed, when I see 5 min., I do expect that I will need more than 1 min. of effort to solve a CoC (which is usually not the case).
That being said, speed is especially why I personnaly hate CoC.
I’d rather have two leagues: one with a 1min. duration, and one with a 15min. duration (which is closer to what was CoC at the beginning, with the 5/10/20 min. formats).
I think guidelines should be renewed since a year has passed and not much from this conversation was solved.
Has conversation moved to anywhere else? If so then why a link to this post is in the guidelines?
I’m interested only in CoC puzzles, here some points about that:
Too easy/too hard:
Moderators can filter out something extremely hard or extremely easy. They cannot accurately predict if 80% of players would be able to solve some problem in a specific time. Are you nuts? You need a team of statists to do it half-accurately. Moderators are biased, they are already good at the game, they already have a concept of ‘perfect complexity’ in their head. Also, rules will change, the player base will change.
It’s a job for a bot. Add a ‘complexity’ property to puzzles, adjust it after each clash based on time spent by players to solve it divided by their rank. Or use some other formula, I don’t care.
The point is, you can’t trust the top 1% to know if 99% would solve a puzzle or not. You need actual data.
Language-specific puzzles:
People usually main one programming language. And it’s pretty frustrating if the puzzle is significantly easier for other languages than yours.
Of course, the world is unfair and if you’ve chosen java for golf you’ve doomed yourself.
But sometimes a small change can make a puzzle a lot more balanced.
Example: changing output format from true/false or True/False to anything else.
And sometimes puzzles feel like they are designed with a particular language in mind.
Example: output format is precisely what JSON.stringify returns.
Yea, it’s cool that you are rewarded for learning another language that is strong when your main is weak. But a majority of players just trying to be good enough in one language.
I suggest adding a rule that forbids heavy favoring of any one language.
And adding recommendations for making a contribution more balanced.
Complexity of statement:
It’s not clear at all if the statement should be as short as possible or just not very long.
Apparently, it should be short but also if you want to include a theme you are allowed but you can’t use flavor text? What?
I think that the ultimate goal of the statement is to be as understandable as possible and not confusing enough to take more than 3 minutes. We should be able to use flavor text / story / whatever we want if it makes the statement easier to understand.
That’s kinda vague and subjective, yes. But it’s a lot better than we have now.
You can actually argue if something is not easy to understand under 3mins and other commenters can practically confirm/reject your point. It’s harder to argue if a statement is “as clear as possible” since it’s not tied to the actual human experience.
The point is that when it comes to the statement we need to focus on the player experience and not some theoretical “best description of the problem.”
That means heavy visualization, less abstraction, always choosing simpler wording, straight-up rejecting puzzles that are too wordy, more examples, and less notation-heavy definition.
Also, a rule for putting formulas/definitions in the statement and not forcing people to google it would be nice.
A subgenres:
I’ve already kinda mentioned the “fastest googler” and “fastest reader” mods. There are also classical “fastest typer”, “guess the formula reverse”, “do ASCII art”, “easy solution complex description troll” and more.
I think that almost all of the subgenres should be presented in the pull but we should have different guidelines, or at least specific recommendations, for each of them.
For example, I can’t tolerate output rounding in the “guess the formula reverse” because it makes figuring out the formula way harder for me. But I don’t know if it’s just me.
And I think that “do ASCII art” should be more than just “see that example art? Remake it but with height and width of N”.
And I think that the “fastest googler” mod shouldn’t exist.
). Despite our disagreements, I don’t believe any moderation can work by force, and I certainly don’t want to exclude people who think differently.