Maybe the difficulty rating could be asked for anyone who accesses the clash. And the author’s solution would only be revealed after the rating is provided.
@miracle It would be indeed interested to have moderators solve contributions before being able to vote. I see some issues with it though:
- as @Boulet wrote, the contribution solution is visible. Maybe we could remove it though.
- nothing prevents anyone to think about the contribution and then solve it, so the measurement of the time needed to solve it could be inaccurate
- it would be much more easy to reject contributions than to accept them. Maybe it’s not really an issue.
Guideline 2. Indeed, the description guideline doesn’t apply to reverse mode.
Guideline 4. Having a meaningful name for a test can help players understand what the test is about. Maybe it does not, but still I think it’s a good practice. Probably it helps reviewers to understand the different cases that are checked too.
Guideline 5. The more CoCs we have, the better for players. Why should we allow close duplicates (same or really close solution but different statement)?
- Because of the modes and the limited amount of COCs puzzles, any player will eventually play again the same Clash, which is a shame. Allowing close duplicates of existing puzzles could be less frustrating for a player to solve them than solving again the same ones.
- Having more CoCs also reduces the frustration of new players encountering those who play a lot and know all CoCs by heart.
- Creators don’t always have access to all the Clash pool and don’t know a similar Clash already existed. Feels bad to reject their contribution because of it.
Of course, we want to avoid having 15 close duplicates of the same Clash. Maybe we could put a limit to 3.
Yes, players could “farm” XP by creating close duplicates of existing CoCs. I don’t see this happening so much though. (most of the times, they don’t know their CoC already exist). We could also think about changing the amount of XP each contribution rewards.
Specific CoC guidelines: CoC is speed game. The light difficulty of CoC puzzles is actually the core of CoC.
If you don’t find it fun to solve easy puzzles, just don’t play or moderate CoC. I understand that really good developers like you would also enjoy a speed game with challenging puzzles but that simply doesn’t work with the current CoC setup (no difficulty level or leagues) and players.
What do you think? Does this help to understand our philosophy behind Clash of Code?
I’ve added a few details to the guidelines following our discussions.
Am I wrong if I write you just 
1- removed the last guideline No CoC puzzle is too easy. (the restriction at the end is so unclear that one can always argue " overly-simple puzzles"
2- put a maximum duplicate of 5 which IMHO is just worst than allowing duplicates since no one will agree on almost duplicate things. Let’s take an example : are Add0 et LeftConcatenateEmptyString duplicate ? One can say yes since the code to solve both are the same. One can say no since the problem are clearly not the same.
Following discussions in the chat with @Stilgart, I’ve updated the guidelines to set the guideline “test cases must have explicit names” as just a best practice. I’ve also mentioned in the “Defining Test Cases” page that sometimes, to cover edge cases, it’s necessary to have the same validator as the test
I didn’t answer @Boulet’s comments earlier because I didn’t know what to do next. (I’ve also been busy with other things). The thing is that I’m not sure we will ever reach an agreement on the difficulty and duplicate guidelines. I see two options right now (which I don’t like at all):
-
Enforce the rule and remove moderator rights to everyone who don’t respect it (I would probably have to ban you from CodinGame if I want to avoid another “Add0” episode
). Despite our disagreements, I don’t believe any moderation can work by force, and I certainly don’t want to exclude people who think differently. - Accept the status quo and live with a double-standard moderation system. I don’t think the current situation makes for a welcoming community when it comes to contributing to the platform (and I know we, CodinGame, share a great deal of responsibility in this situation). Even if CoC is the ugly duckling for many people in the core community, a lot of new players still enjoy playing it so I’d like to ensure they continue enjoying it.
So I’ll brainstorm internally and do some research (perhaps also a survey on CoC players) to find a new option everyone could agree on. Any constructive idea is welcomed.
I hope this is the right place to leave some thoughts from the player’s side (as I don’t find any CoC specific feedback topic). Just fyi, I also read contribution guidelines as well as this entire topic including links here and current pending contributions.
TL;DR there are too many hard/long/confusing clashes and it’s not what you expecting
I solve about 50 clashes in the last few weeks on my way to complete corresponding achievement and I would say at least 30% of them don’t match the letter or the spirit of the guidelines. There was some with overly artistic descriptions, descriptions uses kinda specific terminology (which may be hard to understand to non native english speaker), puzzles that takes 10-12 minutes minimum to solve (not only by me but also other clashers). Can you imagine reverse CoC which take 5 sec to understand what to do and not have time to implement? Despite I’d always try to leave feedback on such clashes with proper explaining, I hardly believe them being read and/or ever taken some actions on it.
Now I read contributions and see similar things. Spending 2 minutes to read and understand problem is not what CoC are supposed to be, IMO.
https://www.codingame.com/contribute/view/46347cb7d31c4113ef5401082e63e9fad28b (reverse?)
https://www.codingame.com/contribute/view/4642c15e5bd89278e05566d571a3add83725 (reverse mode seriously?)
The most violated contribution rules in my opinion:
- Keep it clear and consise
- Avoid flavour text
- CoC puzzles should be solvable in less than 5 minutes.
CoC homepage offers 5 min clashes which is usally false. Speed is an essence of CoC.
About 10-20% of my clashes was solved by 15-45 seconds and know what? They was the best I ever solved and I never was dissapointed or bored.
420 clashes ahead, so help me God.
2 Likes
This guideline is so subjective and thus unusable that I can hardly seen any interest. As already written by myself
Is this a reason to reject it because it is too hard ?
Can you explain us where is the interest of fastest typing CoC. Not a fastest finding but typing. In 15-45 seconds you just have time to type 20 chars and no time to read the statements ? CoC are supposed to be programming challenges not pavlov’s reflex challenges
There is at least one viable strategy already to determine puzzle complexity:
Regarding fastest-typing-challenges… with my non-blind 4-fingers typing I’m able to write ~3 LOC for 30sec if I know what to do. I’m pretty sure that 30-45 sec is enough to understand easy puzzle and write 1-2 LOC of simple math using default code. Writing 20 chars in 30sec is extremely slow, isn’t it?
Not a real interest to write code fast, but faster than others.
Contributions must be original - I want to create a puzzle and I wonder if similar or even the same already exists.
Is there any way to check it, except going through all of the puzzles and read them?
1 Like
You can ask some moderators, in the chat for example.
1 Like
Agreed, when I see 5 min., I do expect that I will need more than 1 min. of effort to solve a CoC (which is usually not the case).
That being said, speed is especially why I personnaly hate CoC.
I’d rather have two leagues: one with a 1min. duration, and one with a 15min. duration (which is closer to what was CoC at the beginning, with the 5/10/20 min. formats).
2 Likes
I think guidelines should be renewed since a year has passed and not much from this conversation was solved.
Has conversation moved to anywhere else? If so then why a link to this post is in the guidelines?
I’m interested only in CoC puzzles, here some points about that:
- Too easy/too hard:
Moderators can filter out something extremely hard or extremely easy. They cannot accurately predict if 80% of players would be able to solve some problem in a specific time. Are you nuts? You need a team of statists to do it half-accurately. Moderators are biased, they are already good at the game, they already have a concept of ‘perfect complexity’ in their head. Also, rules will change, the player base will change.
It’s a job for a bot. Add a ‘complexity’ property to puzzles, adjust it after each clash based on time spent by players to solve it divided by their rank. Or use some other formula, I don’t care.
The point is, you can’t trust the top 1% to know if 99% would solve a puzzle or not. You need actual data.
- Language-specific puzzles:
People usually main one programming language. And it’s pretty frustrating if the puzzle is significantly easier for other languages than yours.
Of course, the world is unfair and if you’ve chosen java for golf you’ve doomed yourself.
But sometimes a small change can make a puzzle a lot more balanced.
Example: changing output format from true/false or True/False to anything else.
And sometimes puzzles feel like they are designed with a particular language in mind.
Example: output format is precisely what JSON.stringify returns.
Yea, it’s cool that you are rewarded for learning another language that is strong when your main is weak. But a majority of players just trying to be good enough in one language.
I suggest adding a rule that forbids heavy favoring of any one language.
And adding recommendations for making a contribution more balanced.
- Complexity of statement:
It’s not clear at all if the statement should be as short as possible or just not very long.
Apparently, it should be short but also if you want to include a theme you are allowed but you can’t use flavor text? What?
I think that the ultimate goal of the statement is to be as understandable as possible and not confusing enough to take more than 3 minutes. We should be able to use flavor text / story / whatever we want if it makes the statement easier to understand.
That’s kinda vague and subjective, yes. But it’s a lot better than we have now.
You can actually argue if something is not easy to understand under 3mins and other commenters can practically confirm/reject your point. It’s harder to argue if a statement is “as clear as possible” since it’s not tied to the actual human experience.
The point is that when it comes to the statement we need to focus on the player experience and not some theoretical “best description of the problem.”
That means heavy visualization, less abstraction, always choosing simpler wording, straight-up rejecting puzzles that are too wordy, more examples, and less notation-heavy definition.
-
Also, a rule for putting formulas/definitions in the statement and not forcing people to google it would be nice.
-
A subgenres:
I’ve already kinda mentioned the “fastest googler” and “fastest reader” mods. There are also classical “fastest typer”, “guess the formula reverse”, “do ASCII art”, “easy solution complex description troll” and more.
I think that almost all of the subgenres should be presented in the pull but we should have different guidelines, or at least specific recommendations, for each of them.
For example, I can’t tolerate output rounding in the “guess the formula reverse” because it makes figuring out the formula way harder for me. But I don’t know if it’s just me.
And I think that “do ASCII art” should be more than just “see that example art? Remake it but with height and width of N”.
And I think that the “fastest googler” mod shouldn’t exist.
I have to disagree with you on this point based on my experience as an Agile developer. In the Agile method, a task difficulty is evaluated by all team members within a few minutes. The more experienced the developers, the more accurate the evaluation because this evaluation is based on how they perform a similar task previously.
A bot is nothing more than a projection of someone’s biases and assumptions and, therefore, will not offer any more guarantees.
You’re talking about work. You don’t have 10000 people in your team at work. You don’t estimate alone, you have a dedicated meeting for that. There are a system in place to help you estimate. It’s probably tracked how much time programmers are spending on a task and in the case of over- /under- estimation don’t you have meeting to figure out what went wrong?
The bot will at least provide agency.
And in cases when a lobby of top-players are all having 0% after 15minutes you don’t need bias to figure out that puzzle is too hard.
Also, another point to my previous post:
It would be nice if people that approved/refused/commented on a contribution wouldn’t be able to get it as the random puzzle in the game of clash.
There is a moderation bot in place (for already approved contributions).
Each clash can be rated after the end. This happens for all modes (fastest, shortest, reverse) separately. When a minimum amount of votes is reached and the average vote turns out to be a negative feedback, the clash will be removed (at least in that one mode).
See the comment section of this contribution for instance. The CodinBot deactivated one mode.
1 Like
Technically if you follow the rules, you should never approve a puzzle you haven’t solved yourself. So we’re talking more about an extrapolation than a estimation.
This rule is currently optional however and that’s more of a problem imo. The author’s solution should be hidden until the puzzle is fully approved, and you should have a solution passing all the tests/validators to be able to approve (CoC or not).
It’s not so formal, just of bunch of guys chatting: no stats, no fancy Excel. Through this example, I wanted to point out that expert advice is much more valuable than you would think.
1 Like
Oh yea, I forgot that evaluation has a ‘difficulty’ point.
The solution is extremely obvious then:
Bot filters some more saddle “too difficult”/“too easy” cases, moderators reject obvious “extremely difficult”/“extremely easy” cases.
So just change “no coc puzzle is too easy” to something like “CoC puzzles is meant to be easy, quick and fun. But if it’s too easy, it becomes too quick and then it becomes a game of waiting for another round to start.” And add something like “would take longer than X minutes to solve” to the approval checkboxes. Ideally, X would be calculated based on the approving moderator rank and number of games played but something like “would take longer than a minute for average 6000-8000rank player to solve” will work too.
There may be difficulties with that tho. Mostly for the reverse where you may sit for 15 minutes figuring out what the trick is and then when you get it puzzle feels like the easiest thing in existence. Of course, a moderator would be able to reflect on that and base his evaluation of difficultness on the actual time spend and not on their feelings of the puzzle. Maybe it needs to be written explicitly that you shouldn’t try to imagine how other players would approach the puzzle and instead look at your time, try to interpolate, and then include a lot of room for error.
That’s why I’m saying that we need different guidelines for different genres.
At least make some separate recommendations for Shortest, Fastest, and Reverse.
But it’s not that important tho.
What more important is that “solution time” of the problem includes reading the statement. What if the statement is complicated but as soon as you understand it the solution is clear. Is the puzzle too easy?
I’ve partly described that problem and the solution to it in point 3 in my post above. TLDR: we need to get rid of the complex wordy statements.
Thanks for darkhorse64 and Djoums for pointing out some obvious things that I’ve missed.
I do now agree that some expert advice may be more valuable. Mostly because I’ve started to look at the rank of commenters and oh boy, there is some interesting correlations there.
I do still think that it brings elitism in at least when it comes to difficulty. It’s may bring some subtle problems like
Interesting suggestion.
Yea, I agree that you should be able to approve only if you have a working solution. And that is a restriction only if the intended solution is hidden.
But there should be an option to show the solution. However, if you choose to show it you can’t approve the puzzle, your only options are to reject or to move by.
I suspect that any version of that is hard to implement and it’s easier to just trust the reviewers.
Just hiding solutions to Reverse puzzles shouldn’t breake anything tho