Today, whenever moderators want to validate a contribution, they must check that the contribution verifies some criteria. Criteria that the creators are not even aware of. They can find it in the documentation but it’s scattered among several pages.
To enhance the contribution process (both for creation and moderation), I’m proposing a new version of guidelines. You can find them here. Edit: Guidelines are up to date now
As you can see, there are several categories with a few guidelines in each: statement, test cases, referee, etc… The idea would be to only show these categories in the validation popup with a link to the documentation page for reference. Also, we would like to add this to the pop-up when contributors publish their contributions.
What do you think? Is what is expected from creators and contributors clearer?
Is there something missing? Do you agree with everything?
The documentation is a GitHub repo, so don’t hesitate to contribute.
“Bad” contributions could still get through by moderators with the wrong intentions. If that happens and the contribution is really broken, let me know about it in the forum and I’ll take care of it. If the contribution is not good enough, it will most likely get rejected by the bot we’re putting in place. After enough votes, if the ratings of the contribution are too low, the bot will reject it. We’re still defining the rating limits.
We’ve discussed internally how we should handle difficulty. Especially for CoC. Clash of Code by nature is a speed game. Most CoC players are looking for a very short break. So we decided that easy CoC puzzles should not be rejected on that ground.
We already have a lot of puzzles and games so every new contribution should be original enough. For Clash of Code, it’s a bit different as it’s interesting for players to expand the pool of different puzzles. Contributions close to current contributions (same programming problem with different description) should be allowed.
Haven’t looked back at them since initial publication, but if I recall my first impressions:
mostly good, a pleasant surprise
slightly bit too heavy on the “must” in places where they really ought to be more guidelines.
a bit haphazard a general outline/organization. IIRC test/validators were mentioned in a technical page while they felt like they belonged more in statement writing, for example.
CoC players don’t expect to learn something from solving a CoC puzzle
This literally made me sad. What is the point of doing this then? If you don’t want to learn new stuff, go take an engineering job where you can do the same thing for 20 years. There are thousands of possibilities in each country.
Also I’m not really sure if the players don’t expect to learn something (I, for one, expect to learn new things even in CoC) or they are not expected to. Probably it sound a little better with anything instead of something.
There is a small typo in section 4, item 3: “correspond” -> “corresponding”.
For test cases/input/output you can add, that variables should have reasonable constaints, and the creator must specify those constraints. A lot of times a task becomes hard because of the constraints. Also, you might think about adding something about specifying all input parameters (and their types) for completeness. I don’t think it is very important, though.
I have some thoughts on how to ‘distribute’ the guideline. You say that the creator does not neccessarily know about the guideline. You can dissect this guideline to match each type of contribution, and when clicking on the new contribution button, after choosing a type, throw a blank page with some colorful button “Hey, have you read the contribution about Optimazation games?”. Obviously, not this, but you get the idea. From this the user can click to go and create stuff. It is a very little price to pay. You can even store if the user had contribution since the new guideline for the type and let him go to the creation page automatically from the second contribution. One more thing that can enchance the user experience is having little tooltips for important elements of the contribution. For tests, validators, inputs, stubs have a short list of good-to-know stuff from the guidelines. As far as I have seen the web page, I think you have the components for these, so probably the worst part is to actually phrase everything and copy them in the right place.
We have released the post-moderation bot which will reject contributions which have too low ratings. Let me know if you disagree with one of its decisions.
Thank you for the feedback on the guidelines. I’ve finally published them officially. I’ll continue to include changes according to feedback; they are not set in stone. This month, I’ll make sure that the platform reflects these changes. Like you wrote @ThePrimeEvil, the idea is that creators know about these guidelines before making their contribution public.
I have a question about the post-moderation bot. If a contribution gets accepted really fast (e.g. shadow accounts), then it will disappear from the list immediately. Wouldn’t it make sense to let the already accepted contribution to be visible for, say, another day after being accepted?
The post-moderation bot makes no sense to me.
There was a puzzle where I recently saw it turned off “shortest mode” because of the ratings. What makes shortest mode special in any way? There is no logic behind that at all.
Turning off one mode and leaving the others is not “rejecting”.
Plus, the issue I see with that puzzle is a relatively minor one, not warranting taking it off at all.
In the meantime we have cases like this:
There is a completely nuts validator test case not reflected by a corresponding IDE test case, and people pointed it out in comments and still no one takes any action.
I could go and fix the test case myself of course, but then no one actually learns they made a snafu and no one is prevented from doing it again. It solves nothing. The only real solution is to improve the requirements for approving puzzles, and ban the people that repeatedly approve bad puzzles from doing so in the future.
I’m not sure what issue you’re trying to tackle with this idea. So in case a contribution gets accepted too fast by shadow accounts, you would expect moderators to edit the accepted contribution and improve it? It’s possible today even if the contribution is not visible anymore (you get the link with the notification). Besides, this is a case that rarely happens and that we can take care of case by case.
@TwoSteps Let me illustrate it with an example. I submit a contribution. As soon as it is out, in my 3 browsers where I logged in with my shadow accounts, I accept it. It was visible for users for like 5 seconds. No one saw it. No one had a chance to comment on it. Or downvote it, so the bot can reject it. It is accepted with 3 upvotes. Do you (personally you/dev team/moderator team/high level players) get any kind of notification that there was a new contribution that was accepted? Do you know that someone looks through already accepted contributions from time to time and makes the effort to delete these by hand?
I didn’t mean to be rude or offensive, I just wanted to point out some perspective, and these are the kind of feedback I can profit the most during my daytime work, and I was taught to ask questions like these. Since I sense that my comments cause more misunderstanding and problems then they solve, I will try to limit them.
I explained in my first post, but I’ll repeat. The people that approved it will not learn from their mistake. You say that the creator will get pinged, but it’s really not the creator’s fault. That’s what the approval process is for, to spot and correct errors by the creator. It is actually the people that approve that should take responsibility for the puzzle being good. And those that repeatedly just approve for the points without really paying attention to the puzzles should be stopped from doing so.
… all ratings are quite good so it seems the validator issue is not impacting the players too much.
Or the puzzle is too old and gets picked too rarely (newer puzzles seem to have much more weight in the distribution) and also the floating point error probably affects only some languages and your ratings stats aren’t split by language so you don’t notice the problem.
@ThePrimeEvil you weren’t rude or offensive or anything, don’t restrain from commenting I’m sorry if my comment made you feel this way.
Every moderator (that didn’t deactivate the notifications) is notified when a contribution is accepted. This kind of abusive behavior is usually reported to me so I can take action on it. Also, the bot we implemented will soon reject it. By ratings, I meant the ones players vote on after playing the CoC or puzzle, not the upvotes/downvotes of the contribution itself.
@Visual I see. It could be nice to ping the approvers to inform them a contribution they approved was rejected.
However, the goal of the rejecting bot is really to ensure the puzzles and COC available to the community are enjoyable to solve, rather than tracking moderators who approved a contribution which eventually got rejected.
I trust the moderators to do their best considering the tools they have and to help each other for the thrive of the whole community. Besides I’m remaining vigilant of any abuse, with their help.
With regards to the CoC guidelines, I think submissions should be rejectable based on difficulty, not only if they’re too hard, but also if they’re far too easy (refer to CoC Guideline 3).
I fully understand different programmers have different skill levels, and that simpler clashes make it more approachable for newer programmers. I totally understand and support approachable problems so it can be accessible to a wider variety of skill levels. I also understand people seeking more complex challenges can tackle practice or non-clash related challenges under the “compete” banner. I still think clash problems should have a minimum difficulty level, and should not include problems that absolute beginners would find too easy.
A CoC being far too easy, in my opinion, should be an absolutely valid reason to reject a clash since it can be very annoying and unenjoyable to wait for 2 mins in queue, load into a clash which can be read, solved, and implemented in under 20 seconds even by beginners, then have to wait for another queue if you want to go again. It would be especially frsutrating if you get these kinds of problems in succession, making the clasher spend way more time in the lobby screen than actually clashing. It also makes the problem significantly less satisfying than those which have that “I figured it out” feeling when you solve the puzzle, and instead gives more of an “that’s it?” kind of feeling. Those kinds of puzzles are un-fun, and while user reviews and rejecting bot may get those clashes out of rotation later on, i think moderators should be able to take over-simplicity into consideration beforehand.
Very fast clashes are where I’ve had the most fun in the clashes’ lifespan on CG. Everybody’s got their chance to win it, a battle of wits to understand the statement the fastest, possibly to distinguish it from lookalikes.
Conversely, “harder” clashes are quite often a lame excuse for “boring”. As in: a succession of those can be either frustrating or a push to copy-paste, which is a can of worms of its own.
Speed is the point of clashes; for harder less repeatable stuff there are plenty of untimed problems available.
Well as I am also doing clashes on a regular basis, I find it pretty hard to say something is “too easy”, as you can’t really distinguish on a first look if a problem is actually easy or not.
I mean… this one here (https://www.codingame.com/contribute/view/4508618d24711919402a7811f21247d27f1d) looks like its a protest against the “there is no too easy” thing mentioned here , and most people will agree that this is actually an example of an “too easy” clash. On the other side this is actually a really extreme example.
I encountered clashes which where really easy on first look, but actually were pretty tough in the 15 minutes given, also the other way around, a problem looked hard on first glance, but was a one liner afterwards.
The Quote: “CoC players don’t expect to learn something from solving a CoC puzzle”, is also actually true. You usually don’t learn something by solving a clash problem ( you usually do this with the puzzles ), but you can actually learn new tricks from the code of other players. Sadly not all players show their code to the others after winning.
So… I kinda lost track here of what matters, but all in all I think I have to agree with the “there are no too easy clashes” thing, as difficulty is still subjective, even somtimes language specific.
@TwoSteps
That’s something else I’d love to be able to turn on in my settings, or thats on by default, where it shares automatically instead of having to manually choose to share my code because i’m happy to do it but sometimes i just want to go to the next one so i forget to.
I completely agree with where you’re coming from, however with the example you gave its easy all around.
Obvious problem, effortless solution. There needs to be a healthy balance to make a clash involve implementing code to create a solution to a problem, even if it can be simplified down to very short code in some cases, and not a speed-reading competition.
I completely get where you’re coming from, it can be frustrating when you have to spend 15 mins solving a problem only to get 1/2 the test cases because its either incredibly complicated or has a very specific solution, and those are no fun either. However, to your point of having to understand the statement fastest, with clashes like the one Karamuto linked, there is no “understanding the problem”. Its a simple incrementation, nothing more. It can be solved with a mere n++ or n+=n or n=n+1 in most languages. It is a painfully easy problem which requires no real programming skills in any shape or form, which goes against the idea of clash.
Clash, in my opinion, is supposed to be a competitive environment to:
Understand a problem
Find a solution
Implement a solution
and if a problem makes it too easy or too hard to do any of those then it is a bad clash.
I can understand feeling like a God when you solve a clash in under a minute, but to have overly easy ones, such as the one listed by Karamuto, don’t grant that satisfaction and instead are just a disappointing way to have spent the last 2 minutes in queue.
I’m not sure you do. I don’t recall the situation you describe ever happening to me.
What did happen to me is get some tedious clash once. Take the time to solve it 100%, edge cases and all, in less than 15 minutes unless otherwise specified. Regardless of the clash outcome, not feel particularly good at the end. Then get that same clash again later the same day, and contemplate ragequitting and wasting your CoC TrueSkill rating because you don’t want to go through that again.
It seems to me your problem is with the 2 minutes queue, not the clash topics themselves.
Maybe my experiences were different than yours, and because of that, I don’t understand exactly how you feel. I have played some clashes where they were pretty tedious getting the fringe cases right, but again maybe my experiences were different.
My issue personally isn’t with the queue, its the very unsatisfying feeling from getting a problem that a beginner would consider very easy. Part of that feeling lies with having feeling like my time was wasted, but most of it stems from the fact that to me, there should be a certain skill-floor to problems in clash where people should be expected to have a certain skill level when competing and having overly simple clashes takes that away from people.
The challenge here is to know where to set the cursor for “too easy” and “too hard”. Since it depends on every programmer and the languages they’re using, it’s probably impossible.
By not having a “too easy” limit, I expect the community (moderators and creators alike) to agree on the best decisions keeping in mind the CoC players’ experience and that, as JBM (and I) wrote, CoC is mostly about speed.
Now, @Boulet , as you were trying to make a point with your contribution, I’d like to apologize for the issues with the moderation of contributions that have been happening for a while.
I understand the frustration you can feel as an experienced player who spends a lot of time and effort to improve the overall quality of the platform when you see others not making the same efforts and getting the same reward. But be sure that I see your work, the whole team sees your work and it benefits thousands of players in the community every day.
I’m doing my best to improve the way things are and have devs working on new features or bugs. I still have a lot to learn though. I know also that, with first the focus on Tech.io, then CG for Work, the improvements have been sparsely coming in. Now, we’re still a young company, and without the focus we put on our business, the platform wouldn’t be up right now.
I’ll continue and double my efforts to follow issues with moderation specifically and requests updates from my team. But I’ll also need your help. I need all of you who want to continue contributing to the best coding platform ever to work with me. Together we can discuss and find solutions with the best interest of all CodinGamers in mind.
So what do you think? Should we set a “too easy” limit? Update the guidelines to be more specific?
I agree that it won’t be easy to judge too easy, however clashes are being judged right now and being declined for being too difficult, something which can also be very subjective depending on proficiency and language as you’ve said.
From what i’ve seen so far, our mods have been doing a great job knowing what’s too hard for the community and what makes for an unenjoyable experience for all users; and feel they would bring the same level of judgement for puzzles that are too easy.
I’m not sure how it could be defined, but I believe there should be some sort of too easy limit when it comes to clashes. I believe the moderators and community will be able to distinguish puzzles which are easy, straightforward, simple, or approachable (all of which can make for fun clashes) from those which are too easy, or overly-simple which can make the experience of a clash feel unrewarding or unenjoyable.
I’m sorry if my comment made you feel this way.