Why don`t you let the users vote?

Everyone who is uploading a map have to judge the other maps (of course not his own)

And the Judge points can be easier to give, like:

Speed
Tech
Creativity
Theme
Difficulty

1Point =bad
10 Points=awesome

Maybe one person has to release the results.

What do you think of this idea?

Asides from internal competitions in tracking teams, I don't think that will work. Mainly because someone might bomb the results of everyone else.

that does not work. just look what happened when players could choose the ESWC maps. And there everybody could only vote with + or -. The last days everyone voted all other maps down..

In the past, one of the criteria was number of awards. This, however, tended to favor authors who were part of a team or "crew", who could rally huge support for their tracks.

I honestly think that a criteria based judging system with limits on how many points can be given in any specific category is good. I suggest everyone tries to be a judge for one month and tries to show how they think the judging should be done - the current system allows a lot of flexibility.

Long post alert

Perhaps debating the judging system should deserve its own thread/blog so that everybody sees what's going on. That said, since the discussion's already in full motion here, let me add my thoughts on the matter:

To begin with, a major reason for me, and possibly several others, to enter the MTCs are the judges. Sure, the restrictions are what make the contest thrilling; and there's the excitement of competition in there as well. But the great souls who donate their free time to drive, notice and deliver matter-of-fact analyses on our tracks are the core of what mania-exchange is all about. Therefore, I think it’s very said that we cannot keep this system. However, in-depth judging takes up too much time and effort, and with an ever-decreasing amount of willing potential judges this system is obviously growing unstable. We have to find some way to automate the judging system in order to keep the contests going.

As has already been stated, a scoring system (be it +/- or 1-10 points) is very vulnerable to the egos of trackmakers, who may decide to push down on the competition in order to get an advantage of their own. I have another suggestion: Let every contestant rank the other tracks.

Say that ten tracks have entered the contest. Thus, every contestant ranks the others tracks from 1 (favourite) to 9 (least favourite). No scores, no pluses or minuses, probably not even comments*. Just a list. The ranks are then collected by the MTC host, who sums them up and creates a final scoreboard where the winner can be announced.

As Alcator said, we should not force previous winners into judging the MTCs. However, I think it is only fair to be able to expect something from the current contestants. If we therefore make it a rule of entering that you submit your list before a certain deadline, this new system ought not to create that big an impact on the amount of submissions. Contestants that fail to send in their lists would be disqualified – perhaps with the exception of slight delays, which the host should be made aware of well before the deadline.

The judging spreadsheet should be easily manufactured in a user-friendly fashion in Excel, as should the (mostly automatic) tallying sheet for the host.

As with any systems, my suggestion has its weaknesses. The first and most obvious is the risk that the contest turns into a so-called “popularity contest”, where friends rank each other highly, thus turning the competition into a question of who has the most friends. It is my opinion, however, that this weakness will be found within every alternative system we can come up with (it has even been said about our current system, where the judges do their best to submit what they think are fair scores). And I think ranking fares better than scoring in minimizing the impact of the weakness – after all, there’s only one “point” in between ranks 1 and 2.

Another risk is “inverse ranking” – that is, a highly competitive contestant might rank the tracks he consider best, in other words his “greatest competitors”, the lowest in order to keep them down. One way to manage this would be to reinstate the system where the highest and lowest ranks of each track are excluded to try and eliminate biased opinions.

The weakness I find the greatest is the demeaning of the theme. The theme offers restrictions or goals which are to stimulate the contestant’s creativity, and to an extent innovate the overall trackbuilding norms in Trackmania. However, in the current system there is a theme score for a reason: in order to give the theme its deserved weight in the contest. If everyone is to judge the tracks, be it with scores or ranks, there is a risk that the theme will be overlooked in full favour of the driving experience. I am somewhat clueless as to what to do (within fair measures) to keep the theme’s deserved importance in the contest. (Any ideas?)

But if we collectively do find a way, I think this system might well be a worthy heir to the contest.



*- At least not obligatory comments, as that would alienate newcomers or those with not enough free time. Let’s face it – they played a major part in getting us here to begin with.

The ranking system would be possible, however what happens if majority of participants forgets to submit their rankings? Do we really want to disqualify half the contestants?

Plus, there's always the problem that this requires everyone to
(a) play all the tracks
(b) keep their RELATIVE quality in mind when filling in the ranking.

Personally, I'm not able to say whether track A is better than track B; what I am able is to select, on a scale from MIN to MAX, how much a particular track fulfill or exceeds my criteria, but only if I can use the same value multiple times; if I would have to order the tracks from best to last, then I'd have a big problem.

Thanks for the informative post, much appreciated!

Other idea: judges just have to rank the tracks, not give a rank to the tracks. They would have to make a ranking for fun, theme or whatever criteria there are. Lemme explain with an example:

Fun:
1)track1
2)track2
3)track3

Theme:
1)track2
2)track3
3)track1

Then points could be given based on the track's rankings in each specific category, by each specific judge. Doesn't sound too bad in my head And then we could still have a seperate, non-obligatory table in which the judges can explain their choices.

but if one track is e.g. extremely fun, and others are extremely not, that one track would earn only 1 point more...

True...

marti says:
Other idea: judges just have to rank the tracks, not give a rank to the tracks. They would have to make a ranking for fun, theme or whatever criteria there are. Lemme explain with an example:

Fun:
1)track1
2)track2
3)track3

Theme:
1)track2
2)track3
3)track1

Then points could be given based on the track's rankings in each specific category, by each specific judge. Doesn't sound too bad in my head And then we could still have a seperate, non-obligatory table in which the judges can explain their choices.

Still the problem of keeping track of how much fun/design/speed/whatever each track had RELATIVE to all others. That's a tall order, I'm afraid.