Posted: 1/23/2009 9:26 PM
Post subject: A judging rubric
Quote
mmbossman
Experienced player (829)
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
GBx TAS of 2008
Recently, there has been discussion regarding the entertainment and technical merit of submissions, and whether each requirement should be relaxed in order to increase game diversity and to encourage newbie retention (respectively). In an attempt to better myself as a judge, and to provide some clarification on the thought process that I use when judging, I have created a rubric to help better define my opinions. It is located at my homepage http://tasvideos.org/Mmbossman.html under "Judging Rubric". I would like to hear opinions on the ideas located within. Also, please don't turn this thread into one which argues the validity of the rating system (including whether or not you can measure technical achievement). I've already voiced my opinions on those topics elsewhere, and I may be prompted to expand on them further if needed, but I'd rather stick to current content for now. So, what do people think?
Living Well Is The Best Revenge My Personal Page
Posted: 1/24/2009 11:12 AM
Quote
LagDotCom
Joined: 3/7/2006
Posts: 720
Location: UK
I think you are awesome for using the word 'rubric'. Aside from that, it all looks nice but the criteria for categorisation into Low/Medium/High is suspiciously absent. If it's just a gut feeling then, okay, I'm glad we have more than one judge :p
Voted NO for NO reason
Posted: 1/24/2009 1:00 PM
Quote
mmbossman
Experienced player (829)
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
GBx TAS of 2008
LagDotCom wrote:
but the criteria for categorisation into Low/Medium/High is suspiciously absent.
You're right. I've tried to define both entertainment and technical qualities on the first tab, but I'll work on adding specific criteria to each box. Thanks for the suggestion Lag.
Living Well Is The Best Revenge My Personal Page
Posted: 1/24/2009 5:38 PM
Quote
Derakon
Joined: 7/2/2007
Posts: 3960
Whatever you use, I don't think it should be based on the ratings people are providing. For example, "low technical quality" should include things like not hitting menu options on the first possible frame, obvious losses of speed (e.g. running into obstacles or not jumping before a fall), unnecessarily taking damage, etc. Not having a tech score in the 1-5 range.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Posted: 1/24/2009 5:44 PM
Quote
mmbossman
Experienced player (829)
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
GBx TAS of 2008
Yeah, I addressed that below the table, but I can see how you missed it: "This rubric also tends to correlate to how I rate movies. However, it will likely not correlate to the scores present in each submission thread, as many people have shown that they have no use for the numbers below 9.5. " The ratings in the submission do help me get a good idea of what type of run it is before I watch it. All high scores, all low scores, and a lot of spread out scores each gives me an indication of what I should look for in the movie, but I certainly don't use the ratings to make a final decision.
Living Well Is The Best Revenge My Personal Page