Revision (current)
Last Updated by Samsara on 11/13/2022 12:20 PM
----
%%TOC%%
----
Judging is the part of the publication process where the submission is looked at and it is evaluated whether or not it follows the [Movie Rules], what [Class] the movie falls under, and whether it meets the qualifications for that class. This process is performed by Reviewers, Judges, Senior Judges, and sometimes Admins.
In the reviewing process, a Reviewer performs basic research on a submission. They verify that it syncs in the emulator, check for optimization, and provide feedback on their findings in the submission thread. Any number of Reviewers can review a submission. A Judge will look over and use these reviews in their own judgement. It is not necessary for a submission to be reviewed before it is judged, however reviews help simplify and streamline the process, allowing submissions to be processed both faster and more thoroughly.
In the judging process, a Judge first claims the submission for judgment to avoid having other judges look at the same movie redundantly. They then do any necessary research, including but not limited to watching the movie, potentially testing the game themselves, and asking for feedback. If improvements pop up, it must be decided whether or not it should either be resubmitted or whether it is acceptable to replace the submission file with the improvement. Once any necessary research is performed and the conversation and improvements have settled down, the judge makes a decision, with a judgment note describing the reasoning behind the decision.
If you are interested in becoming a Judge or a Reviewer, contact a Senior Judge:
%%%[module:WikiUsers|role=Senior Judge]
!! Reviewers
Reviewers make up a trusted panel of experienced members that analyze and provide feedback on submissions when necessary. They operate similarly to Judges, and are expected to have similar qualifications in turn. While they are not directly able to process submissions, they are able to [Reviews|review them on behalf of Judges], and Judges can subsequently sign off on those reviews as official judgements. Reviewers are also first in line to become Judges: The role is used to train and advise potential Judge candidates as a way of getting them adjusted to the position at their own pace.
When reviewing submissions, Reviewers are expected to follow the [Judge guidelines].
! Qualifications
* The ability to verify input files.
** Knowledge of emulators and their usage is key to this position. The biggest unbreakable rule we have is that an input file must sync from beginning to end for multiple users.
* A keen eye for optimization, or high-level game knowledge.
** High-level TAS experience is not necessarily needed, though a Reviewer should be comfortable being able to spot mistakes in submissions.
* Being a kind, caring, and active member and representative of the community.
** Judging is often the first thing a new member experiences. We do not want to scare TASers off with harsh judgements. Above all else, respect the TASers and their work. Encourage them, give them constructive advice, help their creativity flourish, and let them know they're welcome here.
** An active line of communication with the judging team is highly encouraged. Joining our [Discord server] is ideal, as it's the easiest way of everyone being able to stay in contact with each other.
* A good understanding of the [movie rules], [guidelines], and approaches that are applied to the movies being judged.
** While Reviewers aren't expected to handle extreme cases, some submissions that look trivial may end up being a lot more difficult than expected. A good understanding of the rules will help
** Always keep in mind that rules can change, and that they should not always be blindly enforced.
!! Judges
Judges are members of the site experienced with the process of creating tool-assisted speedruns that have been entrusted with the responsibility of approving or rejecting movies that have been submitted. Judges are the people who decide which movie content ends up being present on the site.
Judges follow the [Judge Guidelines].
! Qualifications
Judges need:
* Experience with __high-level TASing__ that involves diversity of skills and games and keen understanding of optimization.
** This helps with checking for obvious improvements and providing helpful insight on them. TASing experience allows a Judge to distinguish between suboptimal and optimal play without demanding unreasonable efforts on behalf of the authors.
* Being a nice, active, and helpful __member of the community__.
** Judges should be prepared to work with not just the rest of the staff team, but the whole TASVideos community at large. Judges should be able to communicate with people in a friendly manner, ask and answer questions, clarify anything people are confused about, and collect opinions.
** TASers make our site exist, so our main goal is respecting their work and encouraging them to do more cool things. Judges should be able to understand what TASers' best interests are and how to satisfy their needs with staff activity.
** Joining our [Discord server] is a requirement to be able to participate in community and staff discussions.
* Being ready to set up and run __emulators__ of variety of systems.
** As the time goes by, more and more systems become TASable. It is important to know how to deal with their emulators, because more and more submissions of their games hit the queue.
** Some emulators have not been designed with user friendliness in mind, and some effort is required to replay movies on them. We do not want such exotic emulators to be neglected by judges.
* Proper understanding of all the __rules, guidelines, and approaches__ that are applied to the movies being judged.
** Nothing in the rules should feel unexpected, unclear, unreasonable to a judge. Judging is not a lottery, it is a trial that should be reproducible regardless of participants. It should only depend on the initial conditions and how those are handled by the rules.
** Sometimes we have to deal with cases that uncover flaws or lack of clarity in the rules themselves. Instead of blindly enforcing these rules, consider how they could change to better reflect what the community wants, and what we can provide to them in turn.
!! Senior Judges
The Senior Judge(s) are extremely experienced and involved judges that act as leads. They are expected to be the community leads on [Subs-List|submission] processing and the rules behind them.
The Senior Judge(s) take on additional roles alongside judging:
* Making decisions on modifications of the [movie rules].
** Collecting and considering all feedback towards the rules to come to a conclusion on how to best implement this feedback onto the site, so that as many people as possible are confident with the results.
* Appointing and training new Judges and Reviewers, as well as setting standards and examples for how they should act.
** Making sure the current judging and reviewing teams are running smoothly by being around to offer help with difficult submissions.
!! Related Links
* [MovieRules]
* [JudgeGuidelines]
* Sub pages, see below: