As of May 4th, 2022, this page is in the process of being rewritten to reflect modern site rules and guidelines. Information is subject to change at any time as long as this notice is here. To suggest changes or ask for clarification, contact Samsara.
For a copy of this page as it was prior to the above date, see JudgeGuidelines/Legacy.
"With great power comes great responsibility."
This page summarizes and details the thoughts that Judges should adhere to when watching and judging submissions, and communicating with their authors.
If you're an author intending to submit a TAS, see MovieRules and Guidelines.
Table of contents
- Summary
- Judging
- Verify the authenticity of the input file
- Act consistent with the message of the site
- Be fair
- Be thorough
- Classes and goals
- Improvements and obsoletions
- Major skip glitches
- Game completion point
- Editing the submissions
- Replacing submission files
- Improving the submission you're judging
- Procedures
- When you're done
- More reading
Summary
- Above all else, treat our community with care. Be kind and courteous to them, always respect their work, encourage them to continue, and use your knowledge and skill to help them improve.
- Always adapt to the needs of the community. Consider every angle of a problem, and work together with the rest of staff to provide the best solution.
- Collaboration and cooperation should be encouraged above all else. In cases of minor improvements found during judgement, let the author know beforehand and offer them the chance to take a second look, or the ability to collaborate with you or another community member. In cases of major improvements found during judgement, offer constructive feedback and input files to help guide the author.
- Value quality above quantity.
- Don't rush your judgements, let discussions play out. When necessary, start new discussions about the submission so users generate more feedback. Take your time and ponder everything relevant to the submission.
The site should change to fit the community. The community should never change to fit the site.
Judging
Verify the authenticity of the input file
The input file should sync through to the end on an approved emulator. This usually means an official packaged release, though some emulators like Dolphin, Citra and Ruffle are allowed to have development builds used and submitted. Ensure that all emulator settings are legitimate and that they are documented in the submission text.
All submissions must sync for someone other than the author. Ideally, they should sync for a Publisher. If you can't get the run to sync yourself, ask the author and the rest of the community for advice on how to make it sync. You should only reject a submission for sync issues, if even after research done by different people, it was impossible to get the movie to sync.
The input file must complete the game with its stated goal, i.e a "100%" submission must attain that game's definition of full completion. It should not start from reset, SaveRAM, or soft-reset unless necessary for the goal choice (such as to unlock a character, or take advantage of newgame+, for instance).
Ensure that there is no blank or unnecessary input at the end of a submission file. This artificially increases the input time and may lead to confusion or misinformation about the nature of the submission.
Act consistent with the message of the site
- Always treat submitters with the respect and care you would like to be treated with.
- Ensure movies consist of superior, optimal play.
- Judge movies in concordance to the guidelines.
- Reject movies that flagrantly break the movie rules.
The goal of TASVideos is to take games to their absolute limits in unique and interesting ways. Any submission that does so should have a reasonable shot of being accepted.
Be fair
A Judge has the greatest control over the content of this website in the long run. All judges must act towards the goal of having an encouraging and rewarding atmosphere for both the players and the audience. You must be fair towards both.
Too many acceptances can send the message that we have low, or even no standards. The audience expects and wants to see high quality submissions, so some standards should be enforced in order to maintain that. However, setting the standards too high through many rejections will turn away the players out of fear of not being good enough. A fine balance must be maintained between acceptance and rejection in order to satisfy the audience's needs of high quality content, and to satisfy the TASers' needs of being able to provide that content.
Being fair means never introducing bias into your judgements. Runs should be judged on their own merits, not on the value of the TASer or even the game. You should not be making your decisions before watching runs.
Be thorough
Ensure the movie is optimized
Judges are not expected to be familiar with every game, though they are expected to understand high-level TASing and general optimization techniques. You should be able to understand the gist of the run from watching it and reading the submission text. If there is anything that looks questionable or wrong in a movie, ask the author for clarification.
You must be reasonable when it comes to optimization. You should not be rejecting movies over some missed improvements. The optimization quality of a submission should apply to the run as a whole: If it looks good on the surface, and the movie consists of superior play which outperforms casual play and any records elsewhere, it's good enough for TASVideos. Check sites like Speedrun.com, YouTube, and Nicovideo for other records, both RTA runs and other TASes to verify this.
Three good rules of thumb for judging optimization:
- Consider how many improvements there are. Are you constantly seeing suboptimal play, or is it only one section?
- Consider how obvious an improvement is. Would a casual viewer notice it, or is it something only a Judge would see?
- Consider how large an improvement is. Does it save seconds, or just frames?
Being fair on optimization is an acquired skill. Feel free to ask for a second opinion if you're unsure, whether it's from other Judges or from the community as a whole.
Use a full range of experiences
Once again, a Judge is not expected to be familiar with every game, so don't stick to just your own experiences in any judgement. Ask others for advice, look to past judgements, and reach out to the community whenever you feel it's necessary.
However, do not discount yourself entirely. If you disagree with a judgement made in the past, talk to other Judges about it. If you disagree with a current rule, talk to the Senior Judge about possibly changing it. If you lack knowledge of a particular game, ask the community to fill you in.
Remember that there can always be change on the site, and that past precedents do not necessarily represent current methods.
Write a great judgement
As a general rule, everything you put into researching and analyzing a submission should go into your judgement notes. Not all submissions require deep analysis, of course, but even if you only need to go surface level, you should always make notes of your discoveries. Should you need to come to a difficult conclusion with a submission, document your thought process in full. Ideally, explain every step so that no questions are left unanswered.
Talk about the game a little bit, make note of your favorite sections of a TAS, point out improvements to the author. Be welcoming to new submitters, be respectful to veterans, be encouraging to those who need more practice, and be precise and straightforward to those who break the rules.
If you mention other submissions or publications, always link to them to make things easier on whoever's reading it. This is particularly helpful with obsoletions.
Classes and goals
Take note of each of the following pages for classes, as they contain the most up-to-date information what goals are acceptable under them:
Make sure branch labels are accurate. Ask other staff, particularly Publishers, for advice on this if need be. "any%" labels can be removed for redundancy.
After having approved the goal of the movie, it'd help Publishers a lot if you also catalog the submission while you still have all the files required for syncing it.
Improvements and obsoletions
A new movie may involve emulation accuracy improvements, which often adds extra lag, resulting in improvements that appear longer than the movie they're trying to obsolete. Always ensure the new movie has gameplay improvements. If the new timesavers can be applied to the old movie, they count as improvements.
Major skip glitches
Major skip glitches (sometimes called game-breaking glitches) require additional confirmation that it's actually a developer oversight, and not a cheat code or a password that's subtly entered and looks like a glitched skip.
The most certain way to determine this is reading what programming code the game is executing, but there are some trends among cheat code scenarios that can make this detection easier:
- Input that works specifically during pause screens, menu screens, or other non-gameplay-based input
- Buttons have effects that have nothing to do with what the button normally does in gameplay or in menus
- Doing certain steps a number of times, or entering certain button combinations ("jump 10 times then press B+Select")
- Easy and consistent to replicate if you know the steps
- Increments or decrements memory values in consistent steps, especially if not related to normal gameplay functions (see also point 2)
Game completion point
Glitched endings
Runs that incorporate "game end glitch" techniques need certain verification on whether the ending really occurs. It can be done by comparing how the game acts after it ended normally with how it acts after it was glitch-ended.
Post-completion input
Manually ensure the game has fully ended. For example, after the game reaches some screen that does not seem to automatically fade out nor lead to the game start again, save a state and try pressing a few buttons to see if it's possible to proceed further (switch to recording if needed).
Sometimes extra input is required after the game's ending sequence has started, so it can fully proceed to the last ending screen. It is preferred to contain such input in the submitted movie, but exceptions can be made and an extended movie can be used for encoding.
Editing the submissions
Apply the Editor Guidelines to your editions. Specifically:
- Do not delete text written by others.
- Use clear, easy to read markup and language.
- Attribute clearly.
When you edit a submission, your markup should look something like this:
----
[user:your username]: Judging!
[user:your username]: Accepting as an improvement to the [420M|published run]!
(
420M (an example) stands for a link to publication 420.)
Replacing submission files
Judges have the ability to replace a submission file. This can be done when the submitter finds a small improvement to their submission; it is usually better to incorporate it immediately rather than cancel and submit a new movie. Before replacing, check that the movie still satisfies its stated goals and that the movie file still syncs.
Use this feature only for small improvements and not major changes. Anything that could possibly invalidate the voting results should definitely be a new submission (such as a controversial trick, a major improvement, or anything that might affect the entertainment value of the movie).
Improving the submission you're judging
Sometimes a judge may come up with a complete improvement and even become a co-author. There's no hard line on how exactly to handle it, but there are two basic options:
- If you've fixed a minor issue without investing too much effort (like trimming blank input at the end, or starting the game a bit sooner), just replace the submission with your updated movie and continue judging
- If you've improved actual gameplay and it was non-trivial, you need to reset the submission to "New", post the improvement for submission replacement, and request co-autorship (or you may replace the file right away and add yourself). Another judge will handle all the rest.
Procedures
Judging
When you're about to handle a submission, set it to "judging underway" so other judges know it is already being taken care of. If you feel strongly about judging a particular submission that someone else has claimed, ask them politely and they will most likely hand it over to you.
A submission must be in the queue for 72 hours before a verdict can be made. This allows for the audience to provide critical feedback. The audience has a say, so let them say it, and give the author a sufficient chance to respond to them. Let active conversations play out, and contribute to them with your own thoughts.
Avoid making blanket statements like "this section looks improvable". Instead, ask the author why they did that section in the way they did, or take the time to show that section IS improvable. Show an existing WIP, or make a short one of your own.
If you expect some people to be unhappy about the verdict you are leaning to, communicate your thoughts and options in a post and discuss them as well. Your decision should ideally reflect a consensus and feel understandable, especially for the author(s).
Reviews
- A Reviewer checks which submissions are unclaimed or need research, and makes a review forum post, adding themselves to the Reviews wiki list.
- A Reviewer can also review rejected submissions that may now be acceptable under rule changes.
- The review doesn't need to cover 100% of the aspects that a judgment covers. Here's a list of the most meaningful individual verification tasks, a review may opt to verify just one of them, or an arbitrary number of them:
- Replayability (sync) of the submission, ideally coupled with cataloging it
- Goal choice: which class the goal belongs to, and whether it's fulfilled in the movie
- Optimality
- A Judge waits with the claiming before actually starting working on a submission (to let Reviewers know which submissions they are free to review), then checks the Reviews list and uses existing reviews for that submission in the judgment.
- In the decision, the Judge addresses which parts of the review were correct and which weren't, so Reviewers could better learn how to approach submissions, but without getting intimidated by criticism.
- Upon judging, the Judge removes the submission from the wiki list.
- If one wants to review a claimed movie, they just make a regular post without adding themselves into the Reviews list.
When you're done
Even after you've finished judging a movie, a judge's job doesn't necessarily stop there.
- Communicate with viewers who have questions regarding your judgment.
- Communicate with the publisher and assist them as necessary.
- Communicate with other judges who need to judge a different submission for the game if they have any questions.
- If it was revealed you made a mistake in your judgment, first of all, don't beat yourself up, everyone makes mistakes.
- Any mistake other than those which resulted in a published movie is fixable.
- If the movie was already published, speak to the other staff, maybe someone has a creative solution. If not, it happens, revel in your ability to be human.
- Learn from your mistakes for next time.
- Any mistake other than those which resulted in a published movie is fixable.
More reading
- For movie analysis: Movie Rules, Guidelines.
- For editing: Editor guidelines, Text formatting rules.
- For movie editing: Movie Tag Guidelines.
If you're also a publisher, please read the Publisher guidelines as well.