EDIT (8/15/2021): Instead of being expanded, the Vault system has officially been retired as of today! We are moving away from tiers and into just general acceptance and archival, no longer dividing the runs on our site by entertainment value or using it as a metric for judging every single run that comes in. Victory is ours!
This thread will now be used to collect and discuss feedback on the implementation of this new system. Implementation is currently still in progress, so any and all feedback is appreciated! We'll be continuing to collect feedback going forward as well, so if I forget to remove this message after several years, feel free to pretend it's still relevant and post your feedback anyway!
Huh, this feels familiar.
So, we've had this tier for nearly a decade now. There's definitely been some opinions on it, some positive, some negative, and not all of the negative ones even belong to me! In fact, there's kind of a growing sentiment in general about Vault not quite being a perfect system, to the point where it's about to replace Communism as the average college freshman's go-to "It's a good idea in THEORY, but in PRACTICE..." argument.
The current nature of Vault is that it's supposed to be a completely objective tier, one that cares only about speed and completion. Play a game in the fastest way possible, and do everything in a game in the fastest way possible. A good idea in THEORY, but in PRACTICE... It's still incredibly restrictive: "any%" being the pure "fastest completion" category means that it is entirely possible for something like a "game end glitch" run to be the only acceptable run for a game, assuming that game allows it and doesn't meet entertainment standards. "100%" is extremely hard to define for certain games (RPGs come to mind), and may not even apply at all, meaning that once again there could only be one acceptable run for a game under our current Vault ruleset if a game just doesn't have a proper "full completion" definition. Games aren't exactly objective to begin with, limiting our objectively accepted categories in such a way is, almost ironically, one of the more
subjective things we're doing on the site.
How do we look at games more objectively, then? This is something that's going to take some community discussion to really iron out, but we've started coming up with a potential new system that should greatly expand Vault in a satisfying way, allowing for more runs to be accepted without opening the doors wide enough to accept
absolutely everything: Instead of strictly only allowing fastest and full completions in Vault, we would allow
any objective goal for a game. By
objective goal, we mean "something that is defined or limited by the game itself".
Here are some big examples, but definitely not EVERY example:
- Pure fastest completion (GEG, ACE, etc)
- Warps (or fastest "intended" completion)
- Warpless/All levels (warps are an intended shortcut so not using them counts as an objective goal)
- 100% (assuming it can be defined objectively)
- Score attack (an objective goal with room for improvement in the same vein as fastest completion)
And here's a list of smaller things that are more dependent on the game, but still objective:
- Gameplay modes (including things like New Game +)
- Number of players
- Character choices
- In-game defined goals (i.e, achievements, things that reward the player with content)
Now this sounds like a lot. In fact, this sounds like several thousand of the runs on the site. You're entirely correct to think that, person I invented for this paragraph! The long term goal here is redefining the tiers in a more standardized way. Currently, Moons is treated as the "default" tier, a relic of the site's past where entertainment was a strict requirement for publication. Even with the addition of the Vault to open the site up, we still limited it pretty heavily at the time, and to this day it's not exactly a welcoming place. It's not fun to be a Judge and have to say "Well, people think your run is boring, so we're gonna put it in the part of the site most people ignore". It reeks of the exact same elitism that the Vault was meant to mitigate.
Expanding Vault in this proposed way would effectively bring it up to the "default" tier of the site. A change as big as this would lead to the following:
Redesign
Vault would no longer be Vault. It doesn't sound like a big change on paper, but it's arguably the most symbolic. Given that this would be the biggest tier by thousands of movies, it shouldn't carry the same sort of aesthetic as it currently does. This could either be getting a new name and logo, or it could be removing the concept of "tiers" altogether and just having "Vault" be a set standard. In that case, Moons would just be a tag in the same vein as Console Verified or Commentary, denoting that the run is "non-standard" and was voted on by the audience to be a part of the site.
Branch discussion
"Too many branches" would have a much higher cap, since most games would automatically gain several publishable branches due to this change. This would also smooth out the debates over particular games getting more attention than others: Gone are the days where SMB1 or Super Metroid can have 10 branches a piece while other games can only ever have one, so there's less implied favoritism over certain games. Of course, Moons would be treated the same way as before, favoring audience reaction heavily, but even these days we don't get many submissions that would
have to be Moons under this new system.
There's a point to be made that "objective criteria" can lead to a massive number of branches for certain games, particularly fighting games with multiple characters, gameplay modes, and difficulties. We would still limit this in some way, even though it does kinda directly go against this proposal. The difference here is that it wouldn't be an unwritten, undefined, arbitrary limitation this time around. We're still a fairly small staff team, and this could be a change that drastically increases our individual workflows, so we want to find the right balance between allowing as much as we can, and not overworking ourselves trying to keep up. We'd find a way, with the help of y'all as the community, to pin down a limit that works for everyone here.
A change in the culture
For me, personally, this is the most exciting change. Entertainment would no longer be a metric for every single submission that comes in, only movies that wouldn't qualify for the new Vault. Vault would no longer be seen as a burial ground for movies that weren't entertaining enough, there's no stigma for making a movie that's "too boring" anymore. I've seen the current sort of "entertainment culture" negatively affect a lot of newer TASers on the site throughout my time here and it would be really very nice if that could stop, thank you very much.
Demonstrations
There's a discussion to be had about what can be allowed as a Moon as well, perhaps even opening that up more, relaxing some of the strict requirements that have
prevented completely legitimate TASes from being showcased on the site. A lot of the reason we never had something like a "Demonstration Tier" is the fact that it was hard to define what would be allowed as a demonstration. Ironically, I think the discussions surrounding that were looking at it a little too objectively, as opposed to just being "Well, if the audience wants it on the site, it should be on the site." With a massive increase in the range of content we explicitly allow, it seems fair that we can encourage similar variety here.
So what do you all think? Does the general idea sound like a good step forward for the site? What needs to change, if anything? Any comments, suggestions, questions, or ironic joke posts you may have,
(Samsara,) post away and let's get this discussion rolling.
but something closer to this
For the extremes, if it's a super clear record, we don't need it to be entertaining (game end glitch), and if it's super entertaining, we don't care about its record value (playarounds). But in the middle, there's some overlap of uncertainty (dunno how to draw it), when we're not sure about acceptace on either ground. And those are the worst to deal with. Which kinda means, if we invent a way to handle this middle ground gracefully, the system will be more satisfying and robust.
