Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Since the "score should be known to be maximum" requirement originated in my judgment of http://tasvideos.org/5478S.html I'll point out that its meaning is that we can't consider max score a version of full completion, if we're not sure it's actually full completion score-wise. This only applies to the current Vault rules, and I haven't read this thread at all.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
In short, I'm figuring out what objective rules make sense to expand the environment to include high scoring runs. There's a few nasty problems I'm running into in defining edge cases.
Games with a "firm maximum" in their Limited Scoring are those that better fit the Vault definition, as there is a relatively easy maximum to calculate. Since 100% completion can be made readily apparent from this, all that matters for a TAS is to achieve that score then beat the game as soon as possible, preserving the time-based metric Vault appears to be based off of.
Ones with a "soft maximum" are those whose system does not lend itself to a readily known maximum, yet the maximum is known to be finite. There is some apparent interest in including such runs. But by Vault rules, you can't get a clean 100% definition, and therefore can't be allowed in Vault. The Vault inherently works off of time-based metrics, not some arbitrary value in the game. The only way for a "100%" Vault run to be obsoleted by a slower 100% Vault run is as a form of correction, where the prior 100% goal is later discovered not to be complete.
So, I'm trying to help design a set of rules that allow for "soft maximums" to be published. Nasty bits include maintaining Limited Scoring where later a trick is discovered to allow Endless Scoring, and some method, taken as far out of the audience hands as possible to minimize subjectivity, to pick which tricks to ban when a combination of two or more allow Endless Scoring.
Under no circumstance is a programmed upper limit to be taken into consideration for the scoring rules I'm trying to come up with. If a game has a score-based ending, such as River Raid, we can file that under Vault rules as appropriate.
I do not intend to design rules to support Endless Scoring, although a provable score/time loop or perhaps even score/stage loop would be my suggested metrics. I am trying to clearly define a line between Endless Scoring and Limited Scoring, and any Endless Scoring rules can build off of that line.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Memory wrote:
To be honest, I'm not sure that I like that high score isn't a valid option for Vault. It seems to me that Vault is heavily biased in favor of speedruns and against other competitive forms of superplay. TAS doesn't stand for Tool Assisted Speedruns, it stands for Tool Assisted Superplay, yet our Vault policies don't really reflect this. Sure for some games it'd be trivial but same is also true for speedruns.
TAS stands for both speedrun and superplay, and we have tiers for both.
Wiki: Vault wrote:
Aims to fulfill the site's goal of being a Tool-assisted Speedrun records site, as well as be an information based central repository for the body of TAS literature.
Aims to fulfill the site's goal of exposing as many people as possible to Tool-assisted Speedrun/Superplay movies as an art form.
Moons is the middle ground, and its page doesn't tell what exactly it means by TAS, but the goal indicates it more or less:
Wiki: Moons wrote:
Aims to fulfill the site's goal of being entertainment based and provide impressive and high quality TAS movies to the audience.
Vault goals have to be strictly limited, because otherwise it turns into a questionable mess of arbitrariness that isn't even justified by entertainment value. They also have to be as clear as possible, because when it's not clear, the record itself becomes iffy too. With pure fastest completion and full completion our definitions are simple and verifiable.
The allowed set of goals can't be painlessly narrowed down further, because those are the most popular speedrunning categories, they have the highest demand, and they are fairly competitive. And it can't be painlessly expanded either.
For boring simplistic games, additional goals won't look substantially different from standard Vault goals.
What we require for games that have no ending is already exhaustive, and playing the game farther just to get a certain score count won't add any unique content.
When a score just overflows at some point, it may take hours of identical gameplay, which has little speedrun record or even superplay value.
Just like with educational games, if we actually get superplay, we publish it to higher tiers: nothing really prevents a high score movie from being entertaining to the general audience. But if it fails to entertain, it doesn't mean we should shove it in through the back door.
For goals where no limit can be found whatsoever, you constantly balance between unpredictable time and unpredictable score, compromising one or another, or both at once, which makes it a hell to compare and obsolete with improvements, because it's so hard to actually measure improvement.
moozooh wrote:
Now to comment on the goals: this is an arcade game, and the vast majority of arcade games are made with scoring in mind (because having a scoreboard on the cab is one of the principal ways to get you to challenge them upon beating the game, so you keep spending coins), so going for score is definitely not a misguided choice in this case—and neither is looping the game until the kill screen. Moreover, what made me watch this TAS in the first place was the fact that it was presented as a bona fide research project rather than something picked up to get another publication on the site. It shows genuine care and intellectual insight into the subject matter—and, game choices aside, that is exactly the approach I think we cherish the most here, in principle. A person who is actively seeking the game's limits out of love for it is the person most likely to end up with a superior result.
This raises an interesting question: whether we can accommodate high score completions into the Vault for the games that warrant them (read: mostly arcade games and their ports) the same way as regular any% runs if they don't meet requirements for a higher tier. It feels like it would open up some doors for fresh and high quality content brought as the result of such a research. Rejecting a run like this—and this one in particular—on a technicality would feel quite counterproductive to me because it's evident there is at least some audience for runs like this (I mean, even this game actually has its own community...), and considering the diligence involved, we would be very glad to have this run without having to ask extra questions if only the policy did account for the goals in question. So I'm with Memory on this matter.
We can't judge research behind a movie, because it's not verifiable and not reproducible.
Loving the game you TAS is indeed a thing we praise. But it's a subjective thing, and subjective feelings have to be shared by broad audience and reach Moons or Stars. Vault is for objectivity and clarity. We want its movies to look as legitimate and reliable as possible to as many people as possible.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
High score should really be treated like low%.
- If a new TAS gets a higher score than the submitted one, it can obsolete it even if it's slower.
- A submitted TAS can be rejected if it is discovered during the judging that a higher score is possible.
Current project: Gex 3 any%
Paused: Gex 64 any%
There are no N64 emulators. Just SM64 emulators with hacky support for all the other games.
We can't judge research behind a movie, because it's not verifiable and not reproducible.
That's not what I meant. What I meant was that you generally don't want to shy away contributors whose approach commands diligence. Why would you not want more quality-made TASes on the site? If a policy becomes a problem, maybe it makes more sense to change a policy. Especially considering it has to do with goal choice and not submission quality—it's not like anyone here suggests dropping standards.
feos wrote:
With pure fastest completion and full completion our definitions are simple and verifiable.
The allowed set of goals can't be painlessly narrowed down further, because those are the most popular speedrunning categories, they have the highest demand, and they are fairly competitive. And it can't be painlessly expanded either.
Competing for the highest score has been a thing since Space Invaders—some two decades before speedrunning even became recognized as a separate hobby, let alone a cybersport discipline. High-score superplay VHSs and DVDs have been made and distributed e.g. in Japan since the early 90s. For many games such as 2D shooters, puzzle games, etc. where manual progression of gameplay is either very limited or isn't even the point at all, it typically remains the most sensible metric of success. It's neither ambiguous nor arbitrary; it's literally a number that only goes up. If a game tracks score, it is typically shown right there on the screen at all times. Many games have elaborate scoring systems where getting an optimal result, or even figuring out the optimal strategy for reaching it, is far from trivial, which makes it interesting for anyone competing in that game.
feos wrote:
What we require for games that have no ending is already exhaustive, and playing the game farther just to get a certain score count won't add any unique content.
Games that loop indefinitely don't have a maximum score by definition, so it's not even a problem...
feos wrote:
When a score just overflows at some point, it may take hours of identical gameplay, which has little speedrun record or even superplay value.
...Yeah, I seriously doubt that ever becomes a problem. If the game has an infinite scoring pattern, it just means the game doesn't have a maximum score, so there's no point trying to reach it to begin with. The arcade score tracking authorities have delisted all games where an infinite scoring pattern was discovered, which is the right way to go about it. Also, I don't see droves of prospective TAS producers lining up to make multi-hour runs just because max score is accepted as a valid category for the Vault. If that problem ever emerges, it would only signify to me that the site is doing too well—which is not something I would say about it right now.
feos wrote:
But if it fails to entertain, it doesn't mean we should shove it in through the back door.
That's kind of the point: right now it looks like shoving though the back door, but it doesn't have to be. There is no liability to having runs like these—well-made, with natural and clearly defined goals—on the site, even if the game isn't popular. We are way past the point where having "too many runs on the site" was a legitimate concern (if it ever was, really). The first time there was a discussion about that, the site had only around 300 active publications, and now we have over 2000. None of the platforms serving gaming content—be it video streaming services, game stores, or something else—have been as picky as TASVideos when it comes to disqualifying content for goal choices that happened to be misaligned with its policies at the time of submission. It's like the site is continuously afraid to expand and is trying its hardest not to, treating any relaxation of its rules as some sort of a dangerous compromise that may hurt it one day (though it never actually has). Something to think about in any case.
I think I said everything I wanted on the matter; the rest is up to you.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Joined: 10/12/2011
Posts: 6441
Location: The land down under.
andypanther, I disagree with that for two reasons.
1. high score implies that there's a max score which should have a TAS instead of a high score one.
2. If you're going to do any% you might as well just go fast and not care about score.
You can see this Touhou TAS (superplay) that maxes the score out at 2,429,908,660 points.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account.Something better for yourself and also others.
andypanther, I disagree with that for two reasons.
1. high score implies that there's a max score which should have a TAS instead of a high score one.
2. If you're going to do any% you might as well just go fast and not care about score.
You can see this Touhou TAS (superplay) that maxes the score out at 2,429,908,660 points.
But that's the thing, it's unreasonable to ask TASers to prove that their score is the absolute highest that can ever be reached. This is the only case where TASVideos uses this logic and all it does is prevent TASes from being submitted or even made in the first place.
Current project: Gex 3 any%
Paused: Gex 64 any%
There are no N64 emulators. Just SM64 emulators with hacky support for all the other games.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
moozooh wrote:
There is no liability to having runs like these—well-made, with natural and clearly defined goals—on the site, even if the game isn't popular.
My point 1 addresses runs like this. Also I'd like to see my other points discussed, they mention bigger problems than dealing with overflowing score (that can be just disregarded from "high score" runs).
moozooh wrote:
We are way past the point where having "too many runs on the site" was a legitimate concern (if it ever was, really). The first time there was a discussion about that, the site had only around 300 active publications, and now we have over 2000. None of the platforms serving gaming content—be it video streaming services, game stores, or something else—have been as picky as TASVideos when it comes to disqualifying content for goal choices that happened to be misaligned with its policies at the time of submission. It's like the site is continuously afraid to expand and is trying its hardest not to, treating any relaxation of its rules as some sort of a dangerous compromise that may hurt it one day (though it never actually has). Something to think about in any case.
Are we past 2012 yet? What you're describing has ended 7 tears ago. With Moons policies, almost any arbitrary goal can be accepted as long as it's entertaining to the general audience. You make it sound like entertainment isn't a factor for us, and boring arbitrary goals either go to Vault or get rejected. If people are not entertained by a movie, why would you try to sell it to them regardless? In Moons we have a place for "high score" movies without any problems. Shoving things through the back door means that when we abstract away and try to objectively assess pros and contras, we find out it's not such a great idea to have fundamentally ambiguous goals in Vault, especially when there's already a good place for them, where they have value and are easily accessed.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Also I'd like to see my other points discussed, they mention bigger problems
Pretty much none of them actually are. Combined with the ones you've mentioned I can identify the following concerns.
1. The infinite score patterns—have already been addressed above.
2. Using continues. Arcade games have been designed pretty much from the very beginning to invalidate a score made using more than one coin (and in many cases deny some of the later-game content), and all high-profile players, scorekeeping authorities, and related communities abide by that principle, so there isn't much room for debate here. Coins are an inherent pay-to-win mechanic, an external resource, so it only makes sense to disallow it in any sort of a serious competitive scenario.
3. Whether to mandate game completion if the optimal scoring strategy involves a game over prior to completion. This is pretty much the only point of contention because there is no single universal consensus on it (unlike with continues), but I'd go with making game completion mandatory for the sake of consistency; besides, that's also what Japanese scorekeeping organizations favor as well (JHA, and Arcadia before it, accepts scores on partial completions of a new game only until a full completion becomes available, which overrides an earlier one even if the final score is lower).
4. Infinitely looping games. Provided they have an actual kill screen that makes scoring finite, it's fair game for the Vault, since its point is, in principle, to document optimal solutions when they fail to entertain. The demand for unique content in this respect is quite arbitrary. We chop off tons of valuable unique content by making large improvements that obsolete earlier runs, and I see this as a much worse problem than adding content that happens to be repetitive. After all, no-one forces anyone to watch past the first loop if there is no actual difference to the gameplay. Hell, no-one even forces anyone to watch a scoring run if they aren't interested in the scoring. I mean I don't watch those multi-hour RPG runs for that very reason. For me, this run was more watchable than those. But hey, some people want them, who am I to judge? Surely some people want this kind of content, too.
All of these aren't emerging problems; they are basic principles that need to be agreed upon beforehand, which solves every related problem well in advance. If you can think of any others, sure, let's discuss them. Perhaps a separate thread is warranted.
And with that in mind, let's try this again: What is the actual problem with having highest score as a Vault-able category? What makes it an unreliable or otherwise unwanted metric?
Speedruns have all sorts of trade-offs and extra restrictions: high-glitch vs. low-glitch, taking damage vs. damageless, using deaths vs. deathless, max speed vs. speed/entertainment trade-offs, etc. Scoreruns are typically far less ambiguous in this respect: it's either a higher score or it isn't. After the basic principles have been established, there should be no further ambiguity.
So far you've called the goal of highest score "arbitrary", "fundamentally ambiguous", and "hard to measure", but it really isn't any of that; these descriptions are borderline FUD based on no actual cases where it was hard to determine whether something was an improvement (or even a good score, period)—you just came up with some vague imaginary scenarios that haven't been problematic since the very beginning of organized score tracking. And all the while multiple national and international organizations and sites have tracked records based on this very metric for longer than you and I have been alive. Surely they have figured it out—why can't we? As I repeat again, it's just a simple counter that can safely be taken at face value (barring the problem with running out of displayed digits, which is adequately solved by memory watch as in this case—a tool we have that unassisted scorekeepers don't!). You don't have to be a PhD to tell that 100 points is more than 99. I honestly don't understand why this seems so difficult; it's really anything but.
feos wrote:
Are we past 2012 yet?
Well, are we? The reason this discussion happens at all is that the run has to either comply with an (arbitrarily chosen) entertainment barrier to make it to the Moons or be rejected as ineligible for the Vault as per its goal choice. In other words, you're trying to determine whether to filter out a technically well-made run by its goal choice—based on a very representative selection of a dozen forum members. You can theorize upon whether the audience is entertained all day, but the truth of the matter is that TASVideos forums have absolutely laughable user presence for anything unpopular, and even then the response has been mixed rather than strictly negative. None of the members of the Bongo community Lizstar is from commented here (or at least they didn't identify themselves). But if you look at YouTube, you'll see (as of right now) the temp encode has 425 views. Unless there is some gross cheating involved (which I severely doubt), some four hundred people have at least partially watched this run, of which less than 3% bothered to vote in this thread. There are more likes there than there are votes here.
This suggests to me that a non-negligible audience exists for this run. But let's be frank here, the chances of it making it to the Moons by entertainment is incredibly slim. So yeah, it's 2019, we have a submission requested by several hundred people on a more visible/accessible platform, and you're about to reject it based on a technicality that artificially reduces the scope of the site. And before that, you suggested that having a run with repetitive content in it hurts the site somehow, which also reeks of the 2012 rhetoric. Thanks for making my points for me, I guess.
Spikestuff wrote:
You can see this Touhou TAS (superplay) that maxes the score out at 2,429,908,660 points.
What makes you believe it maxes the score out? It only maxes out some of the counters. The score can theoretically be increased further. Nothing suggests it cannot.
ViGadeomes wrote:
high score is only valid if we prove that it's the highest score reachable on the game accepted as a full completion/100%.
I'm sorry, but this demand is absolutely ludicrous. No-one demands that speedruns be proven unbeatable upon submitting, so why should scoreruns be? In 100% runs, the item count is a restriction (in other words, a secondary goal): it's still a speedrun for the fastest completion, except the player is obligated to collect all items. The run doesn't end when the last item is collected—it ends when the game is completed with all items collected. But the score is not a restriction; it's the primary goal in itself. Everything else can and must be sacrificed for the sake of the highest score. A submission for the highest score should be fine as long as the strategy is sound, there are no obvious mistakes, and it beats all available records—in other words, the same general requirements we put on fastest completion submissions.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
The allowed set of goals can't be painlessly narrowed down further, because those are the most popular speedrunning categories, they have the highest demand, and they are fairly competitive. And it can't be painlessly expanded either.
Why do we care about what is the most popular speedrunning categories? I'd argue the only reason we care about speedrun records more than other records is simply that we started out with speedruns and it's what is most popular currently. However, I don't see that as a particularly valid reason to favor them in our rules. Given that it's been 2 years since the last publication that didn't aim for speed as one of its primary goals, even for Moons, I'd argue that something has gone wrong here. Communities that don't aim for speed clearly exist and even thrive, but we don't see any of them here.
I also feel that you'd need to take a different mindset to appreciate this run specifically. I feel this game is at it's most interesting when it is aiming for score. While on the surface level, the game appears simplistic and boring, when you take into account the scoring system, it becomes much deeper. Routing decisions become much more complicated. There's the little goofy jumps off the side of the screen which I really like. Sure it goes on for a while but I feel overall it's really unique and is the content I love to see on the site above all else.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Memory wrote:
Why do we care about what is the most popular speedrunning categories? I'd argue the only reason we care about speedrun records more than other records is simply that we started out with speedruns and it's what is most popular currently. However, I don't see that as a particularly valid reason to favor them in our rules. Given that it's been 2 years since the last publication that didn't aim for speed as one of its primary goals, even for Moons, I'd argue that something has gone wrong here. Communities that don't aim for speed clearly exist and even thrive, but we don't see any of them here.
Until 2012, speedrun records that didn't look impressive to the general audience were rejected. The site cared the most about publishing the most entertaining movies. It didn't have to force speedrunning competitions, they felt natural to TASing and gained their popularity naturally too, as long as the result was decently entertaining. In addition to that, some movies that weren't speed oriented were encouraged, made, and published, because of the same reason: aiming to publish the most impressive content.
When Vault was added, its goal was keeping speedrun records that would've been (and have been) rejected until then. It was an inclusive approach, a lot of old submissions were unrejected, a lot of them got published. And Vault has never been a target for the site: it's name, it's icon, it's appearance (not being included into primary movie lists) was considered by some people a discouragement! It just naturally happened that some people preferred to focus on Vault when TASing.
Since arbitrary movie goals were meant to go to Moons, the idea of Vault was limiting arbitrariness as hard as possible. It makes perfect sense to me: if a movie looks impressive to the general audience, it can have a vast variety of self-invented goals. Previously such movies were also rejected, because some goals were considered too arbitrary, and the judge guideline was limiting the branch count as hard as possible, in order to raise the entry barrier and only feature the best movies.
When Moons became a tier, demand for entertaining arbitrary goals was satisfied, just like demand for boring speedrun records was. Both were previously outright rejected, now most of them have a solid chance. It's no one's fault if a movie fails to entertain enough people, but then it means watching it is a fairly niche joy, depending on personal tastes of really small groups of people. We don't blame people enjoying them, but it was a common decision between admins, other staff members, and the community, to limit Vault to the most clear speedrun goals, and to leave Moons for all the rest. Every movie has a chance, but we can't really publish everything. We're limited by the general site vision, by demand, and by manpower. We can't ignore one when we feel like changing or following others, it has to be a general consensus. Maybe there will be, but as of right now, I'm having problems with high score categories in Vault, and I'll discuss them in my next post.
And I feel that most of this talk should be moved here.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
moozooh wrote:
So far you've called the goal of highest score "arbitrary"
Let's see...
feos wrote:
With Moons policies, almost any arbitrary goal can be accepted as long as it's entertaining to the general audience. You make it sound like entertainment isn't a factor for us, and boring arbitrary goals either go to Vault or get rejected.
I didn't call the high score goal arbitrary, I was clearly talking about various possible goals that are eligible for Moons and not for Vault, as per Wiki: MovieRules.
moozooh wrote:
"fundamentally ambiguous"
feos wrote:
Shoving things through the back door means that when we abstract away and try to objectively assess pros and contras, we find out it's not such a great idea to have fundamentally ambiguous goals in Vault, especially when there's already a good place for them, where they have value and are easily accessed.
Indeed, here's I'm referring to the high score goal as fundamentally ambiguous. It's ambiguous because there's no explicit programmatic limit on score in such runs: you can't collect it all, you can only collect some maximum of it, this amount is determined by all sorts of factors, and you have various levels of control over them. There might be a glitch forcing you to die in a certain level, but later on another glitch can be found that would allow you to skip that level or get past the problematic point otherwise. There could be some soft limit that can be bypassed by clever gameplay actions. This goal is competitive exactly because there are often ways to collect higher score. It's fundamentally ambiguous because you can't know in advance what your actual max score is in the end after all the effort. In the same manner as the final movie time is fundamentally ambiguous, unpredictable, improvable, etc.
Strictly speaking, not the goal itself is fundamentally ambiguous in this case, but the main metric behind it. And since Vault is limited to just two metrics - movie time and clearly defined full completion - the goal of high score can indirectly be called ambiguous relating to Vault, where you guys want this goal to be added.
moozooh wrote:
"hard to measure"
feos wrote:
For goals where no limit can be found whatsoever, you constantly balance between unpredictable time and unpredictable score, compromising one or another, or both at once, which makes it a hell to compare and obsolete with improvements, because it's so hard to actually measure improvement.
Since the metric is ambiguous and the final score count is unpredictable (hence the competition aiming to maximize it indefinitely), it makes the primary goal of Vault, record speed, less clear.
By the current Vault rules, you just optimize everything out and try to get the lowest final time possible, optionally fulfilling the full completion requirements. The speed record is legitimate, obvious, easy to understand and to rely on.
Adding an unpredictable metric to it means there will always be a conflict: one can waste time to gain higher score, waste score to save time, get higher score without losing time, and save time without losing score. One also has to be optimal in gaining both score and time, but with 2 unpredictable functions that you now have to definitively control, it becomes really hard to optimize such a movie and to verify improvements. Not impossible of course, but this removes the clarity of goals that Vault is founded on.
moozooh wrote:
these descriptions are borderline FUD based on no actual cases where it was hard to determine whether something was an improvement (or even a good score, period)—you just came up with some vague imaginary scenarios that haven't been problematic since the very beginning of organized score tracking.
You could've just asked what I meant instead of imagining those overdramatized pictures.
moozooh wrote:
And all the while multiple national and international organizations and sites have tracked records based on this very metric for longer than you and I have been alive. Surely they have figured it out—why can't we? As I repeat again, it's just a simple counter that can safely be taken at face value (barring the problem with running out of displayed digits, which is adequately solved by memory watch as in this case—a tool we have that unassisted scorekeepers don't!). You don't have to be a PhD to tell that 100 points is more than 99. I honestly don't understand why this seems so difficult; it's really anything but.
Those communities work with human records, so it's a competition of human skills, and as such it can be so popular and have all the crowd around it. We remove the human element from TASes, so the task of maximizing the score is not so deeply impressive when tools assist you. Of course it can still be highly impressive. And guess what, we solve this already by putting such movies to Moons based on viewer support.
moozooh wrote:
feos wrote:
Are we past 2012 yet?
Well, are we? The reason this discussion happens at all is that the run has to either comply with an (arbitrarily chosen) entertainment barrier to make it to the Moons or be rejected as ineligible for the Vault as per its goal choice.
When dealing with a community of people with various tastes, you have to account for their subjective feelings when making decisions that directly affect what they get to see published. Categories currently allowed for Vault are extremely good at being clear, because over the years I can't remember a single person saying we shouldn't allow any% or 100% for Vault. With categories that some people asked to add to Vault, it always boils down to this: people say some game genre or goal should be included into Vault, but the majority of the requesters completely refuse to enjoy the result of TASing that game or goal. Just like you admit that this movie's content is unlikely to get to Moons, almost no one was honestly entertained by the Math Blaster movie that featured an educational game. Since people in that thread couldn't find vastly convincing reasons to allow such a game for Vault, and then there was a disagreement among judges about rules interpretations, we had a staff discussion that involved clarifying what the site wants, what the community wants, what makes the most sense, and how to clearly express that in the rules. I sparked this discussion and I was leading it, and I can tell you that the way you're describing our policies and how they are designed is infinitely far from reality.
moozooh wrote:
In other words, you're trying to determine whether to filter out a technically well-made run by its goal choice—based on a very representative selection of a dozen forum members. You can theorize upon whether the audience is entertained all day, but the truth of the matter is that TASVideos forums have absolutely laughable user presence for anything unpopular, and even then the response has been mixed rather than strictly negative.
Mixed response means that if we agree with, let's kindly say, half of the crowd and accept a borderline movie, people posting in the thread will completely forget about it, and after publication people who care about ratings will come and fail to get entertained. It happens all the time and it means we can't just rely on half the posters saying "this is not so bad actually". Posters is only one of the factors to account for. Actual movie contents is another one, and judges are supposed to understand what kind of movies we have in Moons based on gameplay and contents. Feedback of those who rate movies after publication is a very important factor as well, and we're also supposed to understand how people are likely to rate; we also encourage raters when we determine whether a movie deserves a star, in addition to other stars related aspects.
moozooh wrote:
None of the members of the Bongo community Lizstar is from commented here (or at least they didn't identify themselves). But if you look at YouTube, you'll see (as of right now) the temp encode has 425 views. Unless there is some gross cheating involved (which I severely doubt), some four hundred people have at least partially watched this run, of which less than 3% bothered to vote in this thread. There are more likes there than there are votes here.
This suggests to me that a non-negligible audience exists for this run. But let's be frank here, the chances of it making it to the Moons by entertainment is incredibly slim. So yeah, it's 2019, we have a submission requested by several hundred people on a more visible/accessible platform, and you're about to reject it based on a technicality that artificially reduces the scope of the site.
Should I go to youtube and find out how many people watch and like videos with Game Genie or manual RAM poking? By your logic this alone is supposed to force us to deny the rules of our community and accept blatant cheating for publication.
moozooh wrote:
And before that, you suggested that having a run with repetitive content in it hurts the site somehow, which also reeks of the 2012 rhetoric.
Quote?
moozooh wrote:
What is the actual problem with having highest score as a Vault-able category? What makes it an unreliable or otherwise unwanted metric?
I elaborated on my point 5 above where I explained why I called this goal ambiguous and hard to measure improvements for.
My point 1 escaped your attention completely (even after I mentioned it explicitly again), but it's one of the main problems the current movie rules have with your suggestion.
Here's our general approach to branching:
Movie Rules on goals wrote:
Arbitrary goal choices need to offer new TAS material to be accepted. Choices which have no goal other than to create a new game branch are rejected.
Keep the number of different branches per game minimal. A run for a proposed new branch for a game should offer compelling differences relative to previously published runs of that game.
Sometimes what is claimed to be an improvement may necessitate a new branch, such as when a huge time-saving glitch is involved (see SMW2), or unusual goal choice is exhibited (as in Princess-only SMB2 run, whose first iteration beat then-current any%).
To determine whether there is new TAS material or compelling differences compared to vaultable goals for this game, I watched this submission side by side with the RTA any% run.
The game is overall not as basic as directly running right all the time. You have an option to ignore items and just have any%, only utilizing the items that can save you time. You also have an option to collect all items, and then it could be considered full completion. In an earlier post I said that max score can't be considered full completion in itself if there's higher score that you can potentially get. But if we define full completion as "all items", that seems to fit just fine. There's a question about loops and in-game conditions preventing full item collection in later loops, but as I said, maybe later loops aren't required.
This movie does collect all items in earlier loops. So if we decide that 1 loop is enough, then a vaultable full completion movie of this game has already been made and only has to be trimmed, checked for optimality, and has a solid chance to be published.
Fastest completion goal is still available for this game and it will also be published if someone makes it, once we determine how loops affect gameplay contents and difficulty.
If you compare this submission to those 2 perfectly vaultable goals, it doesn't feature any unique content or compelling gameplay differences. And this is even more so for games that are simpler than this one. Any% and 100% is exhaustive for most boring games that Vault exists for.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
For goals where no limit can be found whatsoever, you constantly balance between unpredictable time and unpredictable score, compromising one or another, or both at once, which makes it a hell to compare and obsolete with improvements, because it's so hard to actually measure improvement.
Since the metric is ambiguous and the final score count is unpredictable (hence the competition aiming to maximize it indefinitely), it makes the primary goal of Vault, record speed, less clear.
By the current Vault rules, you just optimize everything out and try to get the lowest final time possible, optionally fulfilling the full completion requirements. The speed record is legitimate, obvious, easy to understand and to rely on.
Adding an unpredictable metric to it means there will always be a conflict: one can waste time to gain higher score, waste score to save time, get higher score without losing time, and save time without losing score. One also has to be optimal in gaining both score and time, but with 2 unpredictable functions that you now have to definitively control, it becomes really hard to optimize such a movie and to verify improvements. Not impossible of course, but this removes the clarity of goals that Vault is founded on.
Obviously, one should prioritize score for a high score run. Time should only be considered as a tie breaker and for purposes of clean play.
Mixed response means that if we agree with, let's kindly say, half of the crowd and accept a borderline movie, people posting in the thread will completely forget about it, and after publication people who care about ratings will come and fail to get entertained. It happens all the time and it means we can't just rely on half the posters saying "this is not so bad actually". Posters is only one of the factors to account for. Actual movie contents is another one, and judges are supposed to understand what kind of movies we have in Moons based on gameplay and contents. Feedback of those who rate movies after publication is a very important factor as well, and we're also supposed to understand how people are likely to rate; we also encourage raters when we determine whether a movie deserves a star, in addition to other stars related aspects.
Honestly, over time I have started to lose faith in the value of ratings. Very few people actively participate in ratings... at all. I'm really not sure we really should be relying on them to determine popularity as much as we have. Besides, a rating is a fairly arbitrary number, a post indicates much more thought.
This movie does collect all items in earlier loops. So if we decide that 1 loop is enough, then a vaultable full completion movie of this game has already been made and only has to be trimmed, checked for optimality, and has a solid chance to be published.
That is not correct, this movie actually skips a few items, even on the first loop. That is because collecting enough items total immediately ends the screen. Due to this, it is impossible to collect all items in a single loop and given that all items respawn in later loops, I cannot seriously treat collecting them as "optional one-time, irreversible, or otherwise strictly limited accomplishments that can be objectively measured and maximized" as indicated by the full completion rules.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Okay I can say that for games that have no way to define full completion at all, and were more or less designed for score instead, it might be a lesser evil to allow "max score" for Vault as a replacement for full completion. I'll see where this gets us.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Okay I can say that for games that have no way to define full completion at all, and were more or less designed for score instead, it might be a lesser evil to allow that for Vault as a replacement for full completion. I'll see where this gets us.
That seems reasonable to me. Most games I can think of where high score makes sense as a goal do not have a sensible full completion definition.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
I... hate arguing for denying expansion of Vault, but keep in mind the following points:
Vault, from prior talks, is purely a time metric.
If a game has a clear and obvious maximum, apart from an imposed no-ending score limit when further scoring actions are possible, a 100% definition can say to hit that maximum then clear the game as quickly as possible. We can preserve the time metric as the maximum is known, and further improvements are still based only on time.
I won't mind breaking this barrier and also include a score metric, but it is my impression that Vault is built heavily on time metrics. I'll be happy for allowing "score first, time second" ruling anytime.
Vault keeps only a bare minimum of glitches banned.
If the run merely corrupts the 100% completion tracker of the game, no one wants that published as a 100% completion run, even as Vault. Probably why Vault doesn't allow any and all glitches, but it is otherwise very loose.
For now, this game is in a finite scoring environment. But what if we discover a way to continue surviving past our current barrier? What if that was the one barrier left in our way to flip this into an endless scoring environment? We didn't corrupt the score directly, so we shouldn't restrict the hypothetical trick as per Vault rules, but now we have an infinite score to contend with as our secondary score resource is now endless.
In essence, we can't obsolete a published run with a better score, as we could always just use whatever trick allows for endless score to go higher. And any stopping point (aside from a convenient stage boundary) is probably going to be considered arbitrary, I mean, why stop there when there's more score to get? And since it is endless, it doesn't matter when you stop, you could have gotten more. Therefore, it is impossible to obsolete a prior published run should such a trick be discovered later.
This is the demon I couldn't fight in the other topic. If you guys can "rule" that away, I am all for any changes. My best thought against this is to ask the community if the restrictions chosen or stopping point is a valid one for the run. If there's a big argument, deny the run, it's obviously not clear.
As much as I want score to rule the 100%-like definition in various games, particularly those games that I would call having a "soft maximum," I also want to make sure everything going forward is as clear as possible. I want the lowest chance of problems happening, and we have a Punch-Out!! run to test against any hypothetical rules. And yes, that one has a glitch that allows endless scoring that the TAS avoids.
Don't forget my topic. Probably ideal to continue score related discussions. I have parts of a framework in there ready, so build on it!
I don't see how that second issue is exclusive to high score runs. In our rules for games with unclear endings, we allow fastest completion runs to end at kill screens where one is unable to complete the current round. If it was determined that the kill screen can be avoided and the game can be played indefinitely, we would need to find a different stopping place, which holds true regardless of goal.
EDIT: Now the submission you mentioned, Mike Tyson's Punch Out is a bit of a different case from this game given that the clock stop glitch is a thing. In that case you can actually score indefinitely without passing the defined ending point. I'm not sure how to most elegantly handle that. The obvious answer is that any game where score can be obtained indefinitely should not be allowed to have score go to Vault, but if a glitch similar to Punch Out's is found after, that is a tricky situation. That does not seem to apply to this particular submission at all but it could for a number of others. The easiest solution I can think of would be to not allow such methods of delaying game progression indefinitely but I'm not sure how that fits in with spirit of Vault.
EDIT2: After thinking about it some, the same situation could pop up with our rules against triviality where a glitch could be found that renders completion trivial, yet is faster. I think the spirit of the rules could be similar here.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Having just passed by this thread, and would like to see runs that aim for high score be Vault-eligible, here's an attempt to make a clear and unambiguous definition of what should be allowable.
For any game with a clearly defined score mechanic:
If progress through the game or the theoretical high/best score is infinite, or the game has an achievable, bounded maximum displayable high score (such as 999999) that does not loop/overflow, an "aims for high score" branch for that game is ineligible for the Vault.
If there is both a point where progress through the game cannot be made further (ending at either an ending screen or a "kill screen") and the theoretical high/best score in the game is to our best understanding finite in a way other than being capped by a displayable boundary, the game should allow for an "aims for high/best score, then fastest time" branch to be eligible for the Vault in addition to the current allowable any% and 100% branches, with the following intricacies:
- The movie should make all effort to attain as high/good a score as possible, and reach the ending/kill screen with that score as fast as possible
- The movie can be obsoleted either on time (if it can achieve the same score faster) or on score (if a means of achieving a higher/better score is found, which need not be faster than the existing movie)
- If 2 (or more) attempted obsoletions of the movie land on the Workbench simultaneously and there are both attempt(s) to obsolete on score and attempt(s) to obsolete on time, the attempts to obsolete on score have priority
- If an improvement is found in the game that causes theoretical progress or high score to become infinite, no further movies can be submitted for that game in this category, and the next any% movie for the game should obsolete the high score movie
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
feos wrote:
Okay I can say that for games that have no way to define full completion at all, and were more or less designed for score instead, it might be a lesser evil to allow "max score" for Vault as a replacement for full completion. I'll see where this gets us.
I agree with this, and I think it is in the spirit of the original intent of the vault rules. Full completion was left vague to allow it to mean whatever makes the most sense for a given game.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Okay I can say that for games that have no way to define full completion at all, and were more or less designed for score instead, it might be a lesser evil to allow "max score" for Vault as a replacement for full completion. I'll see where this gets us.
So for Game & Watch games that lack an earlier max difficulty completion point, vault allows entry with fastest to max out the score? I think that works.
Although we probably only want that in cases where the score was incremented legitimately, not by fishy means, right?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I think we should not let the score overflow or stop at digital cap like 999999. The point of the competition with "max score" is that you never know how much you end up getting in the end, exactly like it works with movie length (it always approaches zero). Aside from that, the ending point remains the same as usual, with just another objective added on top - highest possible score.
Aside from that, should we allow sacrificing time to gain more score? I think if we allow that, one may delay game completion indefinitely, and we don't want that. The clearest solution would be disallowing longer movies to obsolete shorter ones: if score remains the same, we want shorter time; and if time remains the same, we want higher score.
And yes, score should only increment legitimately,without memory corruption techniques, as already said in the rules.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Aside from that, should we allow sacrificing time to gain more score? I think if we allow that, one may delay game completion indefinitely, and we don't want that. The clearest solution would be disallowing longer movies to obsolete shorter ones: if score remains the same, we want shorter time; and if time remains the same, we want higher score.
The way I see it, higher score = fuller completion. For full completion we traditionally want as full of a completion as possible. However, if delaying game completion is possible through standard means and score can be racked up as well, score is NOT viable for Vault for said game. Most console games with a scoring system slapped onto the game work like this and should not be allowed for Vault. For example, Kirby's Adventure has a scoring system, but you can just re-enter stages indefinitely, making it a bad choice. They would have to be Moons viable as you would have to place additional limitations to make score finite.
If a method of delaying game completion is found later, ie the clock stop glitch from Mike Tyson's Punch Out, our best bet would be to disallow it.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
MTPO is a good one to bring up. A max out score on Soda Popinski is actually possible with the clock stop glitch. In fact you could fight him indefinitely. When you get down to low health, you can let him hit you with a jab to get more health, then you get more health by hitting him a few times, and repeat over and over.
So yes, we need to mention that techniques like this should be disallowed for vault max score runs.
Also we need to make it clear that max score should only be the full completion category when there is no better option available.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Proposed addition to http://tasvideos.org/MovieRules.html#Vault_2Max score
The goal of obtaining maximum possible score for a Vault movie is allowed under the following conditions only:
There is no way to define full completion for that game, because there is no way to collect or complete "all X" for any given criterion, aside from fastest completion.
Overflowing or saturating the score does not count as reaching "max score", because it leads to delaying game completion for no good gameplay reason and damages the competitive nature of score maximization.
For games that have no clear ending, the movie end requirements remain the same as for fastest completion.
Like with full completion, score must be gained by the in-game means and not by memory manipulation glitches.
A "max score" movie reaching higher score than its predecessor can only obsolete it if it loses no time over it.
A "max score" movie of the same length as its predecessor should reach higher score to obsolete it.
I think this solves the problem with "infinite score" runs trying to obsolete "max score" runs after new tricks get discovered: you have to be slower in corresponding parts of the movie to get potentially infinite score, and we'd disallow it to be slower, even if it allows to get more score.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
