I don't think ANY chess AIs running on that kind of hardware could be called notoriously strong, but even setting that aside, how would you show that one is stronger than another? Is a brute force AI that looks deep and wide but can still potentially make horrendous mistakes better than an AI that has hardset opening books to prevent fools/scholars mate but otherwise searches much more shallowly?
And how would someone back up their claim? By disassembling the games to show the algorithms used between each?
Mothrayas brought up some good points regarding moons. I agree with the statement: "Moons seem too diluted and I'd like to see them go back to "notable publications" rather than 60% of publications." So I propose the following, which is a modification of the previously proposed. It has 3 main categories (no longer tiers) based on main goal, using flags for ID and filtering, and then moons and stars as concurrent categories and flags to the main categories. This way moons goes back marking runs that are important instead of only entertaining, and it solves the problem of low rated moons and high rated vault.
Here are the categories:
1) Speed as a primary goal over entertainment. This is any%, 100%, game end glitch, probably in-game timer, and perhaps low% (I see it as a harder any%, but speed is still the primary goal.) The poll question would be: "Is this run optimized sufficiently to be published?"
2) Entertainment as a primary goal over speed. This is not moons; that is addressed later. This is the playarounds and other non-speed TASes that are considered entertaining. The poll question would be: "Is this run entertaining enough to be published?"
3) Runs that are not speed focused and not necessarily entertaining, but are interesting enough for publication. This is the proposed "demo/board games" category. This is for demostrating something interesting not shown in other TAses for the game, which includes some of the existing Gruefood delight, and board games that are not trivial (they should be interesting.) The rules for being published here need to be somewhat strict because this is not a dumpster. This is not a category for submission, rather this would be decided by the judge(s) if a TAS does not make it into the other categories.
Now that I think about it, despite this complaint about ratings and votes being hard to gauge, along with requests for allowing ratings to be done on the submission bench, what's stopping it from being implemented? I have no experience on web programming, so some insight would be nice.
I don't think ANY chess AIs running on that kind of hardware could be called notoriously strong
Which platform? I wasn't talking about any particular one.
but even setting that aside, how would you show that one is stronger than another?
Many chess engines have an ELO rating.
jlun2 wrote:
Now that I think about it, despite this complaint about ratings and votes being hard to gauge, along with requests for allowing ratings to be done on the submission bench, what's stopping it from being implemented?
This was done once in the past. Somehow the result wasn't satisfactory (even though I don't remember now what the exact reason was.)
Joined: 8/14/2009
Posts: 4089
Location: The Netherlands
Warp wrote:
jlun2 wrote:
Now that I think about it, despite this complaint about ratings and votes being hard to gauge, along with requests for allowing ratings to be done on the submission bench, what's stopping it from being implemented?
This was done once in the past. Somehow the result wasn't satisfactory (even though I don't remember now what the exact reason was.)
There's a whole big thread about it: Thread #7313: Submission voting (VOTE what we'll do!) [TASK COMPLETED]
Skip to this post on page 13 for post-implementation discussion on why it failed.
It failed at the time for two reasons - one, it overly complicated the voting process, which scared away a lot of voters (including site veterans). You have to think about 0-10 rating numbers much more than for a simple "yes" or "no". Second, people at the time noticed a massive inflation of the ratings with the new system (which is actually similar to the current issue, submission votes/ratings being much better than publication ratings).
It doesn't seem like it would solve any problem, and it would just make voting harder for everybody.
http://www.youtube.com/Noxxa
<dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects.
<Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits
<adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Perhaps the easiest implementation would simply be to allow speed-focused runs of board games in the vault. Because it seems the majority of users wants board games publishable in some fashion, and it would be straightforward to amend or strike the "no board games plz" rule that the vault has.
Perhaps with some restriction that e.g. "all implementations of the same game (e.g. chess) on the same platform (e.g. the SNES) can obsolete one another."
Perhaps the easiest implementation would simply be to allow speed-focused runs of board games in the vault. Because it seems the majority of users wants board games publishable in some fashion, and it would be straightforward to amend or strike the "no board games plz" rule that the vault has.
The question of difficulty level still remains. Personally I would go by the rule that games should always be played on the highest difficulty, even board games.
Perhaps with some restriction that e.g. "all implementations of the same game (e.g. chess) on the same platform (e.g. the SNES) can obsolete one another."
I have to disagree, given the vast differences between such games. Why should, for example, Battlechess be able to obsolete Chessmaster? They are radically different.
Perhaps the easiest implementation would simply be to allow speed-focused runs of board games in the vault. Because it seems the majority of users wants board games publishable in some fashion, and it would be straightforward to amend or strike the "no board games plz" rule that the vault has.
The question of difficulty level still remains. Personally I would go by the rule that games should always be played on the highest difficulty, even board games.
White has an advantage in chess over black.
Would all chess TASes need to use black then to be at maximum difficulty?
That holds for most boardgames too, especially things like Monopoly. Going first is an advantage. Going last is a disadvantage.
The question of difficulty level still remains. Personally I would go by the rule that games should always be played on the highest difficulty, even board games.
Yes, that aligns best with current publications.
Tangent wrote:
Would all chess TASes need to use black then to be at maximum difficulty?
I'd say no, for the same reason that (e.g.) all RPGs do not require that you start with the weakest character class.
That holds for most boardgames too, especially things like Monopoly. Going first is an advantage. Going last is a disadvantage.
I'd say no, because who goes first is (in most cases) decided by dice roll, and is therefore subject to luck manipulation.
At any rate, if there are games with unusual difficulty factors, that can be discussed in a thread for that individual game, precisely as is already done for non-board games (e.g. games like Shining Force II where the purportedly-highest difficulty is actually easier than the second highest).
The question of difficulty level still remains. Personally I would go by the rule that games should always be played on the highest difficulty, even board games.
Yes, that aligns best with current publications.
Tangent wrote:
Would all chess TASes need to use black then to be at maximum difficulty?
I'd say no, for the same reason that (e.g.) all RPGs do not require that you start with the weakest character class.
That holds for most boardgames too, especially things like Monopoly. Going first is an advantage. Going last is a disadvantage.
I'd say no, because who goes first is (in most cases) decided by dice roll, and is therefore subject to luck manipulation.
At any rate, if there are games with unusual difficulty factors, that can be discussed in a thread for that individual game, precisely as is already done for non-board games (e.g. games like Shining Force II where the purportedly-highest difficulty is actually easier than the second highest).
Fair points, although I meant to say that there's still a great degree of arbitrariness as to what 'highest difficulty' means. The Monopoly runs, for example, use 4 players. 8 would obviously be harder. 3 would obviously be faster. I think the Clue run (fringe case since pre-vault, I know) uses just 3 human players and no CPUs.
There's also already at least one recently vaulted run that was played on the completely wrong difficulty, so that requirement is already not particularly enforced. Is that policy going to be changing then?
The "hardest difficulty" in this context usually refers to the game's own explicit difficulty settings, rather than ancillary difficulty that may be produced as a kind of "side effect" of, for example, choosing one playable character rather than another.
In the case of a chess game, it likewise would mean that you have to choose the hardest computer player difficulty, the choice of color is not part of this.
The "you must choose the hardest difficulty" is not a rigid rule with no exceptions allowed. If in some game there's a good reason to use a lower difficulty, then it's usually accepted. In general, "it makes the run faster" is not such a good reason (with possible exceptions).
The "hardest difficulty" in this context usually refers to the game's own explicit difficulty settings, rather than ancillary difficulty that may be produced as a kind of "side effect" of, for example, choosing one playable character rather than another.
In the case of a chess game, it likewise would mean that you have to choose the hardest computer player difficulty, the choice of color is not part of this.
That doesn't address things like number of players or other specific settings that can be altered. Let me use a specific example: Uno 52 for the DS. Its default settings are:
1 human
3 CPUs
Easy difficulty
Endless mode
What are the settings then that would make it eligible to be published? Would there be five possible runs for each of Chips, Uno, Poker, Pot, and Survival modes, all with 3 CPUs on Hard? A single run with 1 CPU on Hard on whatever mode's fastest? A single run with 3 CPUs on Hard on whatever mode is the hardest for the player to win?
And to expand from that, what if someone did the same kind of Monopoly run as we have now with 3 players? With 5? Would the 3 player one be rejected for being the same even though it's faster? Would the 5 player one be rejected for being the same even though it's harder?
Most games have a single difficulty setting which makes it simple. Except for chess, most board games tend to have a lot more settings to tweak that would have significant effects on the length of the game, which makes the general guideline of "hardest" vague.
The "you must choose the hardest difficulty" is not a rigid rule with no exceptions allowed. If in some game there's a good reason to use a lower difficulty, then it's usually accepted. In general, "it makes the run faster" is not such a good reason (with possible exceptions).
I know it's not absolute, but it's also not strictly enforced for the vault which is supposed to be pretty cut and dry, publishing with the stipulation that a later run on the correct difficulty obsoletes. Why would another run be published on an incorrectly chosen difficulty but not a board game? Would that be the new guideline going forward? Or would the guidelines about difficulties going to be an absolute rule for board games unlike others?
What are the settings then that would make it eligible to be published? Would there be five possible runs for each of Chips, Uno, Poker, Pot, and Survival modes, all with 3 CPUs on Hard? A single run with 1 CPU on Hard on whatever mode's fastest? A single run with 3 CPUs on Hard on whatever mode is the hardest for the player to win?
I don't think that's really all that common. Most boardgame ROMs I'm aware of simply have one setting for easy/moderate/hard.
And yes, similar runs can be expected to obsolete each other because they're too similar, just like how RPGs simply do not get runs for each possible permutation of classes, and how fighting games don't get a separate run for each character. Branches must be meaningfully distinct, otherwise they count as the same branch; that's also nothing new.
Really, if you start by treating board games like any other kind of run instead of as some weird exception, then almost all of these problems just disappear, because they're situations we've long resolved for other games. When in doubt, create a forum thread asking "hey, I would like to run game X with settings Y and Z, does that sound reasonable to y'all?"
For example, you might as well ask if Super Mario Bros 2 should have thirty different runs, i.e. Mario+Luigi, Mario+Toad, Mario+Luigi+Toad, Mario+Princess+Toad, and so forth for fifteen permutations plus warpless version of each.
In case you were wondering, the answer is "no, obviously not". So that also means that Uno does not require separate runs for 3, 4, 5, and 6-player mode.
For example, you might as well ask if Super Mario Bros 2 should have thirty different runs, i.e. Mario+Luigi, Mario+Toad, Mario+Luigi+Toad, Mario+Princess+Toad, and so forth for fifteen permutations plus warpless version of each.
In case you were wondering, the answer is "no, obviously not". So that also means that Uno does not require separate runs for 3, 4, 5, and 6-player mode.
I agree that there should only be one branch unless a good argument can be made otherwise, but you haven't said what that branch should be. "Hardest" is an inadequate definition. What number of players and which mode is correct in that example? Why?
Board games are different from other games because they generally do have these kinds of freely editable game settings that have significant effects on difficulty and length, as given with the Monopoly examples above. They're neither the fastest they could possibly be, nor the most difficult. Under Moons, they fall back on entertainment, but is someone made another Monopoly run on a different platform, and Monopoly exists on bloody everything, should they still be using 4 players despite that being neither the fastest nor the most difficult to win?
I'm still in the camp that board games shouldn't be accepted unless they qualify for Moons anyway, since again, most are just navigating a few menus and then manipulating luck a couple times if not just really simple dice manipulation. They're intrinsically not like other games, the existing guidelines are inadequate for handling them, and next to none would qualify for entertaining (with stipulations already in place for the few that do). They belong where they currently are off with the sports and rhythm games.
I'm still in the camp that board games shouldn't be accepted unless they qualify for Moons anyway
I do not think we should be that exclusive. Every game deserves a TAS.
If a tier where board games would fit is ever implemented, I propose this kind of admission guidelines:
1) Different implementations of a specific board game (eg. chess) on a specific platform (eg. NES) are accepted if the author can give a good-enough argument of why the submission is different-enough from existing ones. (Example: Battlechess is arguably different enough from Chessmaster that having a TAS of both for the same platform is acceptable.) However, if a new implementation of that board game is not different enough from an existing publication, then it either replaces the existing publication or it gets rejected.
2) By default the hardest CPU setting should be chosen (regardless of how long it takes). In addition a submission for the same game with an easier setting can be published if it wins the computer as fast as possible and is interesting/entertaining enough.
(Personally I would also disallow memory/savestate corruption and arbitrary code execution, and the reset button, and consider them invalid techniques with such board games because they are not the point, but I know this most probably wouldn't get enough support.)
I feel like if we're creating a whole tier specifically for games that may or may not be the best speedgames, then trying to force arbitrary difficulty guidelines doesn't really make sense. By creating a board/sports game tier, we would be accepting that our standards of acceptance for that tier are vastly lower than other tiers, in terms of speed/entertainment goals. If we were to encounter a case where there were hundreds of combinations of difficulty, character selection, cpu character selections, etc. then we chould just wait until more than one submission happens for that game and discuss that individual case at the time. It's not like anyone is going to submit a run of clue for all ~1200 possible endings, that would be stupid and nobody would waste their time or TASvideos's time doing that. Even if someone were to submit ~10 runs for the same game at the same time, it would be relatively easy to select which of the 10 runs would be the best one to have on the site.
In other words, I guess, I don't see why we have to decide ahead of time which runs for a game are accepted, when we can wait for the submission process to do that.
I agree that there should only be one branch unless a good argument can be made otherwise, but you haven't said what that branch should be. "Hardest" is an inadequate definition. What number of players and which mode is correct in that example? Why?
The same as with every other game on the site:
(1) if there's a difficulty setting, set it to hardest.
(2) pick any other options based on "whatever is fastest".
(3) done, you now have your "any%" fastest branch.
(4) any other combination of options needs to either give a substantially different run, or be entertaining enough to qualify for moon tier.
For example, if an RPG allows you to pick out of six classes for four characters, that's already 1296 combinations, which is way more than most board games. And this is an issue that's already been solved: You pick whatever is fastest.
For example, if an RPG allows you to pick out of six classes for four characters, that's already 1296 combinations, which is way more than most board games. And this is an issue that's already been solved: You pick whatever is fastest.
I'm just more concerned that people would then argue for different branches/modes, and when that/those are made everyone seemingly votes yes, thus going to moons. But then the ratings later reveal its quite crap. :P
This can happen to any game with more than 1 mode/character/etc, but with board games it can be quite glaring.
This argument on "modes" reminds me of the mod Hardcore Deus Ex by hejhujka, which changes Deus Ex: Game of the Year Edition such that each difficulty level is instead an entirely different style of playing the game, resulting in different strategies and different timesavers despite playing the same basic game.
I think that if multiple modes of a game are like that, completely distinct from each other, then each mode which IS distinct would be publishable.
I also agree with Warp's sentiment that if/when board games become allowed, memory/SRAM/savestate corruption and arbitrary code execution should be disallowed; using such things to beat a board game completely misses the point of playing the game in the first place.
As an example, in Battle Chess, if you were to execute a program which changed the alignment of pieces, and before the opening move, you used it to make a black pawn at E7 and the black Queen at D8 become white (which I believe creates a checkmate) it would be an interesting technical achievement, but not a game of chess.
I agree that there should only be one branch unless a good argument can be made otherwise, but you haven't said what that branch should be. "Hardest" is an inadequate definition. What number of players and which mode is correct in that example? Why?
The same as with every other game on the site:
(1) if there's a difficulty setting, set it to hardest.
(2) pick any other options based on "whatever is fastest".
(3) done, you now have your "any%" fastest branch.
(4) any other combination of options needs to either give a substantially different run, or be entertaining enough to qualify for moon tier.
For example, if an RPG allows you to pick out of six classes for four characters, that's already 1296 combinations, which is way more than most board games. And this is an issue that's already been solved: You pick whatever is fastest.
For all your Mahjongs, Risks, Mario Parties, Pokers, etc, (2) would mean one CPU at maximum difficulty and then the rest as colluding humans, or using Clue as a (shaky) precedent, all players as colluding humans. That clearly wouldn't qualify under the "has to stand out from regular play" standard. Where do you push the sliding scale to make it stand out then?
jlun2 wrote:
Radiant wrote:
For example, if an RPG allows you to pick out of six classes for four characters, that's already 1296 combinations, which is way more than most board games. And this is an issue that's already been solved: You pick whatever is fastest.
I'm just more concerned that people would then argue for different branches/modes, and when that/those are made everyone seemingly votes yes, thus going to moons. But then the ratings later reveal its quite crap. :P
This can happen to any game with more than 1 mode/character/etc, but with board games it can be quite glaring.
What I imagine will happen is that it becomes "first in wins," since they're almost to a rule entertainment deficient.
By creating a board/sports game tier, we would be accepting that our standards of acceptance for that tier are vastly lower than other tiers, in terms of speed/entertainment goals.
I don't see how. The core standards of the vault tier are: Play at the highest difficulty (unless you have a very good reason otherwise), complete the game as fast as possible. The "board game tier" would have those exact same requirements.
As for entertainment, it's a rather non-issue for vault, so why should it be an issue for this proposed new category?
By creating a board/sports game tier, we would be accepting that our standards of acceptance for that tier are vastly lower than other tiers, in terms of speed/entertainment goals.
I don't see how. The core standards of the vault tier are: Play at the highest difficulty (unless you have a very good reason otherwise), complete the game as fast as possible. The "board game tier" would have those exact same requirements.
As for entertainment, it's a rather non-issue for vault, so why should it be an issue for this proposed new category?
The entire reason why board games aren't excepted to vault is because they're all too "same-y" and typically not interesting. That's why this discussion is happening instead of just accepting them to the vault. If we're trying to apply vault rules to a board game tier, why are we not just accepting board games to the vault tier instead?
By creating a board/sports game tier, we would be accepting that our standards of acceptance for that tier are vastly lower than other tiers, in terms of speed/entertainment goals.
I don't see how. The core standards of the vault tier are: Play at the highest difficulty (unless you have a very good reason otherwise), complete the game as fast as possible. The "board game tier" would have those exact same requirements.
As for entertainment, it's a rather non-issue for vault, so why should it be an issue for this proposed new category?
The entire reason why board games aren't excepted to vault is because they're all too "same-y" and typically not interesting. That's why this discussion is happening instead of just accepting them to the vault. If we're trying to apply vault rules to a board game tier, why are we not just accepting board games to the vault tier instead?
I think a major part is also that by and large, they don't stand out very well from regular play because the path to the 'win' is trivially obvious and there are so few parts where player input is actually given because 'actions' are strictly and discretely defined instead of being continuous as in most other games, as well as so few compared to things like turn based RPGs which also have strict and discrete actions.
Like Othello here. Even if it was played on the highest difficulty, that doesn't change anything in the judgement given.
http://tasvideos.org/4550S.html
And for all the dice rolling games... manipulating luck on an RNG that has a grand total of 6 outcomes isn't really a wowing bit of superhuman play. Nor is manipulating an AI into making one or two stupid moves, especially on these early systems when they're known to be fairly rudimentary.
Belated edit to emphasize what you said:
If the current rules ARE sufficient for board games, then no additional tiering or categories are needed. Just remove the rule saying "No board games." I don't think they are though, and that's also a little different from the sentiment that every game deserves a TAS.
If we're trying to apply vault rules to a board game tier, why are we not just accepting board games to the vault tier instead?
Tangent wrote:
If the current rules ARE sufficient for board games, then no additional tiering or categories are needed.
Board games are not (generally) vaultable because there can't be any% or 100% completions of them. (See the first post in this thread for a deeper analysis of that aspect.)