Posted: 12/1/2005 1:37 AM
Post subject: speedrun vs. TAS stats
Quote
tool23
Player (80)
Joined: 2/8/2005
Posts: 130
I don't know if this has been mentioned in another thread somewhere, but I was thinking it might be interesting to do a statistical comparison between console runs and TASes so that when doing runs of games that have never been done before we could have an estimate as to how much time difference should be expected between the two types of runs. Then maybe type of game could be figured in - mario is 2% faster when TASed (300 seconds vs. 306 s), and zelda is 34% faster (1529 s vs. 2044 s).
Posted: 12/1/2005 1:44 AM
Quote
Dacicus
Editor, Player (69)
Joined: 6/22/2005
Posts: 1050
One problem with establishing "expected times" in that manner would be that TASed games within the same category could vary in time saved very much. For example, one game might have way more programming errors that could be abused than another. If any such system does come into play, I think the TASes should be limited to using just the glitches that are known and can be performed on the real console, to keep it somewhat fair.
Current Projects: TAS: Wizards & Warriors III.
Posted: 12/1/2005 2:56 AM
(Edited: 5/21/2006 3:49 PM)
Quote
tool23
Player (80)
Joined: 2/8/2005
Posts: 130
Taken from this site and SDA except for sonic games. Sonic games are compared by adding total level times together, and console stats are from TSC. Numbers in seconds. _________________________console -- TAS -- difference NES Battletoads warp 1p ____1452 -- 808 -- 80% NES Battletoads no-warp 1p _2201 -- 1460 -- 51% NES Bionic commando ________1100 -- 846 -- 30% NES Castlevania _____________793 -- 701 -- 13% NES Castlevania 2 _________2925 -- 1956 -- 50% NES Castlevania 3 Alcuard __1787 -- 1557 -- 15% NES Castlevania 3 Sypha ___1975 -- 1712 -- 15% NES Chip 'n dale 2p __________840 -- 548 -- 53% NES Contra _________________685 -- 574 -- 19% NES Duck tales _______________621 -- 491 -- 26% NES Gauntlet _______________1213 -- 745 -- 63% NES Ironswd: WizWar2 ________780 -- 509 -- 53% NES Jaws ____________________288 -- 259 -- 11% NES Kid Icarus ______________2024 -- 1436 -- 41% NES Lolo2 _________________1754 -- 1451 -- 21% NES Marble madness ________193 -- 171 -- 13% NES MM2 __________________1839 -- 1535 -- 20% NES MM3 __________________2708 -- 1913 -- 42% NES MM4 __________________3048 -- 2111 -- 44% NES MM5 __________________3084 -- 1984 -- 55% NES Metroid 100% _________3143 -- 2503 -- 26% NES Rygar no-warp ________1830 -- 1227 -- 49% NES SMB warps US____________306 -- 300 -- 2% NES SMB warpless US_______1278 -- 1138 -- 12% NES SMB2 warps US_________582 -- 474 -- 23% NES SMB 2 warpless US _____1593 -- 1214 -- 31% NES SMB3 warps _____________671 -- 635 -- 6% NES TMNT ___________________1172 -- 1085 -- 8% NES Who framed RR __________365 -- 308 -- 19% NES Zelda1 _______________2044 -- 1529 -- 34% NES Zelda2 noglitch _______4330 -- 3239 -- 34% SNES Aladdin _____________1103 -- 1014 -- 9% SNES Contra 3 _____________1020 -- 820 -- 24% SNES DKC 101%______________3000 -- 2845 -- 5% SNES FF MQ _________________9860 -- 7778 -- 27% SNES MM7 __________________ 3569 -- 2335 -- 53% SNES MMX __________________ 2170 -- 1844 -- 18% SNES MMX3 _________________2679 -- 2295 -- 17% SNES SMRPG _______________14220 -- 9626 -- 48% SNES SMW fastest ____________656 -- 636 -- 3% SNES SMW 100% _____________5430 -- 5018 -- 8% SNES SMW2 _________________7094 -- 5937 -- 19% SNES TMNT4 ________________ 1351 -- 1135 -- 19% SNES Metroid (in-game time) 36 -- 27 -- 33% SNES Metroid (100%) 55 -- 39 -- 41% SNES Zelda Alttp (no glitch fast)5987 -- 4816 -- 24% GEN Gunstar Heroes _________2475 -- 1979 -- 25% GEN Quackshot _____________1920 -- 1404 -- 37% GEN Sonic 1 ________________868 -- 762 -- 14% GEN Sonic 2 ________________1076 -- 823 -- 31% SGB Pokemon Blue __________9600 -- 6667 -- 44% GBA Metroid Fusion __________3000 -- 2520 -- 19% GBA Metroid ZM _____________2742 -- 2156 -- 27% GBA Sonic Advance 2 _________899 -- 637 -- 41% N64 Mario 16*_______________1187 -- 987 -- 20% Average 28% STDEV 17% So expect a 11 - 45 % difference. Smallest difference - SMB (warps). Biggest difference - NES Battletoads warp 1p Not included: NES Metroid best time ______1115 -- 566 -- 97% NES Zelda 2nd Quest _______2683 -- 1756 -- 53% (The TAS uses up + A and the SDA runs don't) NES Deadly Towers _______2590 -- 920 -- 182% (I took this out to make the numbers look prettier. I believe the two runs use the same route, and there aren't any big timesaving glitches used in the TAS, its just that the precision of the TAS allows for that much time to be saved on this game.) link to file (I have StarOffice, but it should work on XL) I'll keep updating it as new movies come out.
Posted: 12/1/2005 6:02 PM
Quote
Truncated
Editor, Reviewer, Experienced player (980)
Joined: 4/17/2004
Posts: 3109
Location: Sweden
Arcade TAS of 2023 Sega TASer of 2021 Sega TAS of 2021
The improvement% does not seem to be percent at all. It looks like tastime/consoletime, instead of (consoletime-tastime)/consoletime. Am I missing something here? Where did you get the times? Most speedrunning sites count time a bit differently from us, excluding intro screens and such, but in long runs it shouldn't make that big a difference.
Posted: 12/1/2005 6:41 PM
Quote
JXQ
Experienced player (761)
Joined: 5/6/2005
Posts: 3132
NES TAS of 2008 Speedy TAS of 2008 SNES TASer of 2007 TAS of 2007 SNES TAS of 2007 Funny TAS of 2007
From what I can tell, TAStime * improvement% = ConsoleTime. So (Improvement%-1) * 100 is the actual improvement in percentage form.
<Swordless> Go hug a tree, you vegetarian (I bet you really are one)
Posted: 12/1/2005 7:02 PM
Quote
tool23
Player (80)
Joined: 2/8/2005
Posts: 130
JQX is right, its more like a ratio rather than %, i'll edit that. The times are from SDA, I haven't looked at other sites yet (are there a lot of games done on other sites that aren't represented here?). I guess if the times are calculated differently on SDA than here, the numbers are more of an estimation. Some games I went by in-game clock, but most I just took the time given on the website. I took Dacicus' suggestion into account and left out the zelda glitches. I think things causing differences might be the number of hi-precicion moves (Battletoads, Metroid), and manipulation of luck (RPGs).
Posted: 12/1/2005 8:21 PM
Quote
Bag_of_Magic_Food
He/Him
Player (206)
Joined: 5/29/2004
Posts: 5712
Didn't Xebra say that comparing times like that makes people hate us?
put yourself in my rocketpack if that poochie is one outrageous dude
Posted: 12/1/2005 8:28 PM
Quote
Dacicus
Editor, Player (69)
Joined: 6/22/2005
Posts: 1050
Maybe, but we still do it to some degree. For example, the TASes aren't allowed to be longer than the world record.
Current Projects: TAS: Wizards & Warriors III.
Posted: 12/1/2005 10:40 PM
Quote
Graveworm
Former player
Joined: 9/29/2005
Posts: 460
Bag of Magic Food wrote:
Didn't Xebra say that comparing times like that makes people hate us?
Some "people" already hate you, so it doesn't really matter.
Posted: 12/1/2005 10:58 PM
Quote
OmnipotentEntity
He/Him
Player (36)
Joined: 9/11/2004
Posts: 2631
However, this is actually useful. It gives an expected time of improvement for various games. It would be admittedly far more useful if it had a smaller standard deviation, but I expect that once game length and genre are factored in that'll settle down.
Build a man a fire, warm him for a day, Set a man on fire, warm him for the rest of his life.
Posted: 12/2/2005 6:49 AM
Quote
nitsuja
Emulator Coder, Skilled player (1311)
Joined: 12/21/2004
Posts: 2687
Computer TASer of 2011 TAS of 2011 First edition TAS of 2011 Computer TAS of 2011 Sega TAS of 2010 Sega TAS of 2008 Sega TAS of 2007 Glitchy TAS of 2007
I don't see how this could be all that useful. It can't give better than a rule of thumb in general, and do we really need something like this to tell us how impressed we should be at how fast a TAS is? It's somewhat interesting for the purpose of comparing different games, but not much else.
Posted: 12/2/2005 7:19 AM
Quote
Dada
Joined: 11/22/2004
Posts: 1468
Location: Rotterdam, The Netherlands
Like nitsuja, I also don't see how this has any value.
Posted: 12/2/2005 4:15 PM
Quote
tool23
Player (80)
Joined: 2/8/2005
Posts: 130
I did it just cuz I thought it would be interesting to see, I'm not out to prove any point about how cool TASing is. I think both SDA movies and movies on this site are equally really cool, and I don't think either one counts are more 'pure' gaming than the other. If anything this little excercise shows that stats cannot replace a 'gut feeling' estimate of how much time can be saved given by people on this site who have a lot of experience doing TASes. I think so many factors go into it that only by playing the game and internalizing all of its particulars can someone estimate much time can be saved. After all, who can explain why the 2 zeldas on the list are both exactly 34% faster, but the 2 megaman games are 20 and 47% just by looking at the stats alone? But the stats do give a rough benchmark for people to make their gut feeling estimates against.
Posted: 12/3/2005 8:39 AM
Quote
todd
Joined: 4/4/2004
Posts: 66
Thanks for the stats. I found them very interesting, even if they may not have a practical use.
Posted: 12/5/2005 8:38 AM
Quote
tmont
Player (52)
Joined: 10/6/2005
Posts: 138
Location: Oregon
I like stats, too. I thought they were cool to see, even if they aren't useful. Also, the Crystalis run up on SDA doesn't use the wild warp, so those two runs can't be compared (fairly, anyway). If you really want something to compare TAS Crystalis run to, I did a crappy run a while ago of Crystalis using the wild warp in about an hour (3600 seconds). It's not hosted anywhere; I just did it for fun.