I thought I would make a thread specifically for runs that failed to sync on console. There is a page with a lot of old tests here but a lot of those are too old to be relevant to modern development.
Failed cases tend to find hardware edge cases not expected or covered by test ROMs, so keeping track of them seems like a helpful way to drive improvements to emulation and verification. More generally keeping track of what things can go wrong and why helps get a clearer picture of what path development needs to take.
I expect a lot of older TASes to fail to sync just by virtue of being made on old inaccurate emulators, so I intend this thread mainly for new TASes made on accuracy focused emulators.
Maybe my recent DK run can stand as a pseudo-example.
While it has now been console verified, it took extra preparation work (involving pre-seeding the system's wRAM using a different cart) to make happen.
Thus, the inputs in the publication alone aren't always going to work on console.
This results from the fact that BizHawk uses a standard initialization of wRAM (for determinism sake) before the game code is run. However, the game itself uses uninitialized wRAM to seed its RNG as opposed to a preset RNG seed value. As this uninitialized wRAM can vary from console to console (or even on the same console depending on the previous game played), the published movie isn't going to console verify consistently without actively preparing the wRAM before each and every console verification attempt.
This means that even if the game completely syncs a run on the console without issue, shutting the system down and restarting will likely result in a desync on a subsequent verification attempt, as the wRAM most likely won't match what it needs to for the run. Doing multiple console runs in sequence would require additional human intervention beyond the power cycle and TAS start. Namely, between runs the wRAM would have to be actively pre-seeded by cart swapping once again before the system could be power cycled for the TAS run.
By contrast; typically once a NES game has been console verified it can be reasonably consistently repeated without any extra intervention from a human beyond simply power cycling and then restarting the TAS.
EDIT: Sadly, I don't believe this type of desync issue is something that can be rectified from an emulation or TAS creation standpoint. So it doesn't really help much from a development perspective.
EDIT 2: (Much Delayed) My DK run has been verified.
NES games can also have issues with uninitialized RAM, it's not exceptionally rare.
It is good to identify that problem though and ways to work around it for games that encounter it. Just having things written down in an organized way really helps.
Would this be the proper place to, perhaps, rekindle a bit the discussion I started in another thread (that was less relevant to this particular topic) about the fact that, apparently, we consider the emulator the sort of "highest authority" on what is considered a valid legit TAS, rather than the actual console it's emulating. In other words, as long as the TAS syncs and works with the emulator, it's considered valid and legit even if the same TAS fails to sync on the actual console hardware.
This feels a bit contradictory with the sentiments that TASes should at least theoretically be replicable, as is, in the actual console and that, for example, TASes that rely on emulator bugs or faulty emulation are not considered valid.
In the cases where a TAS does not sync in the actual hardware because the emulator does not emulate the hardware 100% correctly, and it's these tiny differences that are the problem, shouldn't this be considered the TAS relying to faulty emulation to work correctly, and thus technically speaking not fully valid?
(Of course there are consoles where it's essentially impossible to make a prerecorded TAS work correctly eg. because of non-deterministic reading times of disc-based mass media, but these consoles could be considered separately from those that are completely deterministic.)
I'm not saying that TASes that have not been console-verified at all should be considered somehow "invalid" or "dubious". They could be considered provisionally legit, unless proven otherwise. However, in the cases where the TAS does not sync in the console without modification, shouldn't this modified version be considered the official valid version of the TAS (with the version that syncs on the console being just a side version for those who want to replay it on the emulator)?
It's a bit counter-intuitive that even in the clear-cut cases where a TAS needs to be slightly modified to sync on the console because of lacking emulation accuracy, it's still the emulator version of the TAS that's considered the official legit version, rather than the console version.
Except in the case where there is a logic difference between emulator and console (like a glitch that cannot be done on console that's exploited in the TAS), I'd consider the easier ability to reproduce, distribute and introspect a TAS in an emulator to be more useful of a quality than the modified version that plays back on console but not yet on any emulator, since far more people have the ability to interact with the former than the latter (though we should of course still make both available). Yes, the console verified file is more accurate but less accessible.
Except in the case where there is a logic difference between emulator and console (like a glitch that cannot be done on console that's exploited in the TAS), I'd consider the easier ability to reproduce, distribute and introspect a TAS in an emulator to be more useful of a quality than the modified version that plays back on console but not yet on any emulator, since far more people have the ability to interact with the former than the latter (though we should of course still make both available). Yes, the console verified file is more accurate but less accessible.
Given that, as far as I know, it's impossible to create a TAS without an emulator, I find the scenario quite unlikely that there would be a console-only TAS with no emulator version.
Given that, as far as I know, it's impossible to create a TAS without an emulator, I find the scenario quite unlikely that there would be a console-only TAS with no emulator version.
Well, it is theoretically possible if crafted in some specific way to do so, just that said TAS would not work on emulator in the end but would work on console. For GB/C TASes, something of note that would differ would be how reads of disabled SRAM (or perhaps no SRAM exists, or some other quirky behavior like this that was recently implemented in GSR) would work. It's fairly understood how it works, the MBC just simply doesn't respond and doesn't send any data, so the read would pull out the last thing on the bus (which would be the last opcode executed). But this value could end up "decaying" into FF (notably, if said read was executed in RAM instead of ROM). And of course, this behavior also differs between consoles. For the Gameboy Player, read executed from ROM is last opcode executed, read executed from RAM (VRAM/WRAM/OAM/HRAM/etc) is FF. For the handheld GBA and GBA SP, it's the same, but for ROM reads the last opcode executed byte can have unset bits randomly set. For the GB and GBC, I believe it just always results in FF no matter what (although research on these consoles for this behavior is VERY limited so take this with a grain of salt). Some other caveats to this (like how pop/ret work out, and some others that really need to be researched), but it's not exactly implemented into emulators (Gambatte always returns FF on disabled SRAM reads and GBHawk returns 00 for ??? reasons). It could be actually possible to abuse this behavior in some glitchy runs, perhaps in some ACE runs or other glitches, and could maybe sync fine on console but not on emulator (this is mostly considering if the ACE is executed right at the end and it just goes to end sequence on console but doesn't work on emulator).
Of course, this is just one example on one system, but it does show it is possible, but practically impossible by chance and would need to abuse known but not yet emulated behaviors.
This feels a bit contradictory with the sentiments that TASes should at least theoretically be replicable, as is, in the actual console and that, for example, TASes that rely on emulator bugs or faulty emulation are not considered valid.
In the cases where a TAS does not sync in the actual hardware because the emulator does not emulate the hardware 100% correctly, and it's these tiny differences that are the problem, shouldn't this be considered the TAS relying to faulty emulation to work correctly, and thus technically speaking not fully valid?
Except this really isn't a contradiction, because the actual rule is not "do not have any inaccurate emulation in your submission", but instead: "Do not exploit emulation bugs". The rules actually also use the phrasing "we generally aim to publish videos that look like they could be played back on the original video game system". Functional similarity is the priority, not pure frame-for-frame matching (which of course may be complicated in many cases).
Technically speaking, emulation accuracy is only one consideration for validity on the site, and not an objective binary pass/fail system. A judge must evaluate whether the emulation bug is significant enough to warrant rejection. A run is not "technically" invalid when it's not emulated properly, because the site's criteria for validity does not include the requirement that the timing perfectly matches a console-verification.
Warp wrote:
Patashu wrote:
Except in the case where there is a logic difference between emulator and console (like a glitch that cannot be done on console that's exploited in the TAS), I'd consider the easier ability to reproduce, distribute and introspect a TAS in an emulator to be more useful of a quality than the modified version that plays back on console but not yet on any emulator, since far more people have the ability to interact with the former than the latter (though we should of course still make both available). Yes, the console verified file is more accurate but less accessible.
Given that, as far as I know, it's impossible to create a TAS without an emulator, I find the scenario quite unlikely that there would be a console-only TAS with no emulator version.
I think focusing on whether it's possible to create a TAS without an emulator misses Patashu's point. Patashu was not appealing to some hypothetical about a movie file that works on console but doesn't have an equivalent emulated version.
Instead, when considering a publication where we have a movie file that plays back on emulator and a console-verified version of that movie that is different, we should prioritize keeping the emulator file as the main publication file (while still keeping the console-verified version available). The emulator file is more accessible because anyone can run it and analyze it. The emulator file is also the file that has undergone judging and usually double-checking for sync by both the judge and the publisher.
Game: Daiku no Gen-san - Robot Teikoku no Yabou (Japan)
Emulator: BizHawk: GBHawk in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie:http://tasvideos.org/userfiles/info/66431762538224618
Description of Desync: On emulator, player character does not beat the level at the end of the movie. On Console, player character does beat the level. This appears to be deterministic and happens in the same way every time.
Research: I have looked at invalid VRAM / OAM accesses, usage of uninitialized memory, and edge cases of IRQ source selection and nothing points to the cause of this desync. I have made an equivalent movie in Gambatte which is cycle-for-cycle the exact same as GBHawk, so no leads there either.
Possible Next Steps: Sprite + x-scroll tests with x-scroll 4,5,6,7 would eliminate one possible source of error.
Status:Open Closed.
Resolution: This was in fact an edge case of sprite + scroll timing, in particular sprite evaluation at x=0. New test ROMs verified the behaviour.
Game: Warioland II (USA) (GB Version)
Emulator: BizHawk: Gambatte (but also behaves similarly in GBHawk) in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie:http://tasvideos.org/userfiles/info/67631410422598803
Description of Desync: On emulator, player character beats the level using out of bounds at ~ frame 50000. On console, there is a desync in the out of bounds section and the level does not complete.
Research: This is a case of trying to read VRAM on the same cycle it is released by the ppu. This is not emulated correctly and currently the details are not known.
Possible Next Steps: This needs an exhaustive test ROM.
Status: Open. The current work around is to simply avoid the glitch, which is cycle exact so is easy to avoid with irrelevant button presses
Game: Avenging Spirit (USA/Europe)
Emulator: BizHawk; Gambatte in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie: http://tasvideos.org/submissions/6943.zip
Description of Desync: On emulator, the player character beats the second boss. On console, the player character fails a jump to avoid the crusher in the second boss' room, resulting in a Game Over.
Research: Adding 1 frame before the second boss fight results in emulator having the same desync as console. 2.5.2 and 2.5.3 dev have the same result. A resync on GBHawk desyncs at the beginning of the first stage.
Possible Next Steps: ?
Status: Closed
Resolution: Adding 1 audio sample to the GBA->GBC offset and changing the floor to a ceiling resulted in successful sync.
Game: Legend of Zelda: Oracle of Seasons
Emulator: BizHawk: Gambatte (but also behaves the same in GBHawk) in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie:http://tasvideos.org/userfiles/info/68126602397597426
Description of Desync: On emulator, Link walks over a pit after collecting a key. On console, Link falls in the pit instead.
Research: I tried changing input latch timings but that made it worse. Comparing video of the desync to emulator, it looks like input is applied one frame to early after the room transition (or the transition takes one frame longer) after Link gets the key. No idea why. Also a complete run of Oracle of Ages syncs on console, what's different?
Possible Next Steps: ?
Status: Closed.
Resolution: Uninitialized WRAM combined with improperly writing SRAM to 0 instead of 0xFF led to the desync.
Given that, as far as I know, it's impossible to create a TAS without an emulator, I find the scenario quite unlikely that there would be a console-only TAS with no emulator version.
There have been a few TASes created without the use of emulators. All you technically need is a sufficiently deterministic environment for running the game, and some way to replay inputs. (Having savestates available speeds the creation of the TAS up a lot, because you then don't need to replay it from the start whenever you change anything, but isn't a technical requirement.)
Input playback, determinism, and savestates are a combination that is mostly only found in emulators, but you can imagine other platforms which have that combination and such platforms are usable for TASing. I think many of the non-emulator TASes were created via taking a game that's fairly deterministic as it is, and modifying the game to patch in savestates and the ability to play back input.
(We also have several published non-emulator TASes: neither libTAS nor Hourglass is an emulator.)
[14:15] <feos> WinDOES what DOSn't
12:33:44 PM <Mothrayas> "I got an oof with my game!"
Mothrayas Today at 12:22: <Colin> thank you for supporting noble causes such as my feet
MemoryTAS Today at 11:55 AM: you wouldn't know beauty if it slapped you in the face with a giant fish
[Today at 4:51 PM] Mothrayas: although if you like your own tweets that's the online equivalent of sniffing your own farts and probably tells a lot about you as a person
MemoryTAS Today at 7:01 PM: But I exert big staff energy honestly lol
Samsara Today at 1:20 PM: wouldn't ACE in a real life TAS just stand for Actually Cease Existing
Game: The Humans (USA)
Emulator: BizHawk; Gambatte in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie: http://tasvideos.org/userfiles/info/68420986554541398
Timestamps: https://cdn.discordapp.com/attachments/183641696788152320/793392162829762580/humans.txt
Description of Desync: https://clips.twitch.tv/ExuberantAlertRatPrimeMe
Research: Initially, this movie desynced immediately due to reading uninitialized HRAM. I made a ROM to initialize HRAM to match Gambatte and hotswapped the cart, and it was able to sync past the title screen. However, it desynced later on as shown in the clip I linked. I'm suspecting some extra lag is around that spot for some reason, inserting a frame around there can cause a similar (if not identical) desync. Uninitialized HRAM doesn't appear to be the cause of this later desync tho, and WRAM is initialized by the game, so that's out of the question.
Possible Next Steps: ?
Status: Closed
Resolution: Adding 1 audio sample to the GBA->GBC offset and changing the floor to a ceiling resulted in successful sync.
Interesting, this is one I actually can look into, I'll be doing so in the next week or two.
Uninitialized HRAM aside, I find it strange for such a slow paced, fairly standard looking game to encounter a desync, will be interesting to see what the problem is.
EDIT: I researched The Humans in this post: http://tasvideos.org/forum/viewtopic.php?p=502659#502659 I'm confident it is input latch error in the the dump script and not emulation related. I think this one can be closed.
Game: Castlevania The Adventure (JPN, GB)
Emulator: BizHawk; Gambatte in GBA mode
Console Verification Device: Gameboy Player with GBI
Movie: http://tasvideos.org/userfiles/info/71190069389254177 (Note, needs dev build to sync, see https://github.com/TASVideos/BizHawk/pull/2665 for the related change)
Timestamps: https://cdn.discordapp.com/attachments/799578989965737994/838286788807884810/cva.txt
Description of Desync: https://youtu.be/iurOwdqTwvc
Research: This game seems to be really finicky when it comes to input boundaries; I ended up making the timestamps lua respond based off of writes to 0xFF00, which seems to be the best regarding getting the input timing right for timestamps. However, this still resulted in a desync a bit later on as seen in the linked video. On another note, this game does not respond the same on Gambatte w/ ELF as GBHawk, indicating an accuracy difference causing different timing between the two.
Possible Next Steps: ?
Status: Open
Game: Who Framed Roger Rabbit
Emulator: BizHawk: NESHawk, local build with new startup state
Console Verification Device: TAStm32
Movie:http://tasvideos.org/userfiles/info/72277515094486081
Description of Desync: Enemies are not in the correct locations after game start
Research: Items do (occasionally, ~1/10 chance) appear in the right place on console. I have confirmed this by stopping playback and then manually checking the locations they should be in, but the enemies are in the wrong place, in a seemingly deterministic fashion. Un-initialized RAM is not the cause as I played the movie several times with random power up RAM state with no desyncs. This is curious as getting the right item locations requires accuracy to the ppu level already, but for some reason after that it doesn't work. I have looked at some trace logs, and the only interesting thing happening is that the game starts to use sprite zero hit once gameplay starts, after item location is chosen, maybe this hints at something wrong with sprite zero hit, but it passes all the relevent tests so I don't know. I tried the same movie on Mesen 0.9.9 but it desyncs in a different way, also not observed on console.
Possible Next Steps: The game also encounters the OAM address glitch, but resets $2003 properly. Not emulating it all does not cause a desync anyway. Never the less thinking what might be wrong with sprite zero hit seems like the most likely way forward.
Status:Open. Closed.
resolution: Testing of power on state revealed the correct settings that give the observed enemy positions and correct item locations.
Game: Bad Dudes
Emulator: BizHawk: NESHawk, 2.6.3 dev
Console Verification Device: TAStm32
Movie:http://tasvideos.org/userfiles/info/73986905672957577
Description of Desync: Initially desyncs because it needs an extra poll on console at the start of level 1, then desyncs in level 2.
Research: This run initially starts from a reset, which might be part of the problem, but additionally the run won't even play back the first level properly unless there is an extra blank poll inserted right before inputs for level 1 start. I don't know why this happens. The game fiddles with the screen more then most games, turning rendering on and off between loading, maybe this is causing spurious polls in the bot? I don't know. If I do add in the extra poll, it syncs level 1 but then desyncs in level 2. Level 2 uses MMC3 IRQs, so this might be related to imprecise reset timing, but combined with the need for the extra poll there are too many variables to tell. The game also polls $2002 after loading between levels (most other games don't do much with it after initial start up) so maybe this is an issue as well if there is some unknown issue with $2002.
Possible Next Steps: The cause of the need for an extra poll has to be understood. Maybe it is related to the need to always start the bot with one blank poll, and why Paperboy needs 3 blank polls.
Status: Open
GB The Addams Family has a ridiculous way of processing sub frame inputs which changes the time takes to start the game.
I discovered this bug using Ilari's lsnesrr1-18e3triangle epsilon 13... whatever version.
I can hardly imagine if that kind of sub frame inputting would be console verifiable, but since nobody ever tried it, I guess this should be the thread to start verifying the bug.
Latest TAS using BizHawk: [2871] GB The Addams Family "warp glitch" by Kaylee in 01:29.51
My sheet from 2014 with inputs and their differences by pressing them at frame 576:
https://docs.google.com/spreadsheets/d/13f9KWy35O3qA6uudvNheh7EtDYv8QrgH1_iFtrBbmKU/edit?usp=sharing
If you need a usermovie, I can create one for you.
edit: I'm not sure which thread should this be posted, feel free to move it to the appropriate thread!
edit2: I have more of these kind of "does it syncs on console? cause it seems weird" issues mainly because of the difference between fceux and bizhawk, namely:
[2340] NES Driar by MESHUGGAH in 05:14.45 (console verified) vs BizHawk User movie #5197490535444251 (I've already asked this since a month maybe but can't find the thread... ?!)
[2373] NES The California Raisins: The Grape Escape by MESHUGGAH in 04:51.50 (not console verified, obsoleted) vs BizHawk (adding 2 frames before the start and 1 frame for starting the level at the level select screen. Can't recall if you need to do this for each level but I guess it's required to).
edit3: notes to myself:
NES Battletoads FCEUX vs BizHawk
NES Kirby's Adventure FCEUX vs BizHawk
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
