Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 194
Location: Midwest
Over the last week I have been working on Genesis/MegaDrive verifications; particularly trying to add decent support to my replay device and writing dump scripts. The way Genesis games read controller inputs is rather quirky, and existing emulators don't provide great support for capturing those inputs with much precision. Neither BizHawk nor Gens gives you any way to see what values the game writes to or reads from the relevant memory mapped registers for the controller ports. BizHawk has a memory callback API, but as far as I can tell, it always returns 0x00 as the value (indicating the core implementation doesn't support it). Gens' callback API doesn't even provide a value parameter, even though the internal function that handles the callbacks, is in fact provided with the read/written value! In the case of Gens, I was able to write a very simple code patch that passes the value to the lua callback which I'll get posted on github soon. Along with some build instructions, because the repo doesn't offer much help, and it appears you are required to use VS2010 which Microsoft doesn't provide a download for anymore. The callback value is important because it is required in order to understand how the game is polling a controller. Different games poll in different ways. There's no special "latch" signal like on the NES. There is something similar, but it can be in two states, and the input values change depending on how many times the state is changed, what the state is, and the delay between each state change. The game is free to write the same state multiple times, or to read multiple times without changing the state. It also appears that for some games, such as Sonic Spinball, Gens may be misinterpreting which frame the polls actually belong to, which further complicates dumping and replay. The current publication of Sonic Spinball in particular may be impossible to replay on hardware; I kept running into weird problems like the above, or the start button not registering inputs properly for no measurable reason. Additionally, there aren't many games I can't test, because I don't have a flashcart for this system, and very few carts exist at all. Everdrive is the only decent one but that costs either $100 or $260 depending on the version. But I'm concerned it may modify the state of the console (similar to the NES one does). I've made my own flashcart design, but I haven't ordered it (or thus tested it) yet.
However! There is one bit of good news. I did manage to verify [570] Genesis Wonder Boy in Monster World by Aqfaq in 42:10.45! My recording setup is a mess right now due to Genesis research, and I'd like to get some input displays. But I should get a recording up for that within a couple weeks; or maybe I'll record a basic one in the meantime just to show it works.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
Awesome! Really surprising that that one would be the first one to get working. Looking forward to more progress, good luck!
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 194
Location: Midwest
The verification I mentioned for Wonder Boy is now up on youtube, and the publication updated. I also have a PCB and the components to hopefully make a flashcart, but won't know if the design is good until I assemble it and experiment with some firmware. While it's true I had problems with other games, like Sonic Spinball, the fact that Wonder Boy, a TAS from 16 years ago, verified without any issues shows that there may be potential for many more verifications on this system. Though of course, no way to know for certain until more games are tested.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
It's a new year, and almost 3 years already since I started this thread, wow that time went fast. That time though saw big advances in emulation and console verification on a number of systems. Here's my synopsis of the current state of R&D. NES: The biggest deficiency left for NES is reset behavior, but few runs use that anyway. Aside from that many / most TASes should be verifiable, even ones that rely on precise power up timing and DMC glitches. Console verification of NES TASes is now pretty standard, and it is far more rare to see one fail than succeed (as long as its being done on NESHawk.) There are some edge cases that are un-emulated in NESHawk, but they are unlikely to ever come up in practice, probably only if some homebrew dev used them for an emulator check. Implementing these would require a major rewrite to NESHawk, and I'm not sure they even effect all hardware revisions. Anyway, NES is in good shape. SNES: Modern versions of BSNES are now in BizHawk, but even so console verification has limited success. Most desyncs are random, very likely due to the audio chip clock, a well known issue. There are also some deterministic desyncs, though with unknown cause. There have been a few successful verifications along the way, and recently I saw that DwangoAC managed to verify a WIP of SMW 96 exits. Surprisingly, there is even new emulation discoveries still happening, as very recently an un-emulated cpu quirk was found. It's hard to say what is needed to advance verification efforts for SNES, it might even take some kind of hardware mod to tame the SPC clock. Deterministic desyncs make me think that something is not right on the emulation side, although it could still be a bot issue, but where to even look? GB/C: Nothing much new has come up that I'm aware of for GB/C, its pretty mature. GBA: Lot's of progress has happened on GBA. Tests from Flevoriux have nailed down most ppu timings and emulation details. Myself and GhostRain0 have found a lot of details related to prefetcher emulation. I've also found a few of other minor emulation details for other system components. All of this effort on the emulation side has led to many new verifications, including of first party games. There is still a ways to go, with some known desyncs and emulation deficiencies. However I'mn hopeful (and pretty confident) that 2024 will see GBA emulation and verificaiton mature to the level of NES and GB/C. N64: A great amount of emulation effort has taken place for N64 since my original post. The N64 core of the Ares emulator is maturing quickly and is increasingly accurate, with several talented N64 devs contributing. Even cache emulation is now close to cycle accurate, it's an impressive feat. I'm not aware of any actual console testing taking place yet, and perhaps it's still early as there are still many issues, but there are exciting prospects. Maybe I'll do some testing myself later this year. DS: While it seems like a likely next target for work after GBA and along side N64, the DS is a very complicated system and much work still needs to be done. Genesis: Bigbass made a new verification pipeline and tools for Genesis verifications and even made the first new verification for Genesis in a long time. This is impressive work and now there is a lot of room for more genesis testing and verification. I believe that sums up all the major developments. Console verificaiton is overall much more mature than it was 3 years ago, in some cases it is even commonplace. Also compared to 3 years ago, a lot of the low hanging fruit is now gone. So, what now? Personally I don't know what I will do once I finish up work on GBA. N64 looks like the only approachable system, though maybe its worth it to take another look at SNES. Then there are basic questions like what do we do about disc based systems? What do we do when clock speeds get too high to emulate systems in a cycle accurate way efficiently? That about sums up my thoughts, anyone else have any thoughts or predictions?
Dimon12321
He/Him
Editor, Reviewer, Experienced player (597)
Joined: 4/5/2014
Posts: 1227
Location: Romania
Thank you very much for your effort for GBA emulation and verifications! A sceptic question, sorry. Did anyone else verified a movie which has been previously verified by you? I'm just worried that you may be developing GBAHawk with only your improvised verificator and ROM re-writer in mind.
TASing is like making a film: only the best takes are shown in the final movie.
TiKevin83
He/Him
Ambassador, Moderator, Site Developer, Player (155)
Joined: 3/17/2018
Posts: 358
Location: Holland, MI
Dimon12321 wrote:
Thank you very much for your effort for GBA emulation and verifications! A sceptic question, sorry. Did anyone else verified a movie which has been previously verified by you? I'm just worried that you may be developing GBAHawk with only your improvised verificator and ROM re-writer in mind.
I've previously cross-verified many of Alyosha's GBC verifications, if we want to double check some of the GBA ones I would just need the GBI style input log and the exact cart to order.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
Dimon12321 wrote:
Thank you very much for your effort for GBA emulation and verifications! A sceptic question, sorry. Did anyone else verified a movie which has been previously verified by you? I'm just worried that you may be developing GBAHawk with only your improvised verificator and ROM re-writer in mind.
I don't think so, but all my files are available here: https://github.com/alyosha-tas/GBA_replay_files and I use original carts except for homebrew. So, anyone can check, but that kind of thing is kind of unavoidable with things this niche.
TiKevin83
He/Him
Ambassador, Moderator, Site Developer, Player (155)
Joined: 3/17/2018
Posts: 358
Location: Holland, MI
Alyosha's DK King of Swing verifications are working great on my setup, I'll post videos as I get time to record them.
TiKevin83
He/Him
Ambassador, Moderator, Site Developer, Player (155)
Joined: 3/17/2018
Posts: 358
Location: Holland, MI
Alyosha wrote:
Dimon12321 wrote:
Thank you very much for your effort for GBA emulation and verifications! A sceptic question, sorry. Did anyone else verified a movie which has been previously verified by you? I'm just worried that you may be developing GBAHawk with only your improvised verificator and ROM re-writer in mind.
I don't think so, but all my files are available here: https://github.com/alyosha-tas/GBA_replay_files and I use original carts except for homebrew. So, anyone can check, but that kind of thing is kind of unavoidable with things this niche.
@Alyosha I experienced a repeated desync in DK King of Swing, do I need to flush the saveram or do some other kind of ram stabilization before running that TAS? I was clearing the saves with the in game menu
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
All SRAM is cleared to 0xFF, unless of course it is an SRAM anchored movie , which the Diddy mode run is.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
We just reached 350 console verified runs! That is out of 3335 currently published runs, so managing to stay above 10%. I think 400 is reachable without too much trouble, mostly just a matter of time. Past that I think resyncability of old runs will start to become a major hurdle on the road to 500. I also have almost 10% of all GBA runs verified, so that's pretty neat. I'm hopeful to reach 50 total this year, seems doable. The biggest variable here is long it will take me to track down remaining accuracy issues. So onward to 400.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
2 months since my last post and now we are at 365 runs console verified. Pretty steady progress. I suppose we could easily pass 400 with the numerous Action 52 runs alone, but I don't have any way to play those back myself. At least on my end I expect progress to continue at roughly the same pace. Soon I'll be looking at Mega Man Zero I hope those are syncable.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
We just hit 400 current console verified TASes! Another hundred milestone hit about 14 months after the 300 one, relatively quick progress. Despite this being a big numerical milestone, it all is still coming from the same systems. There is however a lot of interesting development happening that could lead to some verifications on newer systems in the not so distant future. There is a big PR open in melonDS ( https://github.com/melonDS-emu/melonDS/pull/1955 ) that implements a lot of the complicated cache stuff. Seems to fix some timing issues so it could be a big step forward in DS accuracy once it gets merged. Looks like it still needs a bit of work but pretty exciting to see such a complicated thing being worked on. DS seems like it is the next likely console where a mature console verification pipeline can be built up. I'm looking forward to seeing this happen. The N64 situation in Ares is also looking good. N64 has RDRAM initialization though, which could impact determinism to some extent. I'm no expert so I don't know how bad it could be. Obviously some runs sync reliably on N64, but this might not extend to more of the library. If anyone was watching the Team 0% stuff in Mario Maker, they know that the last level was uploaded using TAS playback tools for the Wii U. I found this really surprising and it's cool too see stuff like that come out of nowhere. I haven't researched any of the technical details, but presumably the inputs were hand crafted similar to the Mario Maker 2 demo by TASBot folks. Maybe this approach has some potential for longer runs with some more work to make it less painful? I personally don't have time to start any new big projects, so I'll be grinding out GBA verifications for the foreseeable future. There are still a lot I want to do. Onward to 500,
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 194
Location: Midwest
Nice job Alyosha! Also, if you count obsoleted TASes, we're currently at 521 verified publications. :D
As for the N64.. some in-depth testing needs to be performed to really know how viable the system is for verifications at this time. SM64 and a couple other games are verifiable already, even with the much less accurate mupen64 core. But this appears to be because the way those games poll and make use of controller input, makes them unaffected by timing inaccuracies, at least as far as the replay device's ability to stay in sync. If you actually measured the realtime it took to replay, I'm certain it would be different than the emulator (based on what Ridley has explained to me, through her efforts regularly verifying SM64 TASes.) I suspect it might be possible to "scan" the N64 library of games with a Lua script in BizHawk, to try to predict games that are similar to SM64. But you'd have to manually play each game to induce some lag into the test. RDRAM initialization shouldn't affect anything. Maybe if a game depends on the initial state somehow? but I haven't heard of any N64 game doing that, and the initial state of RDRAM may actually be deterministic (needs more research). If necessary, it may be possible to initialize at least most of the system to expected values before starting the game/TAS replay.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3828)
Joined: 11/30/2014
Posts: 2834
Location: US
Bigbass wrote:
Nice job Alyosha! Also, if you count obsoleted TASes, we're currently at 521 verified publications. :D
As for the N64.. some in-depth testing needs to be performed to really know how viable the system is for verifications at this time. SM64 and a couple other games are verifiable already, even with the much less accurate mupen64 core. But this appears to be because the way those games poll and make use of controller input, makes them unaffected by timing inaccuracies, at least as far as the replay device's ability to stay in sync. If you actually measured the realtime it took to replay, I'm certain it would be different than the emulator (based on what Ridley has explained to me, through her efforts regularly verifying SM64 TASes.) I suspect it might be possible to "scan" the N64 library of games with a Lua script in BizHawk, to try to predict games that are similar to SM64. But you'd have to manually play each game to induce some lag into the test. RDRAM initialization shouldn't affect anything. Maybe if a game depends on the initial state somehow? but I haven't heard of any N64 game doing that, and the initial state of RDRAM may actually be deterministic (needs more research). If necessary, it may be possible to initialize at least most of the system to expected values before starting the game/TAS replay.
I was considering starting N64 testing myself with the Ares core, but my impression is that I should probably wait for a few more updates as it's not quite there yet. For NES, GB/C, and GBA, pretty much every edge case emulation detail has been necessary for console verification for at least some games. Assuming this extends to other systems as well, it's hard to dedicate a lot of time to testing before a high degree of accuracy is guaranteed from the emulator. But not being an N64 dev myself it's also hard to judge when that point is reached. Still maybe it would be a good idea to make some simple test runs for games like Goldeneye or Rogue Squadron to check if replay is at least deterministic for games developed by other publishers.