Post subject: Console Verification R & D
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I am making this thread for discussion about console verification research and development, what is currently happening and what is needed for advancement. Here is my synopsis of the current state of console verification (that I am aware of) for various systems and what next steps might be needed to move things forward. I won't go into detail for disc based systems, as likely the first step for a lot of those is replacing disc drives with something deterministic. NES: I think many more NES games can be console verified with current technology. I think the limitation right now is in understanding power on characterization. Improvement here just needs the time investment to do the many tests that it takes to understand the details. I had started doing this with true a few years back but didn't get anything concrete. Admittedly there probably aren't many runs left that would motivate that kind of investment, maybe Punch Out? SNES: My understanding is that the clock of the SPC700 is one of the major obstacles to verifying more games. Also the version of BSNES used in both lsnes and BizHawk is pretty outdated, but I'm not sure how much would actually be gained from updating it. Even sorting out these things, I'm not sure how well the various expansion chips are emulated, which many of the more noteworthy SNES games use. The SA1 memory access timing test still fails in the most recent higan, but I don't think it's known if passing it is even required for verificaiton of e.g. Super Mario RPG. It's probably a long road ahead for SNES. GB/C: I'm pretty confident most games can be verified with GBI. Even when they fail, the pipeline is mature enough that problems can be tracked down quickly. However all of this only applies to relatively standard TASes. Pokemon Yellow ACE for example requires subframe inputs at a higher resolution then GBI can supply. A new replay device would be needed for that. There may be a couple linked play TASes worth verifying that might need some new replay device work as well, and other edge cases like Kirby Tilt 'n Tumble. GBA: Probably some more games can be verified, maybe with some manual adjustment like in SMB3, I think there just needs to be more trials. Beyond that though it is still early days for console accurate GBA timing in many cases, especially ppu, so games requiring careful timing are unlikely to work. But how many of those are there? Currently there isn't enough data to tell, really just need to throw more runs at GBI. N64: Similar to GBA, probably there are more games like Mario 64 and Mario Kart 64 that will work even with current emulation (mupen) that just need to be tried to find out. However it's well known for notable games such as Goldeneye that current emulation is not close enough to console to work. New emulator needed. CEN64 is the closest but even that doesn't do things like cache emulation carefully yet. The huge performance cost perfect memory timings would bring though would make TASing full games tedious, maybe there is a 'good enough' middle ground. Genesis: Not much work has been done with Genesis recently, so it's hard to characterize. Probably Gensplusgx is good enough for many/most games. Again, more trials are needed to see. DS: Not sure how much previous success with Brain Age carries over to other games. Maybe Melon DS in BizHawk can spur more development, but it's still early. Other older cartridge based systems: Need someone interested to put in the ground work, likely not much interest though. Overall I would say that currently many more runs can be verified with current technology, but probably not the ones that would be the most high profile / important. In most cases the hardware side (replay devices) is way ahead of the software (emulation) side. Even where emulation is strong though (NES), the verification pipeline is not nearly as mature as it could be, so more fine tuning is needed. I'm going to try to put more work into NES / SNES later this year, hopefully the success with GB/C can be reproduced. Anyone have any other thoughts? Am I missing anything?
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
I'd like to provide some updates and additional information that I know of regarding some of the named consoles, as I've been working on my own replay device recently. NES: There's definitely a lot of NES games that are available to be verified. ViGreyTech and myself have very recently begun to grind out verifications for this console. Some games are much harder to verify than others, and it's not always easy to nail down what is causing a desync. The easiest games are those that don't employ DPCM (Delta PCM audio), don't have their reset vector point to the same location as power-on, don't use save files (aka SRAM), and don't base their RNG on uninitialized system RAM. Some of these hurdles can be overcome more easily than others. DPCM will cause the console to poll input multiple times in one frame. The replay device must be able to detect these extra latches, and repeat the inputs, instead of advancing a frame as it normally would. This is also known as "window filtering" or "latch windows". The typical period allotted to a window is about 8ms, which is about half a frame. Meaning any latch+clock polls from the beginning of the frame, over 8ms, will get the same input values. Games that use save files or save data of some kind, can be verified by either using a programmer to clear memory, using a flash cart, or maybe by removing the battery (if it is battery-backed). Some games may work by just deleting the save file normally in-game, but this doesn't always clear all necessary data. On some games, when you press the console reset button, it will reset to a point after the splash/intro screen(s), which usually desyncs the run. This is trickier to get around, and the solution depends on the replay device in use, and how you are loading the game (physical cart vs flash cart). On a real/repro cart, if the replay device supports it, you can start from console power-on. Flash carts are much trickier. I believe it's possible to start a run from the menu of a flash cart, if the right number of blank frames are inserted before the actual TAS, or otherwise detecting when the game actually starts (such as looking for long periods of no input polls). On the Everdrive N8 for example, I've noticed that a different amount of input frames will happen depending on which game you play. Uninitialized system RAM is the hardest difficulty. ViGreyTech has developed a ROM that can initialize RAM, and then hold the console in a loop so that the user can swap out that ROM with the real game cartridge, and reset from there. However, it hasn't been tested whether this would work when using a flash cart for the game. There may also be some way to change the flash cart's firmware/software to initialize RAM just before starting up a game. There is also a chance for random, extraneous, clock pulses during an input poll. This is fixed by using "clock filtering". Upon seeing a clock pulse, a replay device will ignore all new clock pulses for a set amount of time. The value needed can vary. Usually 2-4us is plenty. N64: Like Alyosha said, this console really boils down to inaccurate emulators. The only reason that SM64 and MK64 are reliably verifiable, is because of a rare way in which they handle inputs. While the N64 effectively uses a single clock source (though the CPU is scaled up from it), there are multiple components which can run different tasks at the same time as each other. But most importantly, the way laggy frames are handled can easily desync runs if emulators aren't accurate enough. According to fraser from the N64 homebrew community: The SDK used to create N64 games will use the Video Interface interrupt to trigger a request to get the input state of controllers. Ideally this would happen 60 times per second (60 FPS, or 16.66ms per frame), but in practice it turned out to be more like 20-30 FPS when lag frames are taken into account. The request will be sent to the Peripheral Interface (aka PIF), and returned to system RAM during a different interrupt. While the PIF does that task, the CPU and RCP will be at work with the rest of that frame's processing. If the frame takes too long to render (say, 17ms), the frame before that, will be rendered again. When this happens, the frame after that, will still poll for input again. At the end of the frame, the previous (17ms) frame's render will finally be displayed. However, because the 17ms frame used the input from when that frame first started, the following frame's input is essentially discarded. Certain games, like SM64 and MK64 will not poll multiple times after laggy frames, thus lag cannot cause discarded inputs. Bottom line is, either emulators must become far more cycle accurate, or we have to figure out some way to detect when these lag frames occur, and thus which inputs are discarded. The former is likely the most viable option. There are two emulators I know of that are in development, and could become accurate enough for TASing purposes (https://ares.dev/ and https://github.com/Dillonb/n64/). CEN64 is not, and will likely never be accurate enough for TASing. It claims to be cycle accurate, but it isn't. MAME is likely the most accurate emulator right now for the console, but to what extent I don't know, and it doesn't have TASing tools.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
Oh also, for the GB/C/A, I believe it should be possible to verify these games without the GBI/GBP, using the on board clock used by the CPU. The TAS input dump scripts would likely (hopefully) be very similar to those used for GBI, but would forego the additional calculation to convert to audio PWM pulses. I plan to look into this further and try to verify using an actual GBA (SP). I'd like to look into the Genesis eventually as well. If there are any particular TASes for the NES that anyone feels are rather high profile, or any they simply are interested in seeing verified, please feel free to ask me here or on Discord.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Awesome I'm glad to hear about more NES console verification work happening. Please do Mike Tyson's Punch Out if you can. The previous TAS sycned on both NESHawk and Mesen but desynced on console, I don't think the current TAS has been tried though. Also Bionic Commando would be a good one. This one didn't work previously due to bot limitations (interrupt fired mid polling routine and the bot timed out the request if I remember correctly) so it would be a good test case. That's some good info on N64. That's probably the one I have the most interest in, even just characterizing which games can work with current technology would be a big step.
Amaraticando
It/Its
Editor, Player (159)
Joined: 1/10/2012
Posts: 673
Location: Brazil
SNES: My understanding is that the clock of the SPC700 is one of the major obstacles to verifying more games.
I've seen this. The game has to sync with the SPC700 and wait for it to complete its main loop or some routine. When it happens, the duration might vary between consoles or even between powerons/resets, making this a major cause of desyncs. Usually happens when the game lags or not depending on this small timeframe. The TASer might be able to manipulate the uncertainty to always lag or always not lag in various situations though, by attempting different sounds in previous segments of the run or by delaying/sacrificing 1 frame.
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
Alyosha wrote:
Please do Mike Tyson's Punch Out if you can.
I'm happy to report that I tried this, and it has verified! You can watch the recording here: https://www.youtube.com/watch?v=X2IU8S_CB_A
Alyosha wrote:
Also Bionic Commando would be a good one. This one didn't work previously due to bot limitations (interrupt fired mid polling routine and the bot timed out the request if I remember correctly) so it would be a good test case.
Unfortunately, this TAS consistently desyncs at the same spot each attempt. It's at about 8:52 of the movie's encode video, where the player is to move the menu cursor to the right, to select the "Transfer" option. On console however, the cursor doesn't move, thus selecting the wrong option. I don't know exactly what would cause this. Perhaps an emulator inaccuracy, or something wrong with my replay device (which is always a possibility).
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Very Cool! I've always wanted to see Punch Out work on console, nice to see it finally happen. 8 minutes is further then previous attempts have made it on bionic commando, which I seem to remember only made it to around 5. I think I recall testing that run on Mesen also and it worked out fine, so emulation shouldn't be an issue. But it's also been too long now for me to give any accurate details about it. Seems like you are making rapid progress on NES though, cool stuff! Thinking back to when I was briefly working on NES verifications with true, the biggest mystery seemed to be the Streemerz runs. They don't use the DMC channel, so it seemed like they really should work on console, but we could never get them to. Not sure what your goals are but if you want a challenge that is probably the biggest (non-DMC related) issue out there. Also the recent Battletoads runs would be cool to see, but they would only work from power on, and then will only work occasionally, but should be doable.
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
Alyosha wrote:
Thinking back to when I was briefly working on NES verifications with true, the biggest mystery seemed to be the Streemerz runs. They don't use the DMC channel, so it seemed like they really should work on console, but we could never get them to.
I just attempted the "Superb Joe mode" TAS (cuz it's short). It desyncs in a few different spots. Either it'll select the wrong option in the start menu (the wrong option can vary between attempts), or it'll desync on the second screen of the first level. First thing to note is that when I probe the controller port (using either a LA or scope), there are exactly four latches per frame. (DPCM is usually three latches). It could be DPCM, though it also could just be something that game is purposefully doing for debugging or it's own testing of some kind. I went back to FCEUX. I changed the PPU option to "New PPU", loaded the inputs into the TAS editor, and discovered that it desyncs exactly the same way as it does on console (when the console manages to get into the first level). Then went to bizhawk to dump the other TAS, and that desyncs on bizhawk 2.6.1, but syncs fine on 1.11.1. I dumped it anyways and tried to verify it... As I expected, it desynced on console. In particular, I had to clear cartridge save RAM for each attempt or it would desync by selecting the wrong mode. But then it would also, consistently, desync at 2:35 (encode time) by dying. Interestingly, this is vastly different than what the more recent version of bizhawk expected. TL;DR: The existing TASes for this game cannot be verified as they were created on inaccurate emulator versions.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Here are the two runs resynced and working in the most recent BizHawk: http://tasvideos.org/userfiles/info/36428673262038989 http://tasvideos.org/userfiles/info/70450743255778683 Here is a version of one of the runs that works in mesen 0.98, but not BizHawk (current mesen (0.99) doesn't appear to load .bk2 files anymore): http://tasvideos.org/userfiles/info/53160118032498328 I'm not sure if we ever did anything with SRAM in the previous testing so maybe that has something to do with the problems we had? Also these will only work from power on and again only sometimes. I had looked at the code before, and it doesn't run any DCM stuff, it just reads 4 times, maybe the controller code is just meant to be used for different projects possibly using it. Thanks for the testing so far!
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
Alyosha wrote:
Here are the two runs resynced and working in the most recent BizHawk: http://tasvideos.org/userfiles/info/36428673262038989 http://tasvideos.org/userfiles/info/70450743255778683
I'm sorry to say but these both desync inconsistently. Which means we could be looking at uninitialized system RAM being used for RNG or possibly some other unknown cause. Thanks for resyncing them in emulator anyways. I will note that they still require clearing cartridge save RAM before each attempt, or else it will likely pick the wrong main menu option.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Oh well, thanks for trying. Pretty sure we looked at RAM state before and it didn't use any uninitialized. I'll get those working someday.
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
I'd like to give a couple updates/discoveries regarding the NES. The first doesn't exactly affect verification, so far as I've seen, but I still feel it's important to write here. The Everdrive N8 (idk if this applies to the Pro version), appears to have a bug of some kind where if all 8 buttons are pressed for any given frame, the execution of the game will pause for approx 12-13 frames. The time period is consistent down to only +/- 0.002ms, however it's hard to tell where in the frame the pause starts. The pause causes the screen to render a solid color (so far I've only seen either black or white), while also producing zero controller latches/clock pulses, and stopping the audio. The pause will only occur on frames where the previous frame's input is anything except all 8 buttons. So any successive 8-button inputs do not cause successive pauses. As far as I can tell, this is not a save state issue as that feature is disabled. Most importantly though, TASes will still consistently verify despite the pauses (though this perhaps may not apply to all games). It's something to be aware of while verifying TASes. I first discovered this from the Mega Man 2 zipless movie where the author had some fun extra inputs during transition screens (you can see here at about 10:03 what it looks like). The issue has been repeated using test movies on multiple other games. However, when the same test is run on a physical cartridge, the issue doesn't exist; indicating that the flash cart is at fault. Just in case, I also tested without the input display powered (even though it runs on an entirely separate power supply); which didn't change any results. ----- The second thing is FCEUX. The newest version, 2.3.0, has a new feature that has helped me to better understand why many TASes will inconsistently desync, beyond just an educated guess. It allows you to randomize the initial system RAM each time you load a ROM (or re-load the same ROM). Testing a movie with this setting is a bit tricky but possible. With this feature, some movies that would normally sync consistently, now desync inconsistently. One obvious example is Mickey Mousecapade, where the enemies spawn in different locations. Before this, we had to just guess, based on repeating playback attempts alone, whether RNG was the cause for desyncs or not. With this, we can tell with a lot more certainty, whether uninitialized RAM and RNG play a factor in verifying any TAS made with FCEUX. In order to use the feature with existing movies, you cannot load movies how you normally would; as it will reset the initial RAM setting to static default instead of random. This is similar to how the Old vs New PPU mode works when loading movies, whether through the File menu or TAS Editor. First, set the RAM mode: Config > RAM Init > Random. Then open the ROM you want to test. I suggest opening the Hex Editor from the Debug menu at this point just to help identify RAM values. At this point you can see the random values if you pause emulation (NES > Emulation Speed > Pause), and re-open the same ROM over and over. You should see different initial values in the hex editor each time. There are three ways to play a movie. The normal way is to use the File > Movie menu, but this will reset the RAM Init option, and reinitialize RAM with the default static value. Instead, open the TAS Editor from the Tools menu. From here there's two ways. First is to use File > Open, and while this works, it will still reset the RAM Init option, but it won't reinitialize RAM. The safer way is to instead use File > Import Input. This imports only the input of the movie and doesn't touch emulator options at all. From here you can play the movie normally and see if it syncs or not. In order to reinitialize RAM again to new values, you need to close TAS Editor, and re-open the ROM. For whatever reason, you can't open the same or different ROM while the Editor is open. Again, check the Hex Editor to make sure the values have changed and redo the Import Input process.
TAS Verifications | Mastodon | Github | Discord: @bigbass
TiKevin83
He/Him
Ambassador, Moderator, Site Developer, Player (155)
Joined: 3/17/2018
Posts: 358
Location: Holland, MI
I implemented 41 TAS verifications as a test suite for Gambatte Speedrun, the goal being to note any deficiencies outstanding in Gambatte Speedrun and to encourage TAS verification as part of an emulator's automated test procedures in general. https://tikevin83.github.io/gsr-automation-site/
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
TiKevin83 wrote:
I implemented 41 TAS verifications as a test suite for Gambatte Speedrun, the goal being to note any deficiencies outstanding in Gambatte Speedrun and to encourage TAS verification as part of an emulator's automated test procedures in general. https://tikevin83.github.io/gsr-automation-site/
That's pretty cool! Clever to feed back in the GBI inputs. I wonder what happened to Oddworld II though? Any idea where it started breaking? EDIT: looks like the movie still works in dev build (after adjusting for the new power up timing.)
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 783
Location: California
Alyosha wrote:
TiKevin83 wrote:
I implemented 41 TAS verifications as a test suite for Gambatte Speedrun, the goal being to note any deficiencies outstanding in Gambatte Speedrun and to encourage TAS verification as part of an emulator's automated test procedures in general. https://tikevin83.github.io/gsr-automation-site/
That's pretty cool! Clever to feed back in the GBI inputs. I wonder what happened to Oddworld II though? Any idea where it started breaking? EDIT: looks like the movie still works in dev build (after adjusting for the new power up timing.)
Considering how Winnie the Pooh also ended up not syncing at all, it's suspected that there's some flaw with how GBI inputs are played back. Possibly due to Gambatte's runFor overshooting a bit too much occasionally (which should be correcting itself for future runFors, but that might not matter if input overshoots just once).
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
A few updates / thoughts: NES: Power up timing and state is pretty much resolved now. It turns out NESDev folks had already worked out DMC state last year, so that was cool. I just need to do a couple hardware tests now to sort out the last couple details. I also tried matching up to Mesen, and it looks like Mesen state should be -1 cycle from NESHawk. When I use that state in NESHawk I do get runs that sync on console (with lower probability, about 50% less likely then the state I have now, but could be console dependent) but those same movies then do not sync in Mesen. Some memory access timings had speculatively been changed in 0.9.9, so they probably just need to be changed back. Anyway, there aren't many obstacles left for NES , probably by the end of the year most runs will be working on console. SNES: BSNES v115 was unexpectedly added to BizHawk recently, hopefully someone can try the 2 versions of the SOS submission, #7152: Samsara's SNES SOS in 04:34.08 . They may still both desync, but if v115 works where v78 doesn't, that might be a big deal for seeing more working SNES runs. GBA: I had run across this GBA emulator recently that seems to do things at a finer level of timing detail then mGBA: https://github.com/fleroviux/NanoBoyAdvance . A lot of necessary timing tests haven't even been written yet, so it probably doesn't matter much right now, but maybe worth keeping an eye on. I'm not sure what to focus on next once the remaining NES issues are sorted out. I'm open to suggestions, there's so much to do it's hard to decide where to start.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Making an update here now that BizHawk 2.8 is out. NES: Most games that use the USA version and start from power on (and don't use resets) should work on console now. This includes series like Ninja Turtles, Mega Man, and Castlevania. Existing runs of these games don't sync, so a lot of new verifications are blocked just by not having runs to work with. An increasing list of games poll input in unexpected ways that the TAStm32 handles poorly. Not a major issue but something to be aware of. There are still a couple of edge cases of DMC side effects that aren't emulated properly, but I need test ROMs and haven't encountered any games that would potentially encounter them yet, so its not a priority right now (plus it would be a huge rewrite of the CPU core.) Strictly speaking the Famicom also has some effects that aren't emulated correctly yet, but I don't have a Famicom. I guess you can just use a cartridge adapter and a regular NES though. SNES: Thanks to Morilli, poll based dumping is now possible from BNSES115. This brings SNES verification onto BizHawk. I have already verified some runs of Super Mario All Stars. Unfortunately most other runs randomly desync after some time, consistent with SPC700 issues. I am hesitant to put more time into resyncing runs when the probability of non-deterministic desyncs is high. I'll probably wait to see results of TASbot folks clock adjustment hardware work. Also some tests with Mega Man X 2/3 actually give consistent desyncs early in gameplay. This makes me think Cx4 timing may not be correct. But there don't appear to be any timing tests, and even if there were BSNES115 doesn't seem to like running homebrew with extra chips. Also I don't have a Cx4 dev board, so I couldn't compare to my particular SNES anyway. N64: Thanks to the heroic work of CasualPokePlayer, we now have Ares64 in BizHawk. I don't know much about this core, or how accurate it is compared to Mupen, but it does use a better graphics backend, so I'm at least somewhat hopeful. I might dig out my N64 and see how thing shake out.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
We've now passed 250 currently published runs with console verifications. That's ~10% of all current publications, which I think is a pretty impressive milestone. About 230 of these are NES or GB/C though, so while verifications for these systems are now common, other systems haven't progressed much or at all. I looked through some of the previous console verification threads and it looks like true put quite a bit of effort into SMS and Genesis, but they have highly non-deterministic start up states, much worse than NES, so probably only games not timing sensitive will work. I had another SNES that I thought was broken turned out to be working, so I tried Super Mario Kart again, and got the same desync on Vanilla Lake 1. A while ago I had also tried to verify [4389] SNES Super Mario All-Stars: The Lost Levels "warpless, Luigi" by DaSmileKat in 34:45.63, but got a deterministic desync in the water level of world 3. So while SNES suffers from a lot from non-determinism, there are also definitely some deterministic issues keeping some games from working, but no idea what they could be. I've set aside SNES work for now though, as it has been a huge time sink with few positive results. There are still more NES and GB/C verifications I want to do, so onward to 300 for now.
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
If we count obsolete movies, the total is 345 published runs. Of which NES/GB/GBC account for 307 of them. 345 is ~7.2% of all published runs when including obsoleted. Either way, very good progress. And much of that progress was made just in the last 2 years. I'm still planning to explore both Genesis and GB/C (potentially GBA too). Genesis is likely what I'll do first, though. From everything I've managed to research so far, Genesis seems like something that hasn't really been delved into much in the public space. There's a few verifications out there; but little to no information about the problems (if any) faced by those people, or what they used for staying in sync. During this last SGDQ, I talked briefly with numbers about his Gauntlet IV work-in-progress TAS. He described something odd about how some games read controller inputs. Along the lines of games causing the console to poll the controllers multiple times, but not always actually reading their state from memory. However, based on official documentation from SEGA, the pins on the controller ports are directly accessible in memory. It's just that in order to read the entire controller state, the mode/select pin must be toggled in order to access some of the buttons (there are more buttons than available pins). Whatever the case, it should be easy to detect what's happening using memory callbacks in Lua (assuming the emulator core actually supports that in bizhawk). And then there's concerns about initial state and all that fun stuff too heh. If it's just a matter of initial RAM, or something like that, it should be possible to initialize state ourselves, similar to ViGrey's RAM initialization ROM. The Genesis cartridges are extremely simple, even compared to the Atari 2600, so designing/programming a cart for this shouldn't be too hard.
In the mean time, this week I've been catching up on recent NES publications (and some submissions). Verified 9 movies in the past few days. And continuing to grind away at the remaining NES games. Also been working on some tooling projects for dumping Quality-of-Life. Particularly VeriTAS which has the ability to take a TASVideos Pub/Sub ID number, or a local movie, and automatically run the entire dump procedure (including dumping multiple movies in parallel). It does work, but it needs some polishing before it's useful for the public. And ViGrey and I still need to finalize the new dump format we were working on last year.
TAS Verifications | Mastodon | Github | Discord: @bigbass
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Thanks to Bigbass grinding out more NES verifications, we've reached over 300 currently published runs console verified, awesome! With 2890 currently active movies, we remain above 10% of all movies. It's worth saying that the influx of short homebrews for both NES and GB/C have allowed for some quick and easy verifications, so this stacks the numbers somewhat, but oh well. Aside from just raw numbers, there's also a good amount of R&D type work being done. Castlevania III led to some new research into NES edge case behaviours, Streemerz finally works with a new dev board, and SMB Stop and Swap managed to push the envelope of NES verifications in an innovative new direction. Some notable failures also occurred, such as Dragon Quest (J), so even for NES issues remain. A lot of GBA work has happened recently as well. NanoboyAdvance and my own GBAHawk can now make it through the entire mGBA test suite, as well as many other timing sensitive tests, so console verifications of first party GBA games is (hopefully) on the horizon. In both cases issues remain, but I'm pretty optimistic. The pipeline for such verifications was already in place, not being much different than the GB/C version, so it's really just waiting on emulation. Even with mGBA though some runs have worked, with Sushi the Cat being the first GBA verification in over 2 years. In somewhat relevant news I saw this cool video about loading games on Gamecube while bypassing the disc drive: Link to video Maybe something like this can lead to the possibility of Gamecube verifications down the road (admittedly cheating a bit but would still be cool.) I'm not aware of anything happening for other ssystems, but Sour has returned to working on Mesen (which includes SNES and now PCE) so maybe down the road something will be found that can give more SNES games a chance at verification. Also Ares is now in BizHawk, so maybe n64 has more of a chance now besides SM64 and Mario Kart, but I'm not aware of any console testing yet.
Bigbass
He/Him
Moderator
Joined: 2/2/2021
Posts: 193
Location: Midwest
Woo! Yes and for NES specifically, we've reached over 200 currently published verified runs! Out of 696 total active publications. Recent homebrew has perhaps stacked the numbers slightly, albeit if anything it was Nalleland in particular, having 5 different branches. That said, I've recently verified many non-homebrew games too. I'd like to thank Alyosha for his work on NESHawk. It seems apparent to me that the success rate of NES movies made with BizHawk 2.8 is noticeably higher than previous versions, and similarly when compared to FCEUX. Although plenty of FCEUX movies do verify too, NESHawk has addressed many possible edge cases that have been encountered in the last couple years. Up until this month, for awhile I had been fairly absent in recording new verifications. Was working on unrelated projects, but also a new replay device. At this point the NES replay functionality is pretty solid and it has improved how efficiently I can make verification attempts (thus the influx of recent NES verifications). I'm going to continue attempting more NES movies for the moment, though I am looking into which console I want to work toward next. Will likely revisit N64 momentarily, and then either GB/C or Genesis. If anyone reading this is familiar with FPGA development. I'd really appreciate having an open source/hardware NES flashcart, with similar mapper support to the Everdrive, but cheaper, with USB uploading, and purposefully no boot menu (as the everdrive's menu changes the initial state of the NES, making some verifications impossible for me). I have an idea how to implement this, but I probably won't start working on a prototype until after SGDQ.
TAS Verifications | Mastodon | Github | Discord: @bigbass
JosJuice
She/They
Editor, Emulator Coder
Joined: 7/3/2010
Posts: 193
Location: Sweden
Alyosha wrote:
Maybe something like this can lead to the possibility of Gamecube verifications down the road (admittedly cheating a bit but would still be cool.)
While you may be able to get around disc drive inconsistencies with this, you're unfortunately left with all the same problems as the N64. GameCube emulation currently doesn't have CPU and GPU timing accuracy anywhere near what would be required for console verification, and all GameCube games I know of configure the hardware to regularly poll controllers automatically, so you can't detect lag frames.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
JosJuice wrote:
While you may be able to get around disc drive inconsistencies with this, you're unfortunately left with all the same problems as the N64. GameCube emulation currently doesn't have CPU and GPU timing accuracy anywhere near what would be required for console verification, and all GameCube games I know of configure the hardware to regularly poll controllers automatically, so you can't detect lag frames.
Oh, interesting, I had thought such things were already quite advanced for GameCube. In your estimation, how far off is it? Like, for NES for example even a relatively inaccurate emulator can generate console verifiable runs a good fraction of the time, would the same be true with GameCube? Maybe there are games that spend a lot of time in an idle loop for example that would stand a chance?
JosJuice
She/They
Editor, Emulator Coder
Joined: 7/3/2010
Posts: 193
Location: Sweden
Alyosha wrote:
JosJuice wrote:
While you may be able to get around disc drive inconsistencies with this, you're unfortunately left with all the same problems as the N64. GameCube emulation currently doesn't have CPU and GPU timing accuracy anywhere near what would be required for console verification, and all GameCube games I know of configure the hardware to regularly poll controllers automatically, so you can't detect lag frames.
Oh, interesting, I had thought such things were already quite advanced for GameCube. In your estimation, how far off is it? Like, for NES for example even a relatively inaccurate emulator can generate console verifiable runs a good fraction of the time, would the same be true with GameCube? Maybe there are games that spend a lot of time in an idle loop for example that would stand a chance?
Well, I don't know how games work internally well enough to say how much they are affected by Dolphin's inaccuracies, but the fact that Dolphin updates are continually breaking sync for old movies is indicative enough. Getting any major improvements to the timings is probably many years into the future. Emulating all the caches and pipelines correctly is difficult not only in terms of the research required to accurately emulate them, but also in terms of how demanding it is to run.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Currently all published GBC homebrew TASes are console verified, a minor achievement but still something I think worth noting. There are a lot of pieces involved in console verification so I think breaking things down into little categories can be helpful in identifying where gaps exist. I can't say the same about GB homebrew for example because I don't have a suitable MBC1 flash cart. A lot of modern NES homebrew rely on a modern mapper so those need their own dedicated board as well (or a multi-mapper board where timing isn't too critical.) I tried a few of the GBC bootlegs as well, some don't sync in modern emulator versions. Some, like Donkey Kong 5, poll input in an uneven way not suitable for the current GBI based approach. It's already known that some GB/C games require a verification process with a smaller time resolution to work so this isn't anything new, just another example. There is also a lot going on with Genesis homebrew, even if there aren't that many TASes yet, could potentially be a good way to jump start Genesis verifications.