Post subject: GB accuracy tests on VBA-rr
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Recently, with the advent of console verification of TASes, there seems to be an increasing concern about the accuracy of emulators. In view of this, I ran some test ROMs with the GB emulators accepted for submission here. Anyone can do this, a similar test is available in GBdev for general purpose emulators, but I think this is unknown to most here. I'm posting the results here to show that there are some accuracy problems with these emulators, hoping that they'll be fixed if it's easy enough or to help people make decisions of which emulator version to choose if they want their runs to be more likely to sync on real hardware in case GB verification becomes a reality. The test ROMs are due to Blargg, except the last one for SGB (which I don't know what it tests) you can google them if you want to download and do this. The screenshot to the left is VBA-rr v23.5 and the one to the right is VBAM-rr v24. Notice that passing these tests doesn't mean the emulator is accurate, but failing them doesn't say anything good about it. cgb_sound (the answer is the same both in CGB and DMG mode, a real CGB would pass and a real DMG would fail 8,9,10,11,12 with the values 1,1,1,4,2 respectively) dmg_sound (v2) cpu_instructions instruction_timing memory_timing (v2) oam_bug (v2) SGB Pack
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Nice! But if some of the published runs desync after these fixes are implemented, how, and who will fix them? Sorry to go off-topic!
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
I can probably attempt the timing bugs. I studied the GameBoy for my thesis project last year. Though I don't really have time right now.
Skilled player (1326)
Joined: 9/7/2007
Posts: 1354
Location: U.S.
Those look like quite a bit of emulation errors. Are those emulation bugs going to be hard to fix?
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Sonikkustar wrote:
Those look like quite a bit of emulation errors. Are those emulation bugs going to be hard to fix?
Well, MUGG mentioned Gambatte being an accurate emulator, maybe use it as reference when testing the emulator?
Editor, Player (69)
Joined: 1/18/2008
Posts: 663
Once things progress well enough, since I never got my NES playback device done and got beaten to it by someone else, I can work on a GB playback device.
true on twitch - lsnes windows builds 20230425 - the date this site is buried
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
I know the results of Gambatte can be found here. But here it is anyway:
Editor, Emulator Coder, Expert player (2157)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
There is another "good" GBx emulator named KiGB, but it is close-sourced AFAIK. I wonder if it could pass these tests. KiGB official web site: http://kigb.emuunlim.com/ EDIT: I wasn't aware of that the test results with KiGB 2.05 had been included at gbdev.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4036
What about the motion sensors not working? Could that be related to one of these problems or some other emulation error?
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Motion sensors are GBA only. Due to the striking differences of GB and GBA emulation, I don't think these issues are related.
Editor, Experienced player (570)
Joined: 11/8/2010
Posts: 4036
Some GBx games like Kirby Tilt 'n' Tumble (GBC) require the motion sensors, but motion doesn't seem to work for these games. (There's a TAS on YouTube of this game using an old VBA, so I know motion worked at one time.)
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
I know, I meant in VBA it's only on for GBA, looking at VBA's code though, both in GB and GBA the lines that implement it are commented out, so it seems they took the motion sensor off in both modes. It may be possible to make it use them again by uncommenting those lines and compiling again.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
I have evaluated the work which is required to fix memory poke timing, and it pretty much put me off... To fix the timing inaccuracy I need to rewrite the entire timing method and CPU execution core to make it cycle-based. I may give it a shot later to sort it out, but currently it feels like it would be easier and faster to just implement TAS tools into Gambatte and use that for GB(C) TASing.
Skilled player (1741)
Joined: 9/17/2009
Posts: 4981
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Warepire wrote:
I may give it a shot later to sort it out, but currently it feels like it would be easier and faster to just implement TAS tools into Gambatte and use that for GB(C) TASing.
Wouldn't that make the gb(c) runs incompatible with vba? Besides, the more emulators, the harder it is to maintain them.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
jlun2 wrote:
Warepire wrote:
I may give it a shot later to sort it out, but currently it feels like it would be easier and faster to just implement TAS tools into Gambatte and use that for GB(C) TASing.
Wouldn't that make the gb(c) runs incompatible with vba? Besides, the more emulators, the harder it is to maintain them.
That would be the downside, yes.
P.JBoy
Any
Editor
Joined: 3/25/2006
Posts: 850
Location: stuck in Pandora's box HELLPP!!!
We could start a new emulator called BBA (the b taken from bsnes, though I'm not sure how flattered byuu would be by that)
Editor, Emulator Coder, Expert player (2157)
Joined: 5/22/2007
Posts: 1134
Location: Glitchvania
P.JBoy wrote:
We could start a new emulator called BBA (the b taken from bsnes, though I'm not sure how flattered byuu would be by that)
Shouldn't that be named BBC (Byuu-ist Boy Color)? EDIT: Oh, I forgot about the -jin and -rr postfixes.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days <adelikat> no doubt <adelikat> klmz, they still do