I'll be using this opening post to keep track of current progress on console verifications and other testing.
Currently testing:
- HDMA edge cases in single speed mode
_____________________________________
Dev Build: https://ci.appveyor.com/project/zeromus/bizhawk-udexo/build/artifacts
Repo for input timestamp files: https://github.com/alyosha-tas/GBI_timestamps
Dump script for 2.6.1 and older: http://tasvideos.org/userfiles/info/68761073876269603
Dump script for 2.6.2 and up: http://tasvideos.org/userfiles/info/71650785644185599
NOTE: use the setting 'TotalExecutedCycles Return Value' in GB->Settings in BizHawk and set it to GBI for double speed mode games
_____________________________________
Several games use uninitialized RAM in a way that causes desyncs on real hardware, below is a list of some examples I found:
Race Days (Select Dirty Racing) : This game checks a memory address to see if a block needs to be initialized. It does this many times in different locations. The same movie will desync in different ways each playthrough on console. Ex:
Super R.C. Pro Am : This game checks 4 locations in memory before initializing them, these could be related to linking but I'm not really sure. There seems to be a high probability of these cases passing on a real console in my testing:
Pokemon Crystal: When loading up the credits sequence it checks player state in the wrong bank, this doesn't seem to have any real impact, but would possibly cause a desync if you wanted to continue a movie past the credits:
Daiku no Gen-san - Ghost Building Company (Japan): uses uninitialized RAM at the start of level 2, this causes a desync towards the end of the level. It looks like all it does is use these values for starting xscroll possibly for the clouds.
Battletoads: Reads from HRAM at the start of level one to set some values in WRAM. Doesn't effect the TAS but might effect casual play, not sure
My only request is that you make an effort to prepare it for a higher level of debugging quality, even though it can't be used yet, chiefly by giving it the ability to step and exit from emulation mid-frame. We're not likely to address the problem of bizhawk-cant-handle-debugging very soon, but we're more likely to do it sooner if we have a nice in-house c# core to practice with. However, this goal will come into conflict with performance. My philosophy is, if you want it to be _fast_, you shouldn't do it in c# in the first place. A fluffy c# core should be for features, instead.
I would also ask for one request. The one I'm dying to see happen is emulating four Game Boys at once and linking them together for games like F-1 Race and Yoshi's Cookie.
Well, apparently MUGG found some glitches in gb games where it behaves differently depending on the emulator:
1. Daffy Duck The Marvin Missions
2. The Smurfs
3. Wario Land II (There's a gameboy (non-color) version of this too)
So maybe in the long term after the core is matured enough, these can be looked into?
Joined: 8/14/2009
Posts: 4090
Location: The Netherlands
While we're at it with the requests, I have one - bootleg mappers. In particular, no TAS-capable emulator can currently run Terrifying 911 - the only emulator that runs it currently is hhugboy.
http://www.youtube.com/Noxxa
<dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects.
<Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits
<adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
I'm making decent progress here. So far I'm doing everything in single clock steps, so breaking out into a debugger should be simple.
Trace logging works for the new CPU core, and I'm able to compare logs to gambatte. So far it makes it through the bios corectly (even though there is no rendering yet so there is nothing to see.) There is a couple of cycles of variation in timing the scanline counter, so it doesn't exactly match Gambatte, but all that stuff is still very preliminary (and the deviation doesn't show up until about 13000000 cycles in, so I guess it's pretty small.)
Savestates are already functioning as well, so all the basic foundations of the core are coming together. Next steps are to get rendering going. I have sprite evaluation and the timer done, just need to do all the actual rendering steps and start firing interrupts and such. Once that's in place I can start varifying the CPU and working through all the various test ROMs. I think rendering shouldn't be too bad, certainly the biggest hurdle will be the dreaded APU, but that's still a ways off.
Also I looked at Terrifying 911. At first I thought that was going to be some kind of murder mystery that starts off with a phone call to 911, but after googling it ... it's something else entirely 0_0
While we're at it with the requests, I have one - bootleg mappers. In particular, no TAS-capable emulator can currently run Terrifying 911 - the only emulator that runs it currently is hhugboy.
Rockman 8 by YongYong and Beast Fighter by Sachen also only work on hhugboy.
So far so good. It's a good thing the Gambatte core already exists in BizHawk, it probably would have taken me 10x as long to debug some of these tests without it. instruction timing tests pass as well, so the basics of the CPU are functioning correctly. Next steps will be Memory access timing, Timer tests, and interrupt timing. Once those are done the core should be on a pretty firm footing to start on the more difficult stuff.
@ThunderAxe31: I'm not following. Is the glitch supposed to be there or not?
Making good progress so far. Right now all of the CPU, DMA, and timer tests from Gekkio's mooneye gb page pass, so I'm pretty confident the core is internally consistent and accurate up to this point. Background and sprites are working just need to do windowing and graphics will be initially done (though not in a cycle accurate way yet, that will be the next step.)
I have a couple bugs in some register bits and interrupt timing to work out, but those should be easy enough to work out so games should be fully playable (except for sound) pretty soon.
@ThunderAxe31: I'm not following. Is the glitch supposed to be there or not?
Is supposed to be there. If you look carefully in the upper left in my screenshot from the YouTube video, you can see some whitish lines. These should be the glitched sprites just before disappearing, because the player is walking slowly throught the level.
Also note that the glitch happens on BGB as well.
Since I had some time today I decided to see if I could get the last graphics bit, windowing, working. As it turns out, I really should only try to do one thing at a time, because a large chunk of the time I spent on this was just trying to remember what the heck I was doing.
In the end though it worked just fine. This scene from Mega Man 5 is actually pretty challenging to get right graphically, so it's a good sign that things are working well:
A few more small timing things to clean up and it will be time to start audio.
I also did some performance analysis, since this core runs much faster the NESHawk while not really having that much less to do, to see if I could learn anything from it.
One thing I did learn is that statements like this:
state = divider_reg.Bit(9);
are really very expensive. Maybe knowing that can help me speed things up in other places.
Is that line using bit masking and shifting? You could also store each emulated bit as its own boolean.
Yeah it does this:
public static bool Bit(this byte b, int index)
{
return (b & (1 << index)) != 0;
}
It also seems like Bools are just slow altogether, not sure.
I would posit that it's the sheer number of function calls to Bit() that eat away at performance. Can you run a disassembler to determine if the JIT compiler is inlining those calls? If it isn't, you could try the AggressiveInlining attribute that was mentioned in the NesHawk thread.
I haven't done a huge amount of performance optimization in C#, but it's pretty fascinating.
I would posit that it's the sheer number of function calls to Bit() that eat away at performance. Can you run a disassembler to determine if the JIT compiler is inlining those calls? If it isn't, you could try the AggressiveInlining attribute that was mentioned in the NesHawk thread.
I haven't done a huge amount of performance optimization in C#, but it's pretty fascinating.
Tried aggreisve inlining, no effect, probably the compiler already does it.
Tried removing the Bit call altogether and just using explicit statements like (value & 0x200) > 0, no effect.
Tried changing to integers and rewriting the logic. Small but positive effect, not sure it's worth the reduced clarity.
But the most effective change was putting the condition most likely to be false first in the 'if' statement later on in that function. >___> I guess I need optimization 101.
EDIT:
Here's the complete thing if anyone sees anything obvious. According to performance profiler, this tiny block takes about 1/4 as long to run as the entire CPU function 0_0. So something about it must be super slow:
public void tick_2()
{
divider_reg+=1;
// pick a bit to test based on the current value of timer control
switch (timer_control & 3)
{
case 0:
state = divider_reg & 0x200;
break;
case 1:
state = divider_reg & 0x8;
break;
case 2:
state = divider_reg & 0x20;
break;
case 3:
state = divider_reg & 0x80;
break;
default:
break;
}
// And it with the state of the timer on/off bit
state_c = timer_control & 0x4;
// this procedure allows several glitchy timer ticks, since it only measures falling edge of the state
// so things like turning the timer off and resetting the divider will tick the timer
if ((state == 0 || state_c == 0) && (old_state > 0 && old_state_c > 0))
{
timer_old = timer;
timer+=1;
// if overflow, set the interrupt flag and reload the timer (4 clocks later)
if (timer < timer_old)
{
pending_reload = 4;
reload_block = false;
}
}
old_state = state;
old_state_c = state_c;
}
Just a few random ideas:
The "switch (timer_control & 3)" part might be faster as a lookup table?
e.g:
static int[] mask = new int[4] { 0x200, 0x8, 0x20, 0x80 };
state = divider_reg & mask[timer_control & 3];
Might just end up being pretty similar, though.
if (timer < timer_old)
Couldn't this just be if(timer == 0) or if(timer == Int32.MinValue) (or whatever the data type used here) - should technically be faster than comparing to another variable.
I imagine that's probably not really causing much of a performance issue, either, though.
If you don't actually need the values of old_state & old_state_c elsewhere, maybe try something like this?
I had actually tested the look up table earlier but it is slower by a LOT. I guess I could try with unsafe too but it's not worth it just for that.
I also tried stringing together '?' operators but that had only marginall effect.
I also tried just replacing the switch with ordinary logic elements but that wasn't much faster either.
Overall it's currently ~30% faster then when I started, and I guess that's about as fast as it's going to get. It doesn't matter much for this case but I thought I could learn something more from it since it's so simple and obviously slow. Oh well though, too much other stuff to do to get hung up on it.
EDIT: I tried your (Sour) other suggestions but they didn't result in improvements. Thanks for coming up with things to try though.
public void tick_2()
{
divider_reg += 1;
// pick a bit to test based on the current value of timer control
// switch (timer_control & 3) {
// case 1: state = divider_reg & 0b0000001000; break;
// case 2: state = divider_reg & 0b0000100000; break;
// case 3: state = divider_reg & 0b0010000000; break;
// case 0: state = divider_reg & 0b1000000000; break;
// }
int i = timer_control & 3;
i = (i - 1) & 3;
i = 0b0000001000 << (i * 2);
state = divider_reg & i;
// AND it with the state of the timer on/off bit
state_c = timer_control & 0x4;
// this procedure allows several glitchy timer ticks, since it only measures falling edge of the state
// so things like turning the timer off and resetting the divider will tick the timer
// if ((state == 0 || state_c == 0) && (old_state > 0 && old_state_c > 0)) {
if (((state & state_c) == 0) && ((old_state | old_state_c) != 0)) {
timer_old = timer;
timer += 1;
// if overflow, set the interrupt flag and reload the timer (4 clocks later)
if (timer < timer_old) {
pending_reload = 4;
reload_block = false;
}
}
old_state = state;
old_state_c = state_c;
}
^ About the same as the current one. (But I believe your new condition statement is incorrect, as state and state_c never have the same bits set.)
So to summarize so far:
- putting failing condition first is the biggest improvement
- bool to int is also a noticable but smaller improvement
- look up table has a negative effect
- everything else has more or less no effect
So far so good. It's a good thing the Gambatte core already exists in BizHawk, it probably would have taken me 10x as long to debug some of these tests without it. instruction timing tests pass as well, so the basics of the CPU are functioning correctly. Next steps will be Memory access timing, Timer tests, and interrupt timing. Once those are done the core should be on a pretty firm footing to start on the more difficult stuff.
A few more small timing things to clean up and it will be time to start audio.
I also did some performance analysis, since this core runs much faster the NESHawk while not really having that much less to do, to see if I could learn anything from it.
One thing I did learn is that statements like this:
