Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
Some minor follow up here. I started testing with SMB3, but GBP does have an effect on sync, so I will need to more carefully emulate GBP before I can sort that one out. GBP detection uses the serial port with a certain protocol, so it takes up some amount of the processing time.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
Note that with GBI you can just eliminate the GBP detection entirely (it's a configurable option), or alternatively plugging in something to the link port will prevent serial communication from taking place (this is what RTA runners do IIRC).
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
Note that with GBI you can just eliminate the GBP detection entirely (it's a configurable option), or alternatively plugging in something to the link port will prevent serial communication from taking place (this is what RTA runners do IIRC).
Woah really? Thanks for the tip! I'll give this a try.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
Link to video After carefully tuning Flash timing parameters to match my cart, I was able to verify a resync of SMB3. I did not have to otherwise add any lag frames, and I starrted from clear SRAM, so emulation seems correct there. I also disabled GBP detection. I had to use very large parameters for flash timing, 1500000 cycles for a sector erase compared to ~350000 I used for Sonic. The chip in SMB3 is the same one as in pokemon, so maybe I need different defaults for those but I can't seem to find a data sheet that is an exact match. Anyway the sync settings for manual flash timing will be included in the next release. I'll try to do all levels this weekend.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
By "same chip as in Pokemon" do you mean you checked by unscrewing your own cartridge? Flash 1MB chips come from potentially 2 different manufacturers, Sanyo and Macronix (the latter being more common iirc?). The Pokemon RTA community has noted Sanyo chips apparently are much faster than Macronix chips.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
By "same chip as in Pokemon" do you mean you checked by unscrewing your own cartridge? Flash 1MB chips come from potentially 2 different manufacturers, Sanyo and Macronix (the latter being more common iirc?). The Pokemon RTA community has noted Sanyo chips apparently are much faster than Macronix chips.
Yes I unscrewed them and I have Macronix in both my carts. I don't know how much effort I should put (or want to put) into characterizing flash timings my manufacturer vs just having a user settable range big enough to cover the possibilities. It seems like the latter option is probably better, knowledgeable users can fill in the correct values.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
Flash ID checking would end up taking differing amounts of time depending on flash ID returned, so that could be useful for letting the user select it regardless.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
Flash ID checking would end up taking differing amounts of time depending on flash ID returned, so that could be useful for letting the user select it regardless.
Oh yeah, I forgot about that. Why can't it just be simple.
Skilled player (1742)
Joined: 9/17/2009
Posts: 4986
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
I'm not sure if it was asked here, but are the Pokemon runs console verifiable? I'm especially interestested in the ACE ones.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
jlun2 wrote:
I'm not sure if it was asked here, but are the Pokemon runs console verifiable? I'm especially interestested in the ACE ones.
I don't see why not, it should be just a matter of characterizing cart timings.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
All Pokemon TASes would not normally save in the game (except I guess for Hall of Fame? although if that final input is a desync that would be trivial to fix). As such, you shouldn't get any variation in flash timings (outside of I guess just the check for which flash chip is present, you need to have a match for identical timing due to ID check as I've said above). Pokemon games aren't trivial to resync anyways, due to potential of lag to create different RNG results (if mostly from vblanks being inserted in-between RNG rolls "unexpectingly") and Emerald/FireRed/LeafGreen have one of the hardware timers set at the maximum frequency used to seed RNG (not impossible to resync but nontrivial anyways), also Emerald/FireRed/LeafGreen have the wonderful decision to busy loop to wait for vblank, so any timings on a previous frame may affect the current frame (rather than a halt which would "resync" timings). Also, Pokemon is a very very long game. It often balloons with rerecords due to RNG. GBAHawk does not run at full speed. This makes it very painful to TAS Pokemon, and not an enticing option when BizHawk has mGBA as a core which can run well past full speed. I would rather have NBA ported and that tried rather than ever TAS using GBAHawk, as that can at least run well past full speed.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
As a "simple" test, and because I happen to own an Emerald cart, I checked if GBAHawk would actually get the correct TID in Emerald, going through the intro as quick as possible (also assuming a dead battery, since that's what my cart has). Keep in mind the TID is just grabbed off one of the hardware timers running at the maximum frequency, and the busy loop vblank quirk means any deviation early on should in theory cascade downwards towards the rest of the movie, and the TID here is the new initial RNG seed, so if it's wrong, any movie input afterwards would desync once it hits RNG points. The test ended up failing immediately, due to the first input being missed. If I moved all the input in the timestamps back 1 frame (so all inputs happened 1 frame sooner on console than GBAHawk had), then the input syncs. Somehow, it evens gets the right TID (although maybe this is by coincidence if anything?)
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
What script did you use to dump the inputs?
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
I grabbed the one from runs.tas.bot, then modified it to play with GBAHawk movies nicely and correct issues with it.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
Here's the current loop I use, which is also a modified version of that script:
cycle = -1232 * 65
#cycle = 0
with open("GBItimestamps.txt", "w") as f:
  for line in inputs:
    sline = line.decode().rstrip()
    if sline[0] == "|":
      data = sline[:-1]
      data = data[1:]
      #data = data[25:]
      if data != lastInput:
        if cycle < 0:
          f.write(f"{0:08X} {convert_frame(data):04X}\n")
        else:
          #sometimes need a -1 here, depends on timing of inputs per game
          f.write(f"{cycle//4096:08X} {convert_frame(data):04X}\n")
      lastInput = data
      cycle += CYCLES_PER_FRAME
The most important difference is that cycle_per_frame are added last, which explains why you had to subtract one frame worth of timing. I don't remember actually making that change, but I think it is correct since input should occur at the start of the current frame.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
I already had the cycles per frame add set to the end of the movie. The subtraction of the frame is after that change. (in fact, you have some weird -1232 * 65 offset already? Which doesn't make much sense)
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
I already had the cycles per frame add set to the end of the movie. The subtraction of the frame is after that change. (in fact, you have some weird -1232 * 65 offset already? Which doesn't make much sense)
Interesting, well I'll eventually get to Pokemon but I doubt it got the right RNG by chance. That value comes from start up timing tests. Similar to how the tas.bot script has cycle = -83776.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
That still doesn't make much sense? Startup timing tests indicate that input would be polled sooner on console than emulator? Is it GBI somehow being late with its internal timestamp tracking? (by ~20 PWM frames???) Keep in mind by "1 frame back" I meant I just set the starting cycle from 0 to -CYCLES_PER_FRAME.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
That still doesn't make much sense? Startup timing tests indicate that input would be polled sooner on console than emulator? Is it GBI somehow being late with its internal timestamp tracking? (by ~20 PWM frames???) Keep in mind by "1 frame back" I meant I just set the starting cycle from 0 to -CYCLES_PER_FRAME.
Based on my testing, the first frame after power up is shorter. A test called music4.gba, show that the ppu must start either at scanline 161 or 225 (GBAHawk starts at 225, which is where the 65 comes from.) With this timestamps for GBI line up (roughly) with vblank boundaries, which is noticeable when testing games that sample input close to the start of vblank. I imagine this was noticed by whoever put -83776 in the original script as well (1232*65 = 80080 which is pretty close.)
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
Makes more sense. Starting at 225 means it takes 163 scanlines to get to the second frame, 65 scanlines less than the typical 228 total scanlines. I would assume me offsetting by an entire frame more just worked due to how Pokemon polls inputs (once at the start of the main loop, which normally at the beginning of vblank).
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
CasualPokePlayer wrote:
Makes more sense. Starting at 225 means it takes 163 scanlines to get to the second frame, 65 scanlines less than the typical 228 total scanlines. I would assume me offsetting by an entire frame more just worked due to how Pokemon polls inputs (once at the start of the main loop, which normally at the beginning of vblank).
Correct. And even though timing tests give 65, I do occasionally have to apply -1 to the GBI inputs if inputs are being polled really close to the start of vblank, so maybe there is a slight offset to when GBi applies inputs and the original -83776 is closer to the precise value that should be used.
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
Checking my Emerald cart, it appears to be using a Macronix chip (reporting an ID of 0x09C2). GBAHawk appears to report 0x1362 (Sanyo's ID) however (at least according to source code, my quick multiboot rom just says 0xFFFF so I assume it's not emulating a flash chip in that case). So not sure how RNG happened to sync (maybe the difference in cycles had it so eventually the vblank interrupt would occur in the same place within the vblank wait busy loop? Not sure how that's even possible however, considering a single check for a flash ID takes multiple frames? also Macronix is checked first so I would guess there would be a few frames less with Macronix here?).
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
Not sure about emerald, but on saphire the order goes : timer start Flash chip check halt is sued on copyright screen So, as long as RNG isn't used before the copyright screen, the chip check shouldn't matter if it takes less than a frame. And it seems the check doesn't do much more than check the ID, so halt should be able to sync everything up. At least in Sapphire
Emulator Coder, Judge, Experienced player (732)
Joined: 2/26/2020
Posts: 788
Location: California
As I stated before, FireRed/LeafGreen/Emerald busy loop to wait for VBlank (that is what I meant by "vblank wait busy loop"). This was a change from Ruby/Sapphire, which actually halt. Ruby/Sapphire just use RTC for seeding RNG anyways, they don't use the hardware timer for this. Also, nevermind what I mentioned about the "single check for a flash ID takes multiple frames," that's just for reading the flash ID, which is done once. The only difference between the Macronix and Sanyo IDs here is the Sanyo ID will end up going through the while true loop an extra time (the cycle difference possibly just ending up resulting in the same position of the busy loop triggering VBlank, leading to no difference)
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3826)
Joined: 11/30/2014
Posts: 2832
Location: US
No luck resyncing Gunstar Super Heroes or Lady Sia to GBAHawk.