Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
With a little added logic the other DMA test variant, sprdma_and_dmc_dma_512, now passes as well. That's 2 of the most grueling tests passing, so if I can just work out frmae IRQ stuff there will be no more hurdles and only tuning the APU left to worry about.
I'm a little worried about the accruing performance hits though. I'm currently at about a 20% penalty over 1.11.6. On my laptop that equates to being able to run at about 140 fps. Well hopefully it will still be at over 2x speed once I'm done, I think it will need some serious optimization work though.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
After days and days of repeated failure, I finally found the last piece of the puzzle needed get sprdma_and_dmc_dma to pass in a cycle accurate way, with full RDY implementation and no hardcoded hacks.
Visual 2A03 saved the day here, and without it there is absolutely no way I could have figured this out. The key seems to be an overlooked feature of DMC DMA where if it is called from a write to $4015 and the buffer is empty, it will take one less cycle to complete. This 1 cycle is what had me stumped and there is no other conceivable way it could have worked out (I tried every imaginable alternative.)
There is still some work to be done to clean things up, but I doubt there will be any remaining tests as difficult as this one, so onward to 100%!
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
^ glad to see a positive response to what must have been a lot of negative feedback after seeing that pricetag. I was definitely excited to see this done...but not $30 worth of excited, so glad it's free now!
(It's still an awesome project, nice work)
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
cpu_exec_space/test_cpu_exec_space_apu now passes with a few simple fixes. That makes 10 new tests passed, up to 90%!
I did not include DMA timing fixes, as I still don't know the problem is, so that 90% can be taken with a grain of salt, but still this is progress.
Pretty much every remaining test involves the APU. This is good, since it means the rest of the core is really solid, but that is also the least developed part, so will take some effort to get in order.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
Pokota wrote:
Which tests are now failed?
cpu_interrupts_v2/rom_singles/4-irq_and_dma
cpu_interrupts_v2/rom_singles/5-branch_delays_irq
^ those tests fail directly as a result of fixing the timing bug, (which is running the CPU one extra cycle before starting OAM DMA.)
They are both related to IRQs caused by the frame timer at $4017. I tried repeatedly to fix it, but when I do then other tests fail (jitter, NMI and IRQ, even IRQ timing.) I can't get them all to pass at once, which makes me think there is something else going wrong.
Also, srp_and_dmc_dma is consistently off by 1 cycle too many for all 16 tests. Nothing I do makes that 1 cycle go away. I think all these issues are somehow related, but have no ideas.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
I finally got Time Lord working, Hurray! This is the only total incompatibility that I know of that remained so I'm quite happy to get it working. It was actually a simple fix, lucky the NesDev wiki already said what needed to be done, it was just a matter of implementing it.
Unfortunately things have gone awry elsewhere. NesHawk had a timing bug and apparently passed some rather important tests through shear luck and cancelation of errors. Removing this bug has led these tests to no longer pass , despite my repeated efforts to resolve them. So looks like a long road ahead.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
scrimpeh wrote:
Yo, I just wanted to drop in and say I'm amazed by all the work you've been doing for NEShawk, Alyosha.
Keep it up!
Thanks!
SaxxonPike wrote:
I noticed that the DMA work you've been doing relies on the RDY pin. I've put off working much on the 6502 CPU core until after I could get some automated testing going. However, the IncPc micro-op in the MOS6502X core was missing the dummy read (which would be stalled by RDY.) I've committed that fix today to master.
AHX/SHX/SHY illegal opcodes affected by the high byte of the effective address also behave differently based on the RDY pin. More info here: http://csdb.dk/forums/index.php?roomid=11&topicid=94460
If you read that, keep in mind that in the C64 context, RDY is controlled by the video chip and is made inactive when it needs the bus fully.
Oh good catch! After looking at that I noticed that the dummy read for various instructions also is not affected by the RDY pin as it should be, and I think that is part of what is keeping from passing some more of these test, so fortuitous timing.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
Aww, too bad, I hope this gets turned into a neat play around so it can get published.
I also hope the actual fastest run gets submitted. Anyway to ask the authors about it?
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
thommy3 wrote:
Wow, it's crazy how fast you get these improvements out. Great work.
I've a question though, is an emulator passing all those above tests perfect in the sense that it behaves like real hardware?
Or is it just that he passes all pubished test roms but there might still be differences to real hardware because maybe those roms didn't cover every hardware aspect?
Thanks!
In the case of NesHawk specifically, yes there is a very real difference between the emulation architecture and the real hardware.
NesHawk runs at the the timing level of a PPU tick, when 3 of those ticks go by we run one CPU instruction. A real NES though runs at a timing level one step above this, where a master clock ticks off pulses and the PPU and CPU use their own divders to time execution. In particular the 6502 (or the NES variant in this case) actually runs 2 phases which make up one instruction clock, and this level of hardware detail is not emulated at all by NesHawk. The fact it can still pass all these edge case tests though is simply proof that this level of detail is not really needed almost all of the time.
If you are interested in how things really work, you can check out Visual 2A03 (the NES cpu) or Visual 2C02 (the ppu), pretty neat stuff!
____________________________
I finally got around to working on ppu open bus behaviour. It was pretty simple, now cpu_dummy_writes/cpu_dummy_writes_ppumem and ppu_open_bus/ppu_open_bus both pass. Scratch 2 more off the list.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
jlun2 wrote:
I got a suggestion: Try running the console verified runs on BizHawk and see if they sync. Most of them were done on FCEUX, but if for whatever reason it does not sync maybe something is wrong?
Good idea! I think I'll give it a try once I get DMA timings worked out. Only games that are very geenrous with timing alignments will sync without that.
Speaking of which, after several failed attempts and hours of debugging, dpcmletterbox/dpcmletterbox now works properly. This is a signifcant improvement to DMC DMA timing so I hope i can make more progress on OAM DMA now.
No where near that bad. The performance hit so far came from code that runs every pixel (3 times per cpu tick) but that is done now. I think only accurate APU emulation remains that will a signifcant negative impact on speed (my guess is another 2-3%.) After that I do plan to go over everything and make obvious optimizations to gain back some of the performance losses, but I doubt I can reach parity with 1.11.6 again in terms of speed.
I don't really understand what feos is suggesting with the cpu refactor, so I'll leave that up to him, seems like a huge undertaking.
zeromus wrote:
Sure thing I installed it, but I can't tell what, if anything, it is doing, how do I use it?
_____________________
I've been testing various games known to be tricky to emulate. The first game that was previously incompatible with BizHawk now works, Fire Hawk!
Curiously this game runs on FCEUX but went to black screen on previous versions of BizHawk.
If anyone knows other games which are just plain incompatible please let me know (I am aware of Time Lord currently) having games to test on really helps.
EDIT: oh, the recent fixes to OAM reads also fixes cpu_dummy_writes/cpu_dummy_writes_oam, so, scratch another one off the list!
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
The version of AtariHawk in BizHawk 1.11.6 contains numerous, and some rather serious, bugs that have since been fixed. In partiuclar a bug in the 6532 causes measurably innacurate timing, which this game uses. This run does not sync in the current dev build (fails at level 1.)
I'm not sure what the rules are regarding this, are runs from dev builds submittable?
Still entertainging game though. Having to play with both joysticks must make it challenging.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
I really like these technical runs. Impressive result.
Pretty entertaining and good commentary. I'm also curious about those shaky portraits in the credits though, how strange.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
feos wrote:
What about attempting to speed up the core too?
Well, my current improvements to accuracy slow down the core by a noticable amount. At 400% speed I can run SMB3 on my laptop at about 180 fps on 1.11.6. The current Bizhawk master build can do about 160. My current test build can do about 152.
So... we're kind of going in the wrong direction there 8D
These represent almost entirely ppu changes. So that might also be a good place to look for performance improvements.
EDIT: after spending a long time to find a very small bug, oam_stress/oam_stress now passes.
I'm also making progress on sprdma_and_dmc_dma, I'm hoping maybe by the end of the month those tests will pass.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
NesHawk now passes:
apu_test/rom_singles/7-dmc_basics
Figuring out what was going on here took some effort but I now feel the implementation is pretty accurate.
Unfortunately it now hangs completely on sprdma_and_dmc_dma. I haven't the slightest idea why. Previously the test resulted in answers off by a factor of about 5, but at least it finished. I'm not sure what to make of this and posted on NesDev looking for some insight. Safe to say it's still a work in progress.
But other games i tested that use DMC still work fine, so prehaps I'm just off by a couple of cycles, sprdma_and_dmc_dma is a very exacting test.
EDIT:
AFter a bit of work filling in some of the undocumented opcodes, I am now able to pass instr_misc/instr_misc
So 4 extra passing tests means we should be tied for 2nd place with MyNes, alright!
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
I greatly improved my code and committed to my fork for testing. So anyone who wants to try it out please do and let me know if anything is amiss. I have tested with several games so far (SMB, SMB3, Battletoads, Micro Machines) and found no deficiancies.
https://github.com/alyosha-tas/BizHawk
This version also passes 2 accuracy tests that previously failed
sprite_overflow_tests/3.Timing
sprite_overflow_tests/4.Obscure
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
I have been (very) slowly working on NESHawk PPU. So far I have rewrote sprite evaluation to take place simultaneously with background generation, as it is with a real NES. Together with proper read behaviour of 2004 during this time, I finally managed to fix those annoying horizontal lines and shaking in Micro Machines.
This is still very early WIP work and generally breaks other things, but the timing is correct so i should be able to start slowly fixing other timing issues and bugs and such.
Right now it also slows down emulation by a noticable factor as well. I don't think my code takes considerably longer to run then the original, so this is the first major issue to be worked out.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
I happen to notice in Samsara's temp encode that the health never goes down. I have no idea why this might be, different ROM? I just mention it as a note to publishers since health management is a pretty important part of the run.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
After a lot of staring at the code I finally resolved the high pitch sound bug that effects several games, notably Combat, so the tanks don't squeek anymore.
I also rewrote most of the audio generation code to be closer to cycle accurate, although there are still a few minor bugs to sort out. I'm pretty happy to have gotten this done since it is now much simpler and sounds better.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
jlun2 wrote:
I'm not sure if this is the right place to post, but the game Star Wars: The Empire Strikes Back seems to not exactly...work. Disregarding the black bars on the side of the screen, on BizHawk 1.11.6, pressing any of the buttons do not move the ship whatsoever.
Given there's only 1 button on the controller I'm not sure what I'm doing wrong. Is it the emulator's fault or the game?
Seems like it's working fine to me. The game expects you to hit 'reset' before you can play it, so make sure you do that, then it plays fine even on 1.11.6.
The black lines are part of the Atari 2600 TIA and are not an error. I usually see them called 'The HMove Comb', it is a side effect of moving objects on screen.
Editor, Experienced Forum User, Published Author, Expert player
(3572)
Joined: 11/30/2014
Posts: 2744
Location: US
Adding the keyboard sounds well beyond my programming ability. I think I'll stick with fixing core bugs, but if you get the framework for it in place I will definitely help putting everything together and sorting out any issues.
I made the initial commits with all necessary bug fixes so the system loads, so it's just waiting for the keyboard to put it all together.
This also makes me think that paddle support would be great to have, but that is probably even harder then the keyboard.
With a little added logic the other DMA test variant, sprdma_and_dmc_dma_512, now passes as well. That's 2 of the most grueling tests passing, so if I can just work out frmae IRQ stuff there will be no more hurdles and only tuning the APU left to worry about.
I'm a little worried about the accruing performance hits though. I'm currently at about a 20% penalty over 1.11.6. On my laptop that equates to being able to run at about 140 fps. Well hopefully it will still be at over 2x speed once I'm done, I think it will need some serious optimization work though.
I finally got Time Lord working, Hurray! This is the only total incompatibility that I know of that remained so I'm quite happy to get it working. It was actually a simple fix, lucky the NesDev wiki already said what needed to be done, it was just a matter of implementing it.
Unfortunately things have gone awry elsewhere. NesHawk had a timing bug and apparently passed some rather important tests through shear luck and cancelation of errors. Removing this bug has led these tests to no longer pass , despite my repeated efforts to resolve them. So looks like a long road ahead.
Curiously this game runs on FCEUX but went to black screen on previous versions of BizHawk.
If anyone knows other games which are just plain incompatible please let me know (I am aware of Time Lord currently) having games to test on really helps.
EDIT: oh, the recent fixes to OAM reads also fixes cpu_dummy_writes/cpu_dummy_writes_oam, so, scratch another one off the list!
I'm also making progress on sprdma_and_dmc_dma, I'm hoping maybe by the end of the month those tests will pass.
I have been (very) slowly working on NESHawk PPU. So far I have rewrote sprite evaluation to take place simultaneously with background generation, as it is with a real NES. Together with proper read behaviour of 2004 during this time, I finally managed to fix those annoying horizontal lines and shaking in Micro Machines.
This is still very early WIP work and generally breaks other things, but the timing is correct so i should be able to start slowly fixing other timing issues and bugs and such.
Right now it also slows down emulation by a noticable factor as well. I don't think my code takes considerably longer to run then the original, so this is the first major issue to be worked out.