Hi everyone, in this post I am going to give the most exhaustive tutorial I can give on how to use the Dolphin debugger. Since the documentation available for it isn't much there, expect this to be longer than the
Cheat Engine tutorial I did, I just want to have a nice reference that I can link to.
A debugger is a program (or in this case a module to Dolphin) that allows you to go deep inside another program's execution. You can see what it's doing at specific points, what is being done under what circumstances and what variables the program uses to accomplish its job. This is mostly useful here for research purposes because it was designed to go so deep you could identify bugs on the emulator so logically, you can also use it to identify how a game would work in detail.
But before I start, I need to clarify some things on why should you consider knowing or using the debugger assuming you have Cheat Engine (a ram search) set up already which does already quite a good job for research. For TASing, glitch hunting or general researching on games, the purpose of ram search has been made pretty clear and it doesn't need an explanation on why you should be aware of it, but this isn't the case of the debugger. Hence why I made this additional section for this tutorial.
Why use a debugger
For most cases, you actually don't need the debugger. A ram search like Cheat Engine (besides its limitations) would allow you to monitor and identify the interesting data of the game and its format. This is already enough to:
- Get a testing environment
- Find hidden mechanics (like data that points to features not mentioned in the game)
- Test improbable or impossible scenarios that relates to data
- Monitor changing data (like the RNG seed)
- Have an idea of how data works in general and their format
The other good thing about using a ram searcher is even though it helps, you don't need to know programming or coding to understand how to use it, you just need thinking, guessing and trial and error. Basically, you gain a lot of information with little effort.
However, the debugger is basically the opposite. It takes much more effort to get started and to use it because first, it requires to learn the PowerPC instruction set assembly which is a programming language (you don't need to know how to code in it though, just to read it). Second, It also requires much more effort to use it because the code is very primitive and optimised by a compiler most of the time making analysis more tedious. Basically, the ratio between information gain and effort is less profitable for the debugger than the ramsearch even though you can gain more information.
What you gain however can sound interesting in some cases:
- Being able to see EXACTLY what the game is doing between reads and writes of memory
- Backtrace reads and writes of memory (figuring out what happened before that lead to this read/write)
- Investigate the reasons of crashes
- Know when, how many times and under what circumstances a known function is used (like rolling the RNG)
- Test more scenarios, you have more options because you can hack the execution and the registers
- Find hidden data (like tables put in a not very obvious place in the ram)
- Know all data format (as opposed to having an idea of it)
Also, the debugger is the lowest level you can get to the game's execution. The only thing that is lower is machine code, but considering you have the assembly with the debugger, it's the lowest you should go. Basically, there's hardly any things you won't know with the debugger.
The reason why the above things matter is because there's always going to be cases where an accurate emulator and a versatile ram search is just not enough. Glitches that involve incorrect memory accesses, testing the robustness of sensitive data, finding useful data on the games when its location isn't obvious, etc... To summarise, you will end up NEEDING the debugger at some point if you want to get serious on glitch hunting or researching a game. As far as TASing goes however, it's not needed, but since knowing about the game's inner workings helps with TASing, it does end up being helpful in the end.
Lastly, you should not use the debugger without a ram search, it's not very useful. The reason is to breakpoint at a place you are interested in, you either need to know the instruction address in memory (you are not finding these with Cheat Engine for sure) or you knew it beforehand by using a memory breakpoint on a known address which will suspend the game when the memory has been tampered. Since finding addresses with the ram search is easy, this is where its use becomes required to take advantage of the debugger. So if you are planning to learn the debugger, I highly recommend you follow
my tutorial on how to setup Cheat Engine and get a couple of addresses before, you will need these.
With that, let's begin!
Requirements
As I said, you need to learn the PowerPC instruction set. Unfortunately, teaching it here would be way too long so I will instead give a general explanation and for the instruction set, I will put useful reference links so you can at least get started after reading them.
You also need (as of this post) a very recent version of Dolphin and this is because if you tried to pick stable 5.0, the debugger is
AWFUL in that version. This tutorial was actually made possible by the MANY improvements done to the debugger from September 9 2016 while the most essential one was done within around 2 months after this date (and more might still come after, but not as important). I actually made most of these improvements and bugfixes because it was just very hard to use. As of this post, it's very stable for me to be comfortable to write this.
I recommend (this might be updated if huge changes occur) at least Dolphin 5.0-3023 for good debugging usage. In this post, I am going to assume that you are using that or newer, but if you don't use that and you do experience issues, please update it before posting about them here. If you don't want to bother, just take
the latest development revision off here and it should be mostly good..... unless there's a regression or something. Just please do not use stable 5.0 for this because it would even lack critical features of the debugger that you could only have by building from sources in a debug configuration (which is super annoying to do and is very prone to errors). So if you ever heard that memchecks or memory breakpoint requires a debugfast build, this is no longer the case with newer revisions. I also encourage you to update your revision from time to time, there is still improvements done to it lately as of this post.
Lastly, you need time on your hand when you want to use it. Figuring out things can take hours and take much more time than typical scans with ram search. Considering you need to learn an assembly language however, I don't think this should be a problem by that point :p
So now, I should explain what is the deal with PowerPC assembly and why you need to read it.
The PowerPC assembly
Since commercial games don't give you access to their source code for obvious reasons, the only thing you have is the binary file which has likely been the result of a compiler that converted the source code to assembly in PowerPC. This is why that you need to be able to read it if you want to do any sort of debugging and Dolphin can show you these instructions. As I said, I will not explain the actual instructions, but I will for everything else.
Assembly is a primitive and low level language dependant to the architecture they run in. They do very simple tasks like simple math, conditional flow and read/writes to the ram.
PowerPC assembly is fortunately quite simple to learn (from my experience at least) because it is a RISC architecture (reduced instruction set computing). From what I know, a lot of the instructions are actually a derivation of others and their parameter lists tend to be simple to understand (for example, most of the time, the first parameter is the "destination").
The instructions do operations on ram of course, but also on a special memory present on the cpu called the registers. In the case of the GameCube and Wii, since they are 32-bit systems, they have the following amount of registers:
- 32 32-bit general purpose registers (GPR) named r followed by the register number
- 32 64-bit floating point registers (FPR) named f followed by the register number
- Several 32-bit special purpose registers (SPR) called differently based on what they are (a couple of interesting one are the link register (LR), the program counter (PC), the condition register (CR), the count register (CTR), etc...
And some others, but you aren't likely going to need to know about them.
The GPR can contain any data except double precision floating point, but the FPR can contain them (except they won't accept anything else, even single precision floating point will be converted to double). As for the SPR, they have their own format most of the time.
As for data types, since we are dealing with a 32-bit system, they are the following:
- Word (4 bytes)
- Halfword (2 byte)
- Byte (1 byte, duh)
And lastly, these systems work on the big endian format so you read numbers as you would normally.
So once you know this, you should be ready to learn about the instructions set. Here's some useful references to learn about it:
Once you have understood most things, you are ready to use the debugger WOOOO!!!
Starting the debugger
To start Dolphin with the debugging features enabled, you simply have to pass the parameter "-d" when starting it, either use a terminal or create a shortcut/launcher that does that. It shouldn't affect performance at least from what I saw, but it will load all features that you need (and a lot more actually).
Overview of the debugger
You should be greeted by a slightly different UI (it will look different than this shot, but it's going to look similar):
As far as UI is concerned, you gain:
- In the "Options" menu, "boot to pause" and "automatic boot", which are self explanatory, and also an option to choose the debugger font. This font will be used with all the debugger related panels (so you can be like me and put Hey Gorgeous everywhere :D ).
- A new "Jit" menu allowing you to configure the Jit (the recompiler). Unless you know what you are doing, you won't likely touch this and this isn't covered in this tutorial, so just leave it to default unless you need to change options
- A new "Debug" menu which has the same options as the new toolbar buttons on the left + a toggle breakpoint option and also a way to configure the UI perspective which I will talk about a bit later (there's some not so obvious stuff about it, but basically, you can save the UI arrangement)
- A new "Symbols" menu which allows managing symbol maps. There's a lot in here, but the only ones you will likely need are the basic load, save and generate options. View symbols as "bookmarks" for the assembly view which will become very useful when I will talk about them later
- A new "Profiler" menu, which unless you need to test the performance of the emulator, you won't need to mess with it
- New buttons will appear on the left side of the toolbar which are "Step", "Step Over", "Step Out", "Skip", "Show PC" and "Set PC". These are likely to become your friend soon, since they are the main debugging control features. More will be explained on them later, also, it's normal if they don't have icons, this is known already
- The "View" menu has new options for all the new debugging panes, most of them are explained later in this post
- There's a new tab in the hotkeys windows called "Debugging" which gives shortcut for the stepping functions, Show PC, Skip or add either kinds of breakpoints. This may make it more convenient for you so definetely consider setting these.
Before I dive in the actual controls, I should first describe the not so obvious stuff about the perspectives because they are going to really help the convenience of debugging
Perspectives
Even though you can drag and move panels around, they won't be saved until you use a perspective and save it. This is very useful for not having to mess around in the view menu and dragging your panes the same way. One is created by default, but you can manage them (save, create, etc.) in the Debug->perspectives menu.
Now, this is the important part, you need to be in "edit perspectives" mode to make changes to the perspective. This is where you can add panel groups, position them, remove them, etc... Also, if you want 2 different panes positioned at different places, you can't put both in the same group, but have one at the top and one at the bottom. To have that saved, you HAVE to create 2 tabs groups, one at the bottom and one at the top and THEN you put your panels in here. I felt I needed to make an emphasis on this because honestly, it's not obvious to figure out and it's going to save you a lot of menuing since you will likely make use of 3 panes and it can be very interesting to have them visible at once. To save, just toggle off the edit mode and do the save perspective option. When you reboot Dolphin, it should have loaded everything.
Last note on the panel UI, they are draggable and can be docked, but they can also be floating on their own window, just right click on the concerned tab to do it.
The debugger panels
They are actually pretty self expanatory and you are not likely to use all of them. The panels are
- Register view, this is going to become very useful later, it basically displays the values of the registers
- Watch view, this is a kind of like ram watch except it's not very good considering you will likely use Cheat Engine which is much more powerful and can update in real time. I don't recommend using it except if you want to have a watch when stepping and you don't want to have Cheat Engine take space on your window, but that's the only case I would use it
- Breakpoint view, this panel is important because it contains your memory breakpoints, allows to add them and it also lists breakpoints which upon clicking on them will bring you to them with the code view
- Memory view, again, I don't recommend to use it over Cheat Engine's VERY useful memory viewer with configurable hex view. Again, the only time I would use it is if you wanted to check once the memory of something and add a memory breakpoint to it, but tbh, Cheat Engine does much more than that so I guess you could use it to verify you did calculations right on converting a Cheat Engine address to Dolphin emulated address, but that's it
- Jit, Sound and Video view, they are mostly used to debug the Jit (with the x86 instruction viewer), to debug the DSP (digital sound processing) and the graphics rendering. Needless to say, unless you want to debug the emulator, you won't use these at all
Finally, there is one panel so useful it's always on, the Code view.
This is where most of the actions happen, because you see the assembly code, a call stack, a list of your symbols and information on what function is called within the current one as well as seeing what calls the current functions. This view should always be within sight which is why it becomes useful to have the register view and breakpoint view saved on a perspective since the code view tells information about manipulating registers and for breakpoints, it's just nice to have easy and quick access. The memory breakpoints will serve as the link between your ram search and the debugger so it's very nice to have it around.
So put this all together, the panels you will likely use are the code view, the register view and the breakpoint view. That's all you need for most of your debugging needs, but you might also want to have the logger around. It can be useful sometimes, for example when investigating a crash.
Let's start explaining breakpoints because this is where it all begins.
Breakpoints and memory breakpoints
A breakpoint is a point set by a debugger that once the program will reach it, it will trigger causing the suspension of the execution so you can start to monitor it. There are two kinds of breakpoints in the Dolphin debugger: instruction breakpoints (BP) and memory breakpoints (MBP) also formely known as memcheck (MC) in older versions. The BP breaks upon reaching a particular instruction and the MBP breaks and/or logs when reading and/or writing to a memory or a range of memory.
You can add both of these using the +BP and +MBP buttons in the breakpoint view. For the BP, you just need the address, but there's also another way to add one that is more convenient, you just click on the leftmost spot (next to the address) of any instruction in the code view and a red circle should appear. For the MBP, you need to provide more information, but this should be self explanatory, do note however that if you want to log, you need to have the log "MI & memmap" enabled in the log configuration at any level to see them. It can be useful to want to not break, but log the accesses, especially if they are particularly frequent. However, if they are too frequent, it can clog the logs to the point where performance goes down A LOT so just be careful when logging without breaking the MBP, you would need several logs per frame for that to happen though. You can also use hotkeys to add them if you prefer, this can be especially usefull for MBP.
Speaking of performance, be careful when using MBP in general, they do slow down performance, but the difference is very minimal now to the point where it's actually hard to notice it, just be carefull still. Luckily, you usually don't need MBP for a long time since you can rely on BP after you find the read/writes you are interested in.
So the idea is you enable your MBP when you need it and once you are satisfied with where it broke, you can remove it and use BP instead, which causes no performance penalties. MBP are still a very critical feature of the debugger. You would know the interesting addresses already using a ram search, you just check with MBP when they are touched and then you don't need them anymore. Because of this, you will in practice use BP most of the time, even though the MBP are essential to use.
Also, be careful when you choose the address to use of the MBP, it will work if you pick the first byte of the data you want, but if you break in the middle of it.... it can lead to really weird results and you will know by having the MBP not work, but Cheat Engine clearly shows a change for example. It can also seem to corrupt the ram around it, so basically, if you get weird stuff going on, recheck the address you picked. The data you want might take more bytes than you think. this got fixed in 5.0-3023, it will work even if it's not necessarly the first address of the data since the size of it is now taken into consideration
One last thing about MBP, you can also add one by clicking on the leftmost spot (next to the address) of a line of the memory view after setting its options on the right side, but this doesn't allow to add by range.
The registers view
There isn't much to tell about it except that it shows register values. However, there's a couple points you should know.
First, you don't really care about the left FPR cells, but you care about the ones on the right. This is because the right one contains the values, but the left one contains stuff about paired instructions which are exclusive to the cpu's in the GameCube and Wii. Second, even though all registers by default are displayed in hex format, you can change the displayed format by right clicking on them (the GPR allows signed and unsigned integer format as well as single precision floating point while the FPR allows to see them in double precision floating point). To change them, you just double click and you can input new values and lastly, you can jump in the code view to the address in a register by right clicking on it.
As for why it says (sp) and (rtoc) for r1 and r2, it's because the code view doesn't call them r1 and r2, but rather sp (stack pointer) and rtoc (register of the table of content). You should be aware of what this means if you learned the PowerPC assembly, but basically, it's just a meaningful identifier to them.
The code view
This is the most important panel of the entire debugger because this is where you will see your assembly code. For obvious reasons, it can only show them when the emulator isn't running so when pausing or stepping. In that case, it's normal to see a bunch of <unknown> lines.
For the lines themselves, you have the instruction address on the left, the operation code with all its parameters in the middle and some lines to visually see branches. If you have a generated symbol map, you will also see the symbols name in blue, whether for the curent function or the one that is being called on a line. Symbols will also be visually indicated by random colors between each of them so the first thing to do when booting a game for the first time with the debugger is to generate a symbol map and to save it.
About symbols, as I said earlier, they are kind of like bookmarks for the different functions. Dolphin, when generating a map will define a symbol that starts from the address of a function call (usually a bl instruction) and ends at the last return of the function (usually a blr instruction). There may be inaccuracies, but overall, it's pretty accurate so it really helps to be able to identify blocks that are most likely a function. After generating a symbol map, you should see them appear on the symbols panel on the left of the code view. To jump to the beginning of the symbol, you just select it in the list. It also allows having them appear on the function calls and function callers list.
Dolphin already has a preincluded list of symbols, but you will notice most of them are by default named zz_[address]_ and this is because, although Dolphin can detect functions, it of course can't guess the name. This is when renaming symbols becomes useful. After figuring out an important function, you can right click to an instruction in it and choose "rename symbol". That way, you have a more friendly name to it so when you will read other codes, you will literally see it written when it gets called, avoiding having to go through the trouble of figuring it out again.
It should be noted however that some games "leaked" their symbols maps, the file that allows you to see each entrypoints and name of every functions in the game, it's a text file that usually ends with a .map extension, but there's several kinds out there. If you have such file, the file can be loaded using the "Load Other Map File..." option. You can even partially load a map file that would only contain a part of valid symbols by using the "Load Bad Map File..." and Dolphin will tell you how much was loaded and how much was discarded. This may be usefull when you have a map file from a different build of the game that you currently use since Dolphin will load all the symbols that mathes the game's code only. You will still end up with some generated symbols, but that is still way better than having all the symbols generated. Having the true symbols map makes debugging A LOT easier because their names usually describe the functions so you can more concentrate on their code rather than trying to guess what they do.
When you right click in a line, you have several options. "Follow branch" only applies to branching instructions and allow jumping to where they are going to branch to. You have several self explanatory copy options, a way to rename the symbol, have a temporary breakpoint and run until you hit it, a way to see the Jit instruction (so, you likely won't care unless you are debugging the emulator), a way to have the current line be its own symbols and 2 insert options. These insert options will replace the current line with either a nop or a blr which can be very useful when you want to bypass function calls or skip a part of a function, nop will do nothing (so it disables the line) and blr will force a return to the caller.
A little note on the instructions, first, as I said earlier, r1 is called the sp and r2 is called the rtoc. If the instruction involves relative addressing with a register, it will show the offset in hex and then the register in parentheses that is the base. Also, rlwimn instructions have a little bit of a special notation, it will show the parameters, but then put a hex number in parantheses. This is the mask that will be used based on the last 2 parameters, a convenient notation. Also, unless it doesn't say "0x", every number is in hex.
Lastly, one very useful panel on the left is the callstack. It's basically showing the stack of calls that was done from the current point, very useful when backtracing a memory access or a crash.
Stepping controls
Stepping in the debugger is what makes the debugger control your execution. Every debugger I know of have the 3 classic stepping features: step in, step over and step out. Step in (or "Step" here) is basically, that you execute the current line and stop before executing the next one. This allows to literally execute the game line by line and seeing all the little changes that would happen after each line. In fact, the register view will even display changed reigsters since the last time Dolphin was paused in red, just so you see what changed. Step over is the same as single step, but it will skip function calls by instead running the entire function, then stopping after the call, otherwise, it's the same as single step. Step out however is a bit different, it will run until you return from the current function and stop after the call instruction. BP and MBP will also be hit, except when stepping out and having an MBP in the way, but otherwise, all your breakpoints will be hit when running more than a line. If step out takes too long though, it will eventually timeout and a message will be displayed on the status bar in such case.
The Dolphin debugger also has other nice execution control features. Skip will do a step over, but instead of executing the current line, it will skip it entirely. This is of course not following the normal execution so be careful with it. Show PC (program counter, aka the current instruction) will just have the code view show you the PC in the center, very useful when you get lost. Lastly, you have Set PC which is very dangerous, but very powerful at the same time since it allows to literally jump ANYWHERE in the code by saying that the current line is the current PC and the execution will now go from there. So you could jump around in the code at multiple places, just be careful with it, it's basically prone to crashes depending on the game because you destroy its execution. You can add hotkeys for all these controls as well as all stepping controls so you wouldn't have to move your mouse all over the place to click the buttons,
And that's pretty much it, that's all of the debugger controls, let me do a summary of what to take from that.
Putting all this together
You first need to have found addresses with a ram search and have a memory viewer handy (the Cheat Engine one is good) becuase you might want to check what's in some addresses that the registers would have. Also, bring a calculator in programming mode, you will likely use it a lot.
The first thing you want to do before doing any work is pause the game, generate a symbol map and save it. This is going to be very useful to know function blocks and to more easily identify them. If you get a lot of errors, just try again with panic handlers off (just go to configure in the interface tab to see it). They are annoying and from my experience, don't do much when generating.
So, from that point, you would add a memory breakpoint to one of the addresses or ranges you already figured out, configure it to what you are interested in and just do something in the game that will likely trigger the breakpoint. In the event that it does trigger it, the emulator will pause and the code view will show where it stopped. From that point, get rid of the MBP as soon as possible and use BP instead since MBP decreases performance. Then you can do lots of things. Step into the function to analyse it (have the register view near and remember that you can change their displayed format), check the stacktrace to know what happened before, add other BPs, etc... You also have the option to have the MBP log without a break if you want.
Once you have figured out an important function, you can rename the symbol so you will remember it for when you see it in the future. Don't forget to save the symbol map.
And that's pretty much how you would use the debugger most of the time. As you can see, it's very powerful, but even though effort is required to learn it and most cases for research doesn't need it, some will have you end up needing to use it and you would be happy to know how.
Tips and tricks
Since I did work on improving the debugger and I have used it several times myself, I should also do a list of stuff that should help you in the end that took me quite a long time to figure out so you won't have to be like me here:
- Pay attention to the logs ESPECIALLY the ones that says "OSREPORT". These are not logs made by Dolphin, but rather Dolphin reporting game logs that the game itself has. They are game dependant, but you can sometime see useful stuff like a report on a crash
- Ideally, you want the register, breakpoint and code view in the same main window (or at least have them near each other). These panels are the most important panels of the debugger so it just make sense to have them nearby
- Pay attention to pointers in the register values. If it seems like a valid memory address, it probably is one, so you should check what it is from the Cheat Engine memory viewer. You might see familiar data that you saw before, in which case, you should take note of that
- Don't spend all your time on figuring out the reasons each instruction are written so. Instructions that move stuff to the stack usually are just the usual function initialisation and aren't as important as, say taking parameters into registers or read/writes. What you are trying to do is summarise what the function is trying to do, which is already hard enough because it's in assembly. Don't make it harder, a lot of the time, complicated instructions actually do a simple thing all together
- Compilers often have rlwimn instructions actually be a shift left in disguise. If it rotates left by the same amount of bits that is discarded to the right of the mask, it's a shift. Usually, these are done to multiply something by a power of 2
Special thanks
I need to thank the Dolphin dev team here. Even though it is more accurate to say my contributions made this tutorial possible, the reality is I really enjoy learning my way within the codebase and to learn what it is like to contribute to a FOSS project because of how much fun it is to work with them. I had a lot of questions, but they were very welcoming to answer me which really ended up to have the debugger in my opinion much more usable than it was before so we can now do some more deeper research.
In case you still have issues
If you got the latest development revision and you still have issues not mentioned in this post, I would ask you that you would please either post here and I can answer or pm me either on this forum or on #tasvideos irc (I have a bnc so don't worry if I am not here). From what I know, I am one of the few people that is very familiar with how the debugger works which means that if you notice issues with it, I will probably be able to troubleshoot it or maybe debug it. You can also post an issue on their tracker and link it to me if you prefer, just make it clear what needs to be done to reproduce it.
Hope that this will allow a more accessible debugger for people to use :)
