Pokemon Yellow Total Control Hack. Reprogramming the game from the inside!

Game objectives

  • Emulator used: vba-rerecording 23.5
  • Reprogram the Game from the inside

Comments

I've included a detailed writeup here: http://aurellem.org/vba-clojure/html/total-control.html
The following are the highlights:

Introduction

Think of pokemon yellow as creating a little universe with certain rules. Inside that universe, you can buy items, defeat rival trainers, and raise your pokemon. But within that universe, you are bound by the rules of pokemon. You can't build new buildings, or change the music, or change your clothes.. There are some games (like chess), where it is not possible to alter the rules of the game from within the game. No matter what moves you make in chess, you can never change the rules of the game so that it becomes checkers or basketball. The point of this run is to show that you CAN change the rules in pokemon yellow. There is a certain sequence of valid actions (like walking from one place to another or buying items) that will allow you to transform pokemon yellow into Pacman, or Tetris, or Pong, or a MIDI player, or anything else you can imagine.

Background

The speedrun (#2913: p4wn3r's GBC Pokémon: Yellow Version "save glitch" in 01:36.95) by Felipe Lopes de Freitas (p4wn3r), beats pokemon yellow in only 1 minute and 36 seconds. It does it by corrupting the in-game item list so that he can advance the list past its normal limit of 20 items. The memory immediately after the item list includes the warp points for the current map, and by treating that data as items and switching and dropping them, he can make the door from his house take him directly to the end of the game.
When I first saw that speedrun, I was amazed at how fast pokemon yellow could be beaten, and that it was possible to manipulate the game from the inside, using only the item list. I wondered how far I could extend the techniques found in p4wn3r's run.
The gameboy is an 8 bit computer. That means that ultimately, anything that happens in pokemon is a result of the gameboy's CPU reading a stream of 8 bit numbers and doing whatever those numbers mean. For example, in the gameboy, the numbers:
62 16 37 224 47 240 37 230 15 55
mean to check which buttons are currently pressed and copy that result into the "A" register. With enough numbers, you can spell out an interactive program that reads input from the buttons and allows you to write any program you want to the gameboy. Once you have assembled such a program and forced the game to run it, you have won, since you can use that program to write any other program (like Tetris or Pacman) over pokemon yellow's code. I call a program that allows you to write any other program a "bootstrapping program". So, the goal is to somehow get a bootstrapping program into pokemon yellow and then force yellow to run that program instead of its own.
How can we spell out such a program? Everything in the game is ultimately numbers, including all items, pokemon, levels, etc. In particular, the item list looks like:
item-one-id         (0-255)
item-one-quantity   (0-255)
item-two-id         (0-255)
item-two-quantity   (0-255)
.
.
.

Let's consider the button measuring program [37 62 16 37 224 37 240 37 230 15 55] from before. Interpreted as items and item quantities, it is
lemonade     x16
guard spec.  x224
leaf stone   x240
guard spec.  x230
parlyz heal  x55
So, if we can get the right items in the right quantities, we can spell out a bootstrapping program. Likewise, when writing the bootstrapping program, we must be careful to only use numbers that are also valid items and quantities. This is hard because there aren't many different items to work with, and many machine instructions actually take 2 or even 3 numbers in a row, which severely restricts the types of items you can use. I ended up needing about 92 numbers to implement a bootstrap program. Half of those numbers were elaborate ways of doing nothing and were just there so that the entire program was also a valid item list.
The final part of the hack is getting pokemon yellow to execute the new program after it has been assembled with items. Fortunately, pokemon keeps a number called a function pointer within easy reach of the corrupted item list. This function pointer is the starting point (address) of a program which the game runs every so often to check for poison and do general maintenance. By shifting an item over this function pointer, I can rewrite that address to point to the bootstrapping program, and make the game execute it. Without this function pointer, it would not be possible to take over the game.

The Run

Pallet

I start off and name my rival Lp/k. These characters will eventually be treated as items and shifted over the function pointer, causing it to execute the bootstrapping program that will soon be constructed. I start the run the same as p4wn3r's and restart the game while saving, so that the pokemon list is corrupted. By switching the 8th and 10th pokemon, I corrupt the item list and can now scroll down past the 20th item. I shift items around to increase the text speed to maximum and rewrite the warp point of my house to Celadon Dept. Store. (p4wn3r used this to go directly to the hall of fame and win the game in his run.) I deposit many 0x00 glitch items into the PC from my corrupted inventory for later use. Then, I withdraw the potion from the PC. This repairs my item list by overflowing the item counter from 0xFF back to 0x00, though the potion is obliterated in the process. I then take 255 glitch items with ID 0x00 from the computer into my personal items.

Celadon Dept. Store

Leaving my house takes me directly to Celadon Dept. store, where I sell two 0x00 items for 414925 each, giving myself essentially max money. I hit every floor of the department store, gathering the following items:
 +-------------------+----------+
 |##| Item           | Quantity |
 +--+----------------+----------+
 |1 | TM02           |  98      |
 |2 | TM37           |  71      |
 |3 | TM05           |   1      |
 |4 | TM09           |   1      |
 |5 | burn-heal      |  12      |
 |6 | ice-heal       |  55      |
 |7 | parlyz-heal    |  99      |
 |8 | parlyz-heal    |  55      |
 |9 | TM18           |   1      |
 |10| fire-stone     |  23      |
 |11| water-stone    |  29      |
 |12| x-accuracy     |  58      |
 |13| guard-spec     |  99      |
 |14| guard-spec     |  24      |
 |15| lemonade       |  16      |
 |16| TM13           |   1      |
 +--+----------------+----------+
After gathering these items, I deposit them in the appropriate order into the item PC to spell out my bootstrapping program. Writing a full bootstrap program in one go using only items turned out to be too hard, so I split the process up into three parts. The program that I actually construct using items is very limited. It reads only from the A, B, start, and select buttons, and writes 4 bits each frame starting at a fixed point in memory. After it writes 200 or so bytes, it jumps directly to what it just wrote. In my run, I use this program to write another bootstrapping program that can write any number of bytes to any location in memory, and then jump to any location in memory. This new program can also write 8 bits per frame by using all the buttons. Using this new bootstrap program, I write a final bootstrapping program that does everything the previous bootstrapping program does except it also displays the bytes it is writing to memory on the screen.

Finale

After completing this bootstrapping program, I go to the Celadon mansion, because I find the metaness of that building to be sufficiently high to serve as an exit point for the pokemon universe. I corrupt my item list again by switching corrupted pokemon, scroll down to my rival's name and discard until it is equal to the address of my bootstrapping program, and then swap it with the function pointer. Once the menu is closed, the bootstrapping program takes over, and I write the payload....

Other comments

The entire video was played by the computer using bots. I used functional programming to write search programs over different possible game states to find the most efficient way of performing general actions. Some interesting things I developed but didn't use were pretty printing functions to display the game's internal data structures, and an "improbability drive" that forces improbable events to happen automatically using search.
Here are a few example scripts:
 (defn-memo viridian-store->oaks-lab
   ([] (viridian-store->oaks-lab
        (get-oaks-parcel) ) )
   ([ script \]
      (->> script
           (walk [↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ← ← ← ← ← 
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ←
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  → → → → → → → →
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ←
                  ↓ ↓ ↓ ↓
                  ])
           (walk-thru-grass
            [↓ ↓ ↓ ↓ ↓ ↓ ↓])
           (walk [↓ ↓ ← ↓ ↓ ↓ ←
                  ↓ ↓ ↓ ↓ ↓ ↓
                  → → → ↑])
                 
           (do-nothing 1) ) ) )
This script walks from the Viridian City pokemon store to Oak's Lab in the most efficient way possible. The walk-thru-grass function guarantees that no wild battles will happen by manipulating the game's random number generator.
 (defn-memo hacking-10
   ([] (hacking-10 (hacking-9) ) )
   ([ script \]
      (->> script
           begin-deposit
           (deposit-held-item 17 230)
           (deposit-held-item-named :parlyz-heal 55)
           (deposit-held-item 14 178)
           (deposit-held-item-named :water-stone 29)
           (deposit-held-item 14 32)
           (deposit-held-item-named :TM18 1)
           (deposit-held-item 13 1)
           (deposit-held-item 13 191)
           (deposit-held-item-named :TM02 98)
           (deposit-held-item-named :TM09 1)
           close-menu) ) ) 
 
This script calculates the fastest sequence of key presses to deposit the requested items into a PC, assuming that the character starts out in front of a computer.

Other Comments

The final payload program is multiple programs. I created a reduced form of MIDI and implemented it in gameboy machine language. Then I translated a midi file from [dead link removed] into this reduced MIDI language. The payload program contains both the music data and the MIDI interpreter to play that data. The picture works in a similar way. There is code to translate a png file into a form that can be displayed on a gameboy, and other code to actually display that image. Both the image and the display code are also written by the final bootstrapping program. Even though my final payload is rather simple, you can write any program at all as the payload. The source for the sound and image displaying code is at http://hg.bortreb.com/vba-clojure
This entire project is open source and I encourage anyone who wants to take the code and play around!

Suggested Screenshots

Or whatever you all think would be best.
I encoded the video with/without button visualization here:

FractalFusion: Replaced movie file to fix time (note that the parser works in such a way so that the time listed for a VBM can easily be faked).
FractalFusion: Response is well-received for the new concept. Accepting for publication.
natt: Processing...
FractalFusion: Changed branch to "Executes Arbitrary Code".


1 2 3
7 8
AnS
Emulator Coder, Experienced player (728)
Joined: 2/23/2006
Posts: 682
It's been 5 years since people voiced similar dreams, and now it's reality. http://tasvideos.org/forum/viewtopic.php?t=5939 This is the most epic scale of TASing so far. It blurs boundaries between TASing and hacking.
Skilled player (1743)
Joined: 9/17/2009
Posts: 4986
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
Emulator used: vba-rerecording 23.5
Wait. Is this version accurate? I remember using that to TAS Wario Land 2 with MUGG, and I think it gave a different result for the OoB glitch in the GBC version between VBA 23.5 and 24. Here's the quote:
Eventually we ended up with an awesome WIP, but after talking to klmz I realized that the OoB areas were incorrectly emulated on VBA 19~23 and the WIP basicly died...
Edit: bortreb, can you try repeating this on VBA 24 if you have time? Edit 2: Any other games that could be glitched up like this? :D
Joined: 5/2/2009
Posts: 656
simply amazing
My first language is not English, so please excuse myself if I write something wrong. I'll do my best do write as cleary as I can, so cope with me here =) (ノಥ益ಥ)ノ
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Of all the numerous codes you could inject into the game you chose to inject MLP. You should definitely be burned alive for this heresy, but for now, a Yes vote for this run will suffice. Anyway, this seems to be the same video you posted on the Pokemon thread some time ago. I think the run would be better if you followed the suggestion in that topic to manipulate the RAM faster and optimize the shellcode. But well, if speed is not important in this movie, then speed is not important in this movie. Congratulations and hoping for a high tier. If the number of Pokémon movies is a problem, then we could just pick the most boring one and get rid of it.
Spikestuff
They/Them
Editor, Publisher, Expert player (2656)
Joined: 10/12/2011
Posts: 6449
Location: The land down under.
p4wn3r wrote:
Of all the numerous codes you could inject into the game you chose to inject MLP.
I was upset that it wasn't cupcakes
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2178
Location: A little to the left of nowhere (Sweden)
p4wn3r wrote:
Of all the numerous codes you could inject into the game you chose to inject MLP. You should definitely be burned alive for this heresy, but for now, a Yes vote for this run will suffice. Anyway, this seems to be the same video you posted on the Pokemon thread some time ago. I think the run would be better if you followed the suggestion in that topic to manipulate the RAM faster and optimize the shellcode. But well, if speed is not important in this movie, then speed is not important in this movie. Congratulations and hoping for a high tier. If the number of Pokémon movies is a problem, then we could just pick the most boring one and get rid of it.
He stated in the topic that he probably wont have time to improve this for a long time. And it was agreed upon that it would be best for now if this version was submitted.
TheKDX7
He/Him
Player (118)
Joined: 7/9/2011
Posts: 393
Location: Switzerland
I did not really understand the objective of the TAS, is it a playaround or?
Active player (407)
Joined: 3/22/2006
Posts: 708
11 minutes of a bunch of weird boring stuff that's impossible to follow, and then I got Brony-rolled... This isn't April Fools Day, is it?
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
p4wn3r: I thought to myself, "what's the most trolltastic thing I could inject," and came up with mlp. There's two alternate songs at http://aurellem.org/pokemon-hack/mother.wav, and http://aurellem.org/pokemon-hack/regret.wav jlun2: I'd love to run it on a newer version of vba (and suspect that it would work, since the hack is robust), but their autotools scripts are broken, preventing me from compiling it for GNU/Linux. When I started the project, I downloaded 23.5 and spent a week learning autotools and fixing their script. That's what I used to make the movie. I emailed the vba guys and offered to contribute my fixes, but never heard back from them. Anyway, If someone with a working vba24 would check this, I would greatly appreciate it. Also, the only part that may suffer from emulation problems is the "restarting glitch" used at the beginning, taken from p4wn3r's run. I believe that someone actually did that on a console if I'm not mistaken?
Skilled player (1743)
Joined: 9/17/2009
Posts: 4986
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
bortreb wrote:
jlun2: I'd love to run it on a newer version of vba (and suspect that it would work, since the hack is robust), but their autotools scripts are broken, preventing me from compiling it for GNU/Linux. When I started the project, I downloaded 23.5 and spent a week learning autotools and fixing their script. That's what I used to make the movie. I emailed the vba guys and offered to contribute my fixes, but never heard back from them. Anyway, If someone with a working vba24 would check this, I would greatly appreciate it.
Uh..try PMing "klmz", since he seems to be the only one working on VBA-rr Development atm.
Joined: 2/28/2012
Posts: 160
Location: Philadelphia
Yes vote because this TAS really changed my idea of what could be done with TASing and completely floors me to think about.
Editor
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
This is what TAS can do. Screw beating the game, let's make a NEW game by using the old game's bugs.
When TAS does Quake 1, SDA will declare war. The Prince doth arrive he doth please.
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
bortreb wrote:
Also, the only part that may suffer from emulation problems is the "restarting glitch" used at the beginning, taken from p4wn3r's run. I believe that someone actually did that on a console if I'm not mistaken?
Do you mean the reset on the first frame I used to manipulate the trainer ID? If I don't reset in v23, I'll get the ID 1 unit smaller, which won't work. In v24, the ID is always the value that doesn't work, no matter where I reset. I have a good guess to why that happens. The RNG depends on the GB divider register, that works like a timer. Possibly, when you reset in v23, it probably leaves some time ticks before starting emulation again, so the register is incremented by one a little earlier, if earlier means before the luck manipulation frame it'll increase the RNG by 1. In v24 they changed resets, now it probably clears the time ticks after a reset, so the trick won't work. That's some dirty emulation trick in the TAS. I don't know if the DIV register always starts cleared after a hard reset, so I'd just consider it a nitpicky emulation case. Besides, if the RNG is sensitive to timing variations of that magnitude, chances of console verifying TASes of this game are almost negligible.
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
p4wn3r wrote:
bortreb wrote:
Also, the only part that may suffer from emulation problems is the "restarting glitch" used at the beginning, taken from p4wn3r's run. I believe that someone actually did that on a console if I'm not mistaken?
Do you mean the reset on the first frame I used to manipulate the trainer ID?
I mean the restart at about 52 seconds in which overwrites the pokemon list and other values with 0xFF. I don't need any trainer ID manipulation for this run, just a way to blow past the end of the item list. You could do this with the ZZAAZZ glitch too. In summary, you can do this hack as long as you can corrupt your item list, there's nothing else special needed.
Player (42)
Joined: 12/27/2008
Posts: 873
Location: Germany
Ah, in that case yes, it's been verified that save corruption does that. IIRC you just need to change the reset inputs in the .vbm from the old format to the new one and it'll work in v24.
Active player (328)
Joined: 2/23/2005
Posts: 786
This was totally nuts! A new giant leap for TASing, for sure. Blows the mind. My only criticism is that I think the hacked program could have been a little more complex; maybe with a slideshow of pictures and more music. This being the first TAS of its kind, it could have made an even bigger impression. Also, maybe after the surprise at the end, you could have returned to the game and completed it, just to make it look like the surprise was part of the game. I wonder how many other games will have something like this possible, especially on systems like the SNES with more RAM space. I think the "S" in "TAS" is starting to stand for "Showcase", since more and more of them are blurring the lines of "completing" the game.
Player (12)
Joined: 11/23/2012
Posts: 94
When I saw this run's initial time of just a few seconds, I assumed that it was some terrible newbie thing and skipped the description, but the comments suggested otherwise. Scrolling up, I saw an encode, and looked at it. The technical quality was amazing, but the ending disappointed me. If published as is, I would rate it 10 for technical achievement and 2 for entertainment. I have had rotten experiences with the MLP fandom. I will try not to derail this thread too much with off-topic crap, but in summary, I just don't like the animation style and the overt sugariness that exists to make the cartoon double as a marketing device to young girls, and would prefer a cartoon like Powerpuff Girls where these sugary qualities are not played so straight. Unfortunately, an early encounter with a brony gave me the impression that my opinion on the show was worth absolutely nothing because it was based on watching the wrong episodes, which has fueled a virulent dislike for the omnipresently pushy fandom, and for the show via association. I've since met some more polite bronies whom I have nothing but respect for, but the gag reflex remains. My point is that the negative emotions associated with the fandom are so strong, to the extent that it can take a crapton of conscious effort to maintain diplomacy whenever the topic of MLP is involved. I can't really find myself liking a TAS that glitches out the game and does nothing more with it than play the MLP:FiM theme. I agree with CtrlAltDestroy that maybe something more complex could be done. How about invoke a number of different franchises instead of just this one that leaves a horrible taste in my mouth? The Brain Age TAS represents a lot of different franchises in the artwork which is used in lieu of writing actual numbers. If you invoked Kirby, Mario, Link, Samus, Sonic, or a bunch of other random crap somehow, the appeal would be broadened. You could even include MLP, and I'd be willing to forgive it, because it's not 100% the focus of the punchline. (If the Family Feud TAS included an MLP reference, I'd be cool with it. If every single answer were MLP, I'd be annoyed to death and find almost no entertainment value.) Alternatively, just make a TASVideos logo/reference, or a reference to some Pokémon thing that happened after Pokémon Yellow's release (Mudkip? Zekrom and Reshiram?), so as to maintain the sense of "Okay, I don't think THAT was deliberately put into the game by the developers" that MLP has, without unnecessarily provoking the wrath of the hatedom. The target audience is, after all, TASVideos users and Pokémon fans. On the other hand, you outright admitted that you chose MLP for the sole purpose of being "trolltastic", so I doubt my opinion is doing anything constructive here. DNFTT and all that.
Editor, Expert player (2080)
Joined: 6/15/2005
Posts: 3284
I played this run back in VBA and looked at the memory. I'm very pleased with how it works. Basically, this run hijacks the program counter so that it starts executing from RAM (which should never happen). The program counter is redirected to D53A (not at first, but close enough) to execute the first bootstrap program (~90 bytes). It writes the second bootstrap program at D162 (~200 bytes) and jumps there. It then writes the third bootstrap program at DAD9 (~730 bytes) and jumps there. Finally, it writes (while visibly showing the hex) the payload program and data at C000 (~4300 bytes) and jumps there. It actually starts writing a bit behind C000 but it doesn't matter since only the region C000-DFFF can be written to. Unfortunately, this region is only 8K, and this run uses over half of it for the payload. Compare the Super Mario Land ROM, which is 64K. Of course space optimization is a possibility, as well as just overwriting RAM data once it has been used up. By the way, if you play the run back, if you leave it running after it finishes the MLP jingle, weird sounds start playing back as the program reads sound data from uninitialized RAM. This isn't really a surprise though.
Joined: 3/18/2006
Posts: 971
Location: Great Britain
Cool run. What is "mlp"?
Post subject: The slow clap for antd begins *clap*
Spikestuff
They/Them
Editor, Publisher, Expert player (2656)
Joined: 10/12/2011
Posts: 6449
Location: The land down under.
antd wrote:
Cool run. What is "mlp"?
this
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Post subject: Re: The slow clap for antd begins *clap*
Player (34)
Joined: 3/8/2012
Posts: 398
Location: Windfall Island
Spikestuff wrote:
antd wrote:
Cool run. What is "mlp"?
this
AHHH MY EYES! Sorry, I thought I saw some previous gen ponies on there. Scared me.
IronSlayer wrote:
Your counterargument would be like me saying that the Earth is round and then you telling me that I need to show it's flat so I can "prove us all wrong".
Seems legit.
Skilled player (1743)
Joined: 9/17/2009
Posts: 4986
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
mathgrant wrote:
Text
I'm not sure what to comment about, except what's "DNFTT"? Also, any other game that can also do something like this?
Player (12)
Joined: 11/23/2012
Posts: 94
jlun2 wrote:
I'm not sure what to comment about, except what's "DNFTT"?
Do Not Feed The Trolls. It means that trolls get their jollies from knowing that they've offended people, and the only way to make them stop is to not show a sign of being offended so they get bored. Gosh, I'm really sorry to have derailed the conversation so much by using an abbreviation. I just hoped to express my opinion on this run (that it would have greater appeal with a punchline besides nothing but ponies) without making every brony in the universe ever offended at me for being a heathen who doesn't worship their show. :)
Also, any other game that can also do something like this?
I sure hope so. It would be interesting if Super Mario Bros. were as broken as Pokémon Yellow, and a TASer suddenly made Sonic appear.
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
jlun2 wrote:
Also, any other game that can also do something like this?
IMO you'd probably want to look at games that have function pointers in RAM and some user editable feature with a lot of degrees of freedom, like an item list or level editor. Quite a lot had to go right for me to take over pokemon, though, like 0x00 being a no-op and also worth so much money, and the game allowing items to have a quantity over 99.
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
With regards to having more content: You can switch memory banks creatively to get access to about 36kb of program space. HOWEVER, you can only write 60 bytes per second, because that is the maximum input rate of the buttons (it actually isn't, but you'd need a different emulator than vba-rerecording to go any higher.) You can write a decompressor first but that takes time too. The 4kb payload program in my run takes about a minute to write. Writing 60 bytes a second is just about fast enough to play music on the fly using all the sound channels, but that's about it. When writing your payload program, you are not allowed to use any interrupts or the fixed jump instructions, since the interrupt handling code and fixed jump vectors are inside pokemon yellow's ROM. Without interrupts it's harder to play music and get input. SO, the payload program is limited to 36kb of space (plus whatever you can recycle from pokemon's ROM like sprites and such), can't use interrupts, and every byte costs 1/60 of a second to write to RAM. I think someone could make something very impressive indeed with enough knowledge of gameboy programming, especially if they reused some of the data in pokemon yellow's ROM and a decompressor. (Maybe something like the ending of earthbound, where there is a "roll call" with all the sprites.) I welcome anyone who wants to brave the gameboy's machine code to take what I've done as a starting point and make something even cooler --- it's all open source and available at http://hg.bortreb.com/vba-clojure . I was more interested in the attack side of this exploit so I just made something basic. (Though even that little program at the end took quite a while for me to write in machine language.)
1 2 3
7 8