R.C. Pro-Am II is an extremely fun 4-player racing game. Besides its fast-paced racing action, it contains upgrade mechanics which allows players to improve their remote-controlled cars throughout the game's 36 race circuit.
This movie was produced in two parts: (1) the first 24 races and the entire upgrade optimization were solved by FatRightKnight back in 2018 and (2) the rest of the last 12 races were solved now by eien86 using a routing bot and the help of ktwo
Objectives
Aims for shortest movie
Heavy luck manipulation
Story
Quoting [eien86]
The reason I started working on this movie was pure serendipity. After finishing my Castlevania movie, Alyosha contacted me to see if I could improve the boss fight of Ironsword 2 (spoiler alert: I could).
At this point, ktwo noticed that I was open to challenges, so he brought to my attention the NES Solar Jetman TAS. This game contains a warp that allows you to skip a few stages. The thing is that, although the warp can be manipulated to skip even more stages, nobody was able to enable this efficiently. So ktwo asked me to run the bot to see if I could find an efficient solution (spoiler alert, I did -- movie upcoming).
While we were working on Solar Jetman, ktwo insisted that I should check this game, as FatRightKnight had gone almost all the way but did not finish it. At first, I was reluctant because I thought the bot would fare badly in racing games. However, in the end he persuaded me and I started experimenting with it. After successfully running race 1 faster than the current TAS, I was convinced that this was a feasible project.
I worked for a few busy weeks on this project and I am content with the result. Not delighted, only content. There are many ways in which this movie can be improved, and I most surely pursue them in the future. However, I am happy that I could bring FRK's work to light, as it shows how much he put into this game, and it would have been a disservice to him if I had replaced the entire movie at once.
Boot. All the frames from power-up until the start of the first race.
Race. A race starts as soon as the Traffic Light Timer's value becomes 64, indicating a green light. A race ends when the Race Status flag changes from 2 to 1.
Bonus Challenges. After races 8, 16, 24 and 30 there is a bonus challenge where the player needs to tap L/R repeatedly. The same rules of timing as a normal race apply here.
Transition. A transition starts as soon as a race ends. This segment includes the results screen and all the shopping made in between races.
Movie End. This movie's end time is taken at the frame of the last input.
Game End. The actual moment when the last race ends. An alternative movie is provided that optimizes this goal instead.
Race 1: Right at the very start, I hop on that hill at only 102 speed, when eight more frames of acceleration gets me 104. Of course, eight frames means I'm not past that hill, and it would take 200 frames to make up for losing about 800 distance units with an extra 4 speed. Actually, it's a bit more optimistic than that, but I don't think the hop takes more than a second or two.
Race 2: This is where I get a bunch o' letters. Nothing of note, aside from tricky manipulation that would inconvenience anyone trying to improve race 1. Silver Motor has a tight need, so any awkward collections done here were necessary.
Race 4: Just a reminder to anyone improving earlier races, this is another tricky letter manipulation spot.
Race 5: I'm focusing on that cruddy jump spot before that turn for this note. First lap, I get the nitro. Second lap, I use zipper twice. First lap, I pretty cleanly take that hairpin, but that's because I have less zipper speed, and dropped the motor down to 112 speed on top of that. Second lap, I basically thought 'forget it' and took the outside wall without bothering to try slowing down after the second zipper, or skipping said zipper a second time. It's an inconveniently placed boost, that's for sure. I actually do have time to stack a nitro on top of that second lap jump (well, I do need to waste a frame or two...), so taking the speed to a further extreme is possible. Not sure if that would be a good answer for tightening up our money route, but it's possible. This one jump, I tell you...
Race 6: Our last tricky letter manipulation zone. Manipulate the two "random" letters to be I, and all others are different. Fail, and you delay car upgrade by two races minimum. This reminder is my only note for this race, for now.
And then, from ktwo:
Quoting [ktwo]
Race 7: The one missing letter before the car type can be upgraded is found in this race. It's a 50% chance of being the missing 'P', so the manipulation is straight-forward.
Races 25-36 (by eien86):
Race 25: It's immediately apparent that a different logic is applied in this race than in the races TASed by FatRatKnight. There are more "twitching" motions present, even on straights. It's unknown why the bot chose these inputs. Lag reduction strats? Too big exploration space and not enough simulation time to grind out minor imperfections? Or something else? This should be investigated in the next TAS iteration of this game.
There is possibly also a different approach in how to negotiate corners in the bot's solutions. There are more examples of driving straight into the corner and bump out, as opposed to FatRatKnight's general method of navigating smoothly around most corners. Again, the finer details have not been investigated.
Race 26: There are some odd-looking crashes into the wall at the start of the lap 2. This was investigated after the fact and found to be nonoptimal. Driving in a straight was faster. It's possible was still not fully calibrated for this race and the simulation time was possibly also too short.
Race 27: While race 25 was on the same track as race 1 and race 26 was on the same track as race 10, they were played with different cars and car upgrades. Direct comparisons between the bot's and FatRatKnight's solutions are therefore not possible.
Race 27 is significant because it's played on the same track as race 19. Unlike the previous two races, race 27 and 19 were played with the same cars and upgrades. In addition to that, there is no lag in either race 27 or 19 or nitro route decisions to make, so a comparison between the bot's and FatRatKnight's solutions is fair and measures the difference in execution.
The result?
FatRatKnight (R19):
lap 1 - 558f
lap 2 - 506f
lap 3 - 505f
Total - 1569f
The bot (R27):
lap 1 - 555f
lap 2 - 498f
lap 3 - 498f
Total - 1551f
Races 28 and 29: These races can't be compared with earlier races due to car differences. The gameplay bears the signs of typical bot inputs. it looks fast and efficient, but there is again a question mark why the bot finds solutions that don't go in a straight line on straight sections.
Race 30: This race can be directly compared with race 20 (same cars). And it's an interesting comparison, because it highlights other gameplay aspects than race 27/19. There is lag in this race and the routing is complex. There are considerations such as "should you go at high speed and take a wide turn or slow down and take a tighter turn?" and several nitro uses to portion out appropriately.
The result?
FatRatKnight (R20):
lap 1 - 908f
lap 2 - 835f
Total - 1743f
The bot (R30):
lap 1 - 842f
lap 2 - 837f
Total - 1679f
The bot took a big lead on the first lap by both reducing random lag, gameplay and by using more nitros. But it lost time on the second lap, mainly due to having fewer nitros to burn.
Race 31: Not possible to compare this race. Same comments as previously.
Race 32: Not possible to compare this race. The nitro use looks a bit suspicious and there is a probably a better place to use it. Question mark also on shortly letting the blue car pass on the first lap. It looks like there could have been a solution, where the player car continued to be pushed by the blue car.
Race 33: This race can be compared with R21, which is 15f faster. The issue here is that nitros aren't used in the right places. Like it's done in race 21, one should have been used in the second big bumpy section and another one at the jump at the top, instead of using them early on.
Race 34: Not possible to compare this race. It would likely have been a better decision to collect the nitro in the upper right corner (and using is cleanly on straight to the finish line) than using both of the zippers. The next bot iteration should test that alternative.
Race 35: Not possible to compare this race. It might be possible to pick up one more nitro on the right side of the track. Should be tested in the next iteration. It's worth noting how the bot collected the 1-up at the top. This is a subtle lag reduction strat and it's impressive it managed to find it.
Race 36: This race can to some extent be compared with R22, but the comparison is not completely fair. There is a lot of random lag from having too many objects on screen in these races, but it's worse in R36 because of the CPUs having faster cars and staying more on screen than in R22. As a result, R22 is 35f faster, but has 38f less of random lag.
Technical Discussion
An in-depth explanation of the game mechanics by ktwo can be found here.
Upgrade strategy
Quoting [ktwo and FatRatKnight]
There are three upgrade paths in this game:
Car type
A new car is awarded between races when the letters P, R, O, A, M and I (twice) have been collected. The car type can be upgraded twice. Each upgraded car type has much improved specs and it's a top priority to get these as early as possible.
The letter boxes are always in the same locations, but the content is determined by RNG (see https://kb.speeddemosarchive.com/R.C._Pro-Am_II/Game_Mechanics#Letters for a detailed description). The first four races contain 8 letters. R5 doesn't contain any letters, but R6 and R7 have 7 letters in total. A TAS should therefore aim to have the best car type at the start of R8.
Manipulating letters in real-time is out of the question (except in race 1), but is possible in TAS conditions. However, to make a long story short, you're in most races restricted to a range of 256 possible outcomes and they are reached by waiting the corresponding number of frames. Again, the interested reader is referred to the links provided for more background. That is more than enough when a race only has a few letters to manipulate. But in e.g. race 6, where 6 letters need to be manipulated, and each letter can have several outcomes, the odds of finding a solution suddenly plummets.
One minor mitigation that FatRatKnight used to slightly improve the chances of finding suitable letters was by playing as the yellow car (player 4). Playing as P4 was not strictly needed, but was time-neutral. For a better understanding of the thought-process behind this decision, a dive into the game mechanics and how the letter-generation process works is first needed. FatRatKnight's own explanation can then be understood, https://tasvideos.org/Forum/Posts/516896
Motors
Better motors can be bought in the shop between each race. The motors improve both the car's top speed and the acceleration. The latter has not been investigated in detail, but the top speed stat is the most important of the two and is described here: https://kb.speeddemosarchive.com/R.C._Pro-Am_II/Game_Mechanics#Max_speeds
The motor purchases depend on available money in the races, the motor costs, if a purchase in one race delays an upgrade in a later race and of course the motor parameters. It's not trivial to provide a solid reasoning for one upgrade strat over another one. The motor purchase strat used in this TAS grew out of theoretical forum discussions, but eventually had to be decided from TASing the most promising options and see which one turned out to be faster. The interested reader is referred to the game's forum topic to see how the purchase strat evolved and how it ended with the one shown in this TAS (Red after R1, Silver after R3 and Mega after R15).
Tires
Better tires can be bought in the shop between each race. Tires influence how fast the gap between car facing and movement direction (referred to as "momentum" in the forum by FatRatKnight) is bridged. The better the tires, the faster the gap closes. So essentially, better tires give faster direction changes instead of sliding around. This might be best described by FatRatKnight's own words: https://tasvideos.org/Forum/Posts/468257 .
While excessive sliding impacts the speed negatively and tighter cornering is beneficial in many places, the tires generally have a smaller impact on the time than motors. The purchase strategy for the tires underwent a similar development as that for the motors, finally resulting in buying Skinny after R4 and Scoopers after R18.
Botting
Botting this game required determining the RAM map of all the relevant values and obtaining a more or less in-depth understanding of the game mechanics. Thankfully, ktwo helped with both aspects.
The reward function for the bot was pretty easy to establish, since the game already provides a faithful progress indicator. The only addition to it was the reward for grabbing boosts -- the bot would ignore a boost if it meant deviating from the immediate fastest route, so it required an extra incentive to go grab them.
Another scripting requirement was the need to restrict places where the boosts could be used. Particularly in race 36, the player obtains 6 boosts. The optimal solution required using these in very precise locations. Using them in other parts of the track is also possible, but leads to a slower solution. Since the bot doesn't know this, it proceeds to burn them as soon as possible, so I had to explicitly limit this freedom.
Running the bot on this game provided a few additional challenges.
First, the movie by FatRatKnight used the FourScore adapter to enable him to play with P4. This allowed him to manipulate RNG, but complicated my life because QuickNES does not support it. In order to make it work, I had to modify the source of QuickNES to encode the FourScore inputs from the standard input produced by directly connected controllers.
Second, since FRK had worked on FCEUX, I had to resync his movie to BizHawk (NesHawk core). This still was not enough, since the bot only accepts QuickNES save states as a starting point. So I had to develop a 'transplant' tool, that copies all the relevant RAM/PPU memory regions for a Mapper07 game from a NesHawk savestate to a QuickNES savestate. Miraculously, this worked (except that I didn't care to copy the palette, so all colors were wacky during botting).
Third, I had to resync the bot's quickNES output back into the NesHawk movie. This was pretty easy because these emulators only seldom differ. The only case where they desync was when NesHawk interpreted some frames as lag, which QuickNES didn't.
Future Work
This movie is by no means perfect. Here are all the ways in which it could be improved in the future:
Refine Bot's exploration
Some aspects of the solutions provided by the bot gave us suspicion that were not really optimal. In particular, the bot sometimes hits itself against the wall, even when not evidently necessary. ktwo noticed this and could find some local refinements of the route by manual optimization. These inefficiencies must be polished in the next version of this movie.
Remove unnecessary inputs
The bot repeatedly presses L and R in parts of the track where only going straight is necessary. This is not necessarily an inefficiency, but could be simplified in the future. I believe this happened because, to reduce exploration space, I didn't take the decimal components of the position/speed into consideration, so without these presses, the bot could have interpreted that straight lines contained repeated states.
Apply the bot to the first 24 movies
The next obvious improvement is to apply the bot to all races, not just the last 12, as The bot showed it could find faster solutions than FRK's manual optimization for comparable races. This is not as simple, however, since in the first part of the game, the entire upgrade logic needs to be taken into account, which is not as simple, and was perhaps the biggest contribution of FRK's research.
Acknowledgements and Attributions
FatRatKnight: Author of the first 24 races and prime mover of this game's TAS effort
eien86: TASer, botter and author of the last 12 races
ktwo: Speedrunner, TASer, researcher and the reason why I ended up involved in this game. His constant review of my work, suggestion, and technical advice were instrumental to getting this TAS done. He has contributed to the technical discussion part of these submission notes.
PinoBatch and Fiskbit: NesDev Discord Server users, who helped me understand how the FourScore adapter worked.
Encoder Information
Suggested Thumbnails
Re-Record Count
The count from FatRatKnight's original 24 races should be used: 14962, plus the ones in this movie (653). The re-records from botting is not counted (otherwise, would contribute hundreds of billions)
Your bot never ceases to amaze me. I feel like it's getting to a point where it can figure out optimality in almost anything. I wish I understood how things worked haha. Yes vote for sure!
More impressive bot work, nice! The bot driven races do look a bit rough though, maybe the fact that you can just bounce off the cpu's to keep pace is effecting the search (ex 25:57 in the encode.) Still cool to see another type of game tackled by your bot. Maybe try puzzle games? I wonder what it could find on ex. Adventures of Lolo games.
As mentioned in the comments, I took part in the making of this TAS. Initially, I thought it would be a directly contributing part by working in tandem with eien86's bot Jaffar in races 25-36. In the end, all the inputs in races 1-24 are from FatRatKnight and all the inputs in 25-36 are from the bot. I'm therefore not listed as a co-author.
I speedran this game on console in 2018 and during that time had an active discussion in the forum thread with FatRatKnight. Most of what we know today of this game's mechanics is a direct result from that discussion (by far the most of the advancements were done by FatRatKnight). So while I focused on the real-time speedrun, FatRatKnight eventually TASed up to race 24. This was done through several iterations to hone the execution, the strats, test different upgrade options and use the latest findings.
I thought it was a pity that the last 12 races were never completed and put it on my own to-do list. The remaining races should normally be the easiest ones to complete, because the car was already fully upgraded at that point and races 25-36 re-use tracks from previous races.
However, before starting on race 25, I planned to TAS one or a few of the earlier races. This would allow me to compare with FatRatKnight's times to ensure I had the right understanding of the methods used to efficiently navigate corners etc. I started this a few times, but I had trouble matching FatRatKnight's solutions and didn't get deep enough into the logic behind how to take corners. If I couldn't match (to be more fair - not even come close to) the execution in the first races, it didn't feel right to continue from race 25.
In conclusion, the above was a somewhat convoluted way of saying that the route and execution in the first 24 races seem solid (and even very impressive) to me. While it has been some time since I was into the details of this game, I can't really think of anything that looked questionable or out of place when watching it played back in real-time.
When I started working with eien86 on improving Solar Jetman, I also mentioned R.C. Pro-Am II as a game that would be interesting to test the bot on. It took some convincing before eien86 decided to try it out though and I was of the understanding that this was new territory for the bot, so it was difficult to predict the results.
While I was busy manually TASing Solar Jetman, eien86 fed race after race to the bot. Before I took a break to look at the bot's solutions, several races were already done. My immediate reaction was that it looked impressively fast, but also a question mark regarding the constant twitched steering, even on straights. I was even able to demonstrate short sections where a few frames could be saved simply by driving in a straighter line. However, due to desynching issues, eien86 was hesitant to redo earlier races.
The question was then how the bot's execution and solutions compared to FatRatKnight's? Well, since a few of the races in the range 25-36 are directly comparable with earlier races, it was just a matter of comparing the completion times. In all four races that were possible to compare, the bot came out on top in terms of execution, was unparallelled to sniff out ways to reduce random lag and on shorter segments managed to pick up some non-obvious strats. However, it struggled when it came to nitro uses. Nitros are collected on the track and can be used at any time in the same race. It's understandable that a human has an advantage in this area over a bot that can't get the same overview of a track. The biggest mistakes were in R33, where the bot was too eager to use the nitros for short-sighted benefits, but was then penalized later on. It should be a lessons learned for the next iteration that the bot's nitro uses need to be more tightly supervised.
I'm very happy to see FatRatKnight's effort be part of a submission for this game. It's also exciting to see eien86's bot adapted to racing games. The bot solutions themselves are in my opinion clearly improvable, but they are at the same time clearly one notch above a very accomplished TASer's concerted efforts. So while a human would be able to provide a smoother a visually more pleasing playthrough of races 25-36, it would take a lot of effort to match the bot's completion times.
My recommendation is to accept this submission, but also that we keep reminding eien86 at regular intervals about his promise to give this game an updated TAS in the future. When work on that update begins, I'd be happy to contribute in whatever way I can to make the end result as good as possible.
Reading the description on the player choice and letter-based upgrades, while the CPU cheating in letters is true, that wasn't my primary reason for selecting P4.
The actual reason is that the letters ask for particular players. The letters on the track can ask for a particular slot, and each slot generates a random letter based on what some player doesn't have.
The slots are as follows:
(1) Based on Player 1
(2) Based on Player 2
(3) Based on Player 3
(4) Based on Player 4
(R) We read the race number, then select a player based on that
(?) We ask the RNG for a player number
Then it's a matter of figuring out what slots each race uses.
Race 1: (1)
Race 2: (?) (2) (R) (3) ... The Race letter picks P2
Race 4: (R) (4) (1) ... The Race letter picks P4
There are eight possible letters out of our PRO-AM II to get in the first four races. When I selected P4, I just wanted to abuse the letter selection at race 4 to guarantee the two generated letters are ones we didn't pick up, as the P4 slot and the Race slot both ask P4 for the letter. Which basically meant the only manipulation necessary at that specific race is to ensure I don't double up on the same letter, apart from I if my route didn't get any of that.
Race 6: (1) (2) (3) (R) (?) (?) ... The Race letter picks P2
Race 7: (4)
In the case of Race 6, while two letters pick the Random slot, they can't be independent of each other, as they're on the same slot. Which means those two letter boxes are tied together no matter what the RNG said. With only 7 letter pick-ups in these two races, the only way for the upgrade to happen at the end of 7 is for the Random slot to pick I, as that's the one letter you can double up on. On the randomness scale, I is bottom priority for letter generation RNG, so that was a bit of "fun" manipulating that, along with every other letter, when I was fresh from the first upgrade and the other CPUs didn't have a lot of cheated letters.
I will point out that, by picking P4, Race 6 could not generate the letter P. The Random boxes are already locked into the need for the I, and the other boxes all depend on the CPU players, who each cheated their own P by now. That left RO-AM II as whatever is left for me to pick up. Once at Race 7, as that letter was asking for P4, it was just a trivial 50% chance of generating the one letter I was missing, another advantage for picking P4. The other letter it could have generated is the I, which is the fallback case for when it couldn't generate any other letter.
So, for the most part, P4 was selected not on the CPU cheating their own letters (with P1 cheating at races 1 and 5), but on the maze of letter pick-ups the developers decided on.
Oh, and fix the link to the forum post under your tires explanation. You included the period in your tag.
As for other thoughts that aren't on the technical reasons why I chose P4 on RNG...
Well, I can't think of a lot to say already. I'm actually not all that good on encouraging descriptions anyway, I claim. It is interesting to note that the bot can beat me on the comparable races, so it's clearly doing something right. I'm curious if it can optimize my early races a little better, with particular constraints of what it must pick up. The bot appears to be pretty robust to have beaten me. ... Sorry, there really isn't a lot I can think of to say. Um, have a thumbs up in writing from me? Again, I claim not to be a particularly wonderful speaker of encouraging things, but just throw me at the technical details or explanations and you'll see what I write.
Thanks for the clarifications and corrections. The comments have been updated.
eien86 is probably better to explain the bot's features, but picking up loot along the way will add complexity for sure. But if I understand it correctly, it's entirely possible to tell the bot what to collect.
When I tried TASing it myself, I first made my own solution and then looked at yours. I couldn't really make out what it was in your method that made it faster though. Same with the bot's solutions compared with yours. It looks to me like it's more aggressive, bumping into the walls, while you're more smoothly drifting around the corners. I didn't deep-dive in a frame-by-frame analysis though, so I don't know if it's the bumping into walls or something else in the bot's solutions that make them fast. Your solutions are visually much more appealing and intuitively look faster (at least in my eyes). But the actual numbers have clearly been in favor of the bot in every comparison we made, except for when the nitro use became too complex. I know that eien86 also botted race 1 as a test before starting the actual TASing and the bot came out comfortably on top there as well.
