The Arduino is wired to a simple circuit with a button to begin playback. I copied the circuit from the Arduino KeyboardMessage tutorial. I paid about USD 40 all together for the Arduino, breadboard, pushbutton, jumper wires, etc.
I made some changes to the original JRSR file, for convenience or to get the run to sync. You can see the changes by diffing tas/comic1.jrsr (original) and tas/arduino-verification/comic1.jrsr (modified) in the Git repo. Apart from the first two changes in the list, the modified file still syncs in JPC-RR.
I was not interested in playing back the boot sequence, so I trimmed the inputs up to the title screen. You have to boot the computer and run COMIC.EXE yourself, then press the button to begin playback.
I shortened the delay between the title screen and the first game inputs by about 165 ms (equivalent to 1.5 game ticks). This is for sync; without it, Comic always made the first jump too late. I attribute this to the real laptop running the game's initialization code faster than the emulated CPU does.
After the first death in CASTLE0, I moved the release of the Right key to happen about 1 s earlier. I don't know why this was necessary, but without it, the game would not register this particular key release and would desync.
The original JRSR file continues holding the Left and Space keys at the end of input. On a real PC, this led to automatic keyboard repeat and annoying beeping at the high score screen. I added release events for these keys at the end of the input.
Even with the above changes to the JRSR file, the playback does not sync every time. At the archive page, you can see all 11 attempts I made, of which 3 synced to the end. The embedded video above is attempt #7.
I attribute the sync failures to a lack of synchronization between the externally timed keyboard inputs and the game loop. In the emulator, inputs occur at certain times relative to the game's tick cycle. Because the USB keyboard inputs are not locked to the game loop, they may occur earlier or later in each tick, perhaps sometimes even splitting inputs that are meant to be in the same tick across adjacent ticks. Captain Comic is a forgiving game in that it has ticks of long duration (110 ms). The same technique may not work directly with other games that require more precise timing—though an idea for the future is to connect the VSYNC pin of the laptop's VGA port to the Arduino, as a timing source for games whose game loop is locked to the screen refresh.
It seems to me that good candidate games would not be too long, not do much loading from disk, have a low frame rate, not use subframe inputs, and not use the real-time clock (this last criterion eliminates the Commander Keen games).
From a skim of the DOS movies list, some good first runs to try could be
I don't know whether fast-paced adventure games like [4391] DOS Hero's Quest by c-square, mrprmiller & davidtki in 00:40.90 would be good or not. Because something is happening on nearly every frame, even a small inaccuracy could desync the run. On the other hand, they are short in real time, and may buffer keyboard inputs, providing some resilience against timing inaccuracies.
Here, you can see some failed attempts at syncing Dangerous Dave. Seek to 8:10 for the attempt that gets the farthest, into level 3. The farthest I think I've ever seen it get is the end of level 3, stopping just short of the door.
Link to video
There are two things I want to talk about:
It's not immediately apparent in the video, but the game is running slower than it should. This is because the game loop is locked to the screen refresh, and this computer's video card outputs a 60 Hz signal despite the game using mode 13h, which is supposed to be 320x200@70Hz.
The refresh rate discrepancy should be okay, because the Arduino playback device reads the VGA VSYNC signal and adjusts its own timing to the vertical refresh, rather than using absolute timestamps. But it still doesn't work.
60 Hz video output
My preliminary attempts were very far from syncing, more than can be explained by the gain or loss of a few milliseconds. The cause was unexpected to me: the video signal output by the PC's VGA port had a refresh rate of 60 Hz, not 70 Hz as it should have had. I checked this by enabling an option on my monitor to show the current video mode. You can also verify it by counting cycles of the trophy animation in level 1 of Dangerous Dave. On hardware the animation runs slower than in emulation with DOSBox.
Every computer and graphics card I had at hand had the same problem, outputting a 60 Hz signal for graphics modes that are nominally 70 Hz:
Many LCD monitors from the recommended time period already have digital DVI inputs that can provide better image quality than the analog VGA input. Still, using DVI is not recommended for DOS games because refresh rate issues may appear where VGA modes run at 60 Hz refresh rate instead of 70 Hz standard. A typical indication are speed and/or sound issues in games and demos that use the refresh rate for time synchronization.
Support for Mode 13h, and especially for undocumented VGA tweaks like those used by Doom, has dropped to near non-existence in modern video hardware and operating systems.
The problem arises under modern machines that do not actually support mode 13h - this support is becoming exceedingly rare, especially on newer ATi video cards, some of which will not even set into ANY screen mode lower than 640x480.
TLDR: I patched the VGA BIOS ROM of my 9600 Pro to output 1280x1024@70.08 Hz instead of the VESA default of 75 Hz.
I would have assumed that playing games on hardware under FreeDOS was a more authentic experience than playing in emulation, but that is not necessarily so. Whatever Crystal Caves does with the video mode fails in an especially noticeable way: it's like the top row of pixels gets stretched over the whole screen. I suppose the only remedy is to track down a vintage video card that implements the right timings. I'm curious what all-in-one games-oriented hardware packages like the weeCee (LGR video) do with video timing.
VGA VSYNC to the Arduino
I altered the Arduino circuit and program to be able to read VGA VSYNC signals. I connect a Y splitter to the PC's VGA port. One end of the splitter goes to the monitor and pin 14 (the VSYNC pin) of the other end goes to one of the Arduino's digital inputs.
The JRSR format does not store video refreshes, only nanosecond timestamps. But you can often infer where the video refreshes are, because of how authors tend to make TASes: hit frame advance, enter inputs for the frame, then hit frame advance again. Most input events occur at times that are multiples of the video refresh interval, plus a constant. The constant depends, usually, on non-frame-aligned activities at the start of the movie, the boot process and DOS prompt.
For example, here are the event timestamps from [1718] DOS Dangerous Dave by Ilari in 06:32.80, modulo the VGA refresh interval of 14.268 ms (= 800 × 449 / 25.175 MHz). Notice how most timestamps are aligned at an offset of about 9.507 ms. There is a secondary line of timestamps about 1.3 ms after the primary one—this happend when there are two events in the same frame.
Similarly, there are strong primary, secondary, and tertiary timing alignments in the events of [3706] DOS Crystal Caves: Volume 1 - Troubles with Twibbles by DungeonFacts in 20:33.15. But there is also a substantial minority of events that are not so well aligned. I looked into this case—the unaligned events are where 200 ms timed advance was used, rather than frame advance.
In contrast, the timestamps of [2119] DOS Avoid the Noid by turska & Ilari in 03:33.52 are mostly not aligned to the VGA refresh. This suggests that this TAS was not made using frame advance. (Another reason for a lack of alignment could be an incorrect refresh rate, but I'm pretty sure this game is 70 Hz like the rest.)
I wrote a program, guess-vsync, that tries to infer VSYNC parameters, given the event timestamps in a JRSR file. With these VSYNC parameters, the program gen-table produce a table of input events; not only RELEASE, PRESS, and WAIT events, but also VSYNC which means to wait until the next VSYNC pulse from the VGA port. Having a reference for the PC's timing, the Arduino should (in principle) be able to cope with timing variations, like the 60 Hz for 70 Hz I talked about above. The source code of these programs is available in 20220124-dave-pc-tas-verification-attempts.zip, or from git clone https://www.bamsoftware.com/git/pc-tas-arduino.git.
Here's what I did to program the Arduino for the video above. First, run gen-table, just to get a timestamp for when the boot-related events are over and we're ready to start the program. We will pass this timestamp to later commands as the argument to the --skip option.
From this, we see that the timestamp of pressing Enter to start the program is 493728396ns. Now we run guess-vsync, telling it to ignore the events before that timestamp:
$ ./guess-vsync --skip 493728396ns ilari-ddave.jrsr
14268123.1380338ns+9507455.38628574ns
average delay between VSYNC and event: 0.386ms
The interval+offset 14268123.1380338ns+9507455.38628574ns is the guess for the VSYNC parameters. We'll tweak this guess, appending -10%, to shift the VSYNC phase earlier by 10% of a frame—this is because we don't want our inputs to occur at the exact same time as a vertical refresh. Now run gen-table again, this time passing the VSYNC parameters and the skip timestamp, and save the output to playback/table.h.orig:
We are not quite done yet. The timing of events is most uncertain at the beginning of the movie. I manually tuned the timing of the first few events and saved it as playback/table.h. Then I uploaded the program to the Arduino.
But it still doesn't work
As the video proves, all these considerations are still not enough for successful verification. Here are some of the things I've tried:
Scheduling inputs early in the frame (--vsync interval+offset-10%)
Scheduling inputs late in the frame (--vsync interval+offset+10%)
Removing WAIT events and using only VSYNC for timing
Different debounce parameters in the VSYNC interrupt handler
I've noticed the first run after a reboot is likely to desync earlier. My guess as to the cause is that the program is slower to start up when the disk cache is cold. Timings are affected by disk delays more than I would have thought. The disk image of #2961: Ilari's DOS Dangerous Dave in 06:32.80 contains only a single file, DAVE.EXE. Once, I mistakenly ran the input file in the emulator with a disk image containing three files: DAVE.EXE, DSCORES.DAV, and EGADAVE.DAV. The presence of those two extra files was enough to desync emulator playback in level 2, despite the fact that the main program does not even open those other files by that point (I think).
I started the movie with pressing Enter at the DOS prompt. This is problematic because it means there is a video mode change in the middle of the event stream. I had initially tried to start playback at the title screen, or at the beginning of the first level, but that was worse. There's a sort of "frame rule" at the beginning of a level, while Dave is blinking. He must finish a blinking cycle before he can begin moving, which means that if you start the playback at a random point in the blinking cycle, the inputs that follow will be off by a random amount.
I am pretty naive about electrical engineering. To give you an idea: I didn't know what a pull-down resistor was until a few months ago when I had to wire up a pushbutton for the first iteration of the playback device. It's possible that the circuit I'm using for reading VSYNC signals doesn't make sense, or is susceptible to noise that causes timing errors. I'm not sure it's right that the VSYNC GND pin is not used, nor that the VGA input and the rest of the circuit do not have the same ground reference potential. Here is the circuit:
arduino-verification-circuit-20220124.cddx from https://www.circuit-diagram.org/
I'm using the attachInterrupt interface to react to changes on the VSYNC input pin; i.e., attachInterrupt(digitalPinToInterrupt(VSYNC_PIN), vsync_isr, FALLING). I've read that attachinterrupt goes through a lot of abstractions that add overhead, and that for high performance you need to program interrupts at a lower level. But the overhead estimates I've seen are in the realm of single-digit microseconds, which seems like it shouldn't matter much for VGA vertical refresh, which is in the realm of milliseconds. I think that the USB HID key events can be no more precise than 1 ms, anyway.
Great write up!
I'm not really surprised you are getting desyncs with all the in between steps you have to do. Do you think working with original hardware from the time period would give better results? Would it be harder or easier to work with?
Do you think working with original hardware from the time period would give better results? Would it be harder or easier to work with?
That's a thought. An old graphics card will likely be needed at some point, to get an actual 70 Hz signal for games of the era. An older, slower CPU may better line up intra-frame timings. A rotating hard disk, I would guess, would be more variable in timing than an SSD, but I've been trying to avoid games with loads from disk anyway.
A next step is probably to start eliminating variables. I'm thinking it would be good to program a custom "game" that is well-behaved: no randomness, no video mode changes, no filesystem I/O, just a very simple loop locked to vsync, with visual feedback. Like a single pixel that you control with the keyboard. The screen could draw a predetermined path, and the input sequence would try to make the pixel follow the path. That would make it easy to see, with 1-frame resolution, whether current setup is able to sync at all under ideal conditions, and if not, whether it fails to sync in a consistent way.
I wrote a simple game, TESTGAME.COM, designed to help diagnose sync problems. It draws a deterministic pseudorandom sequence in gray in the background. The player must move a pixel up and down using the keyboard to match the sequence, as the pixel advances to the right at one pixel per frame. Matches are drawn in blue and mismatches are shown in bright red. Syncing the game requires providing rapid frame-perfect inputs. Apart from the quick inputs, the game is well-suited to console verification: there's no randomness, no dependence on a realtime clock, and inputs are sampled and latched once per frame.
Here is me attempting to play the game by hand in DOSBox:
And here is a TAS playing it perfectly in JPC-RR:
After experimentation and tweaking, I have had some success in syncing TESTGAME.COM on hardware. (Despite the 60 Hz timing mismatch, which still exists—VSYNC-relative timestamps seem adequate to deal with that problem for now.) It doesn't always work, but its syncs 100% more than half the time. Here is an example of a successful verification:
Link to video
The tweaks that seem significant are:
Connecting the VSYNC GND pin of the VGA port to GND on the Arduino (though this still seems wrong to me).
Batching multiple keyboard state changes per frame into one USB report.
VGA VSYNC GND
Previously I had left pin 10 of the VGA port, VSYNC GND, unconnected. Now I have it connected to the Arduino GND:
arduino-verification-circuit-20220227.cddx from https://www.circuit-diagram.org/
I cannot shake the suspicion that this is still wrong, electrically. The +5V VSYNC pin and the VGA GND pin make their own circuit that passes through the Arduino—the Arduino's "ON" LED worryingly remains lit at half brightness even after the USB power cable is disconnected, if the VGA port is still connected. But it seems to make playback sync more reliably.
Batching USB reports
The way USB HID keyboards work (at least in the boot keyboard profile required by FreeDOS) is they do not separately communicate every key press/release transition. Rather, they say which keys are currently pressed (up to six at once) in a data structure called a "report". Previously, the playback code was sending one report for every press/release transition. Now, it attempts to consolidate multiple press/release events that happen in a short period of time, and send the result of all the events in one report.
So, for example, where the code formerly used PRESS and RELEASE events:
it now splits those out into ADD, REMOVE, and SEND events. ADD and REMOVE only update the pending USB report but do not send it; SEND actually sends the report:
I made this change because it seemed that the OS was sometimes dropping inputs that were closely spaced. The USB keyboard polling frequency is only 1000 Hz. 1 ms per key event starts to eat into the 14 ms frame interval pretty quick, and deviates significantly from JPC-RR's KEYEDGE event spacing of 0.67 ms for most keys. I suspect (though I haven't checked) that this keyboard report batching would have solved the sync problem in CASTLE0 in Captain Comic: it occurred at a time when multiple keys were pressed rapidly.
I initially believed that the GND circuit change was insufficient, and that the USB report batching was also necessary for sync. But my guess was not backed up by experiment: when I went back to test it, the movie synced with about the same success rate with and without the USB change. I still think the USB change is worth keeping, but it is apparently not as essential as I though. Here are 12 attempts without USB report batching (synced 7/12):
All videos and source
And here are 6 attempts with USB report batching (synced 4/6):
All videos and source
Note cases #1 and #6 in the latter set, where there is a long flat line in the final row. This indicates that the Arduino was providing inputs in the correct sequence—but too fast. It ran out of inputs too quickly, and that's why the final part of the trace is a straight line. This effect I attribute to something going wrong electrically, causing the VSYNC interrupt in the code to fire too quickly.
I made some changes to the original JRSR file, for convenience or to get the run to sync. You can see the changes by diffing tas/comic1.jrsr (original) and tas/arduino-verification/comic1.jrsr (modified) in the Git repo. Apart from the first two changes in the list, the modified file still syncs in JPC-RR.
The JRSR format does not store video refreshes, only nanosecond timestamps. But you can often infer where the video refreshes are, because of how authors tend to make TASes: hit frame advance, enter inputs for the frame, then hit frame advance again. Most input events occur at times that are multiples of the video refresh interval, plus a constant. The constant depends, usually, on non-frame-aligned activities at the start of the movie, the boot process and DOS prompt.
For example, here are the event timestamps from [1718] DOS Dangerous Dave by Ilari in 06:32.80, modulo the VGA refresh interval of 14.268 ms (= 800 × 449 / 25.175 MHz). Notice how most timestamps are aligned at an offset of about 9.507 ms. There is a secondary line of timestamps about 1.3 ms after the primary one—this happend when there are two events in the same frame.
Similarly, there are strong primary, secondary, and tertiary timing alignments in the events of [3706] DOS Crystal Caves: Volume 1 - Troubles with Twibbles by DungeonFacts in 20:33.15. But there is also a substantial minority of events that are not so well aligned. I looked into this case—the unaligned events are where 200 ms timed advance was used, rather than frame advance.
In contrast, the timestamps of [2119] DOS Avoid the Noid by turska & Ilari in 03:33.52 are mostly not aligned to the VGA refresh. This suggests that this TAS was not made using frame advance. (Another reason for a lack of alignment could be an incorrect refresh rate, but I'm pretty sure this game is 70 Hz like the rest.)
I wrote a program, guess-vsync, that tries to infer VSYNC parameters, given the event timestamps in a JRSR file. With these VSYNC parameters, the program gen-table produce a table of input events; not only RELEASE, PRESS, and WAIT events, but also VSYNC which means to wait until the next VSYNC pulse from the VGA port. Having a reference for the PC's timing, the Arduino should (in principle) be able to cope with timing variations, like the 60 Hz for 70 Hz I talked about above. The source code of these programs is available in 20220124-dave-pc-tas-verification-attempts.zip, or from git clone https://www.bamsoftware.com/git/pc-tas-arduino.git.
Here's what I did to program the Arduino for the video above. First, run gen-table, just to get a timestamp for when the boot-related events are over and we're ready to start the program. We will pass this timestamp to later commands as the argument to the --skip option.
arduino-verification-circuit-20220124.cddx from https://www.circuit-diagram.org/
I'm using the attachInterrupt interface to react to changes on the VSYNC input pin; i.e., attachInterrupt(digitalPinToInterrupt(VSYNC_PIN), vsync_isr, FALLING). I've read that attachinterrupt goes through a lot of abstractions that add overhead, and that for high performance you need to program interrupts at a lower level. But the overhead estimates I've seen are in the realm of single-digit microseconds, which seems like it shouldn't matter much for VGA vertical refresh, which is in the realm of milliseconds. I think that the USB HID key events can be no more precise than 1 ms, anyway.
And here is a TAS playing it perfectly in JPC-RR:
After experimentation and tweaking, I have had some success in syncing TESTGAME.COM on hardware. (Despite the 60 Hz timing mismatch, which still exists—VSYNC-relative timestamps seem adequate to deal with that problem for now.) It doesn't always work, but its syncs 100% more than half the time. Here is an example of a successful verification:
Link to video
The tweaks that seem significant are:
arduino-verification-circuit-20220227.cddx from https://www.circuit-diagram.org/
I cannot shake the suspicion that this is still wrong, electrically. The +5V VSYNC pin and the VGA GND pin make their own circuit that passes through the Arduino—the Arduino's "ON" LED worryingly remains lit at half brightness even after the USB power cable is disconnected, if the VGA port is still connected. But it seems to make playback sync more reliably.
Batching USB reports
The way USB HID keyboards work (at least in the boot keyboard profile required by FreeDOS) is they do not separately communicate every key press/release transition. Rather, they say which keys are currently pressed (up to six at once) in a data structure called a "report". Previously, the playback code was sending one report for every press/release transition. Now, it attempts to consolidate multiple press/release events that happen in a short period of time, and send the result of all the events in one report.
So, for example, where the code formerly used PRESS and RELEASE events:
All videos and source
And here are 6 attempts with USB report batching (synced 4/6):
All videos and source
Note cases #1 and #6 in the latter set, where there is a long flat line in the final row. This indicates that the Arduino was providing inputs in the correct sequence—but too fast. It ran out of inputs too quickly, and that's why the final part of the trace is a straight line. This effect I attribute to something going wrong electrically, causing the VSYNC interrupt in the code to fire too quickly.
