This post attempts to explain how NesVideoAgent's automatic screenshot feature works. It should answer all questions, though it is a little technically oriented. <h3><a name="Architecture"></a>Architecture</h3><div class="deeper"> <h4><a name="OuterLayerAPhpProgram"></a>Outer layer: A php program</h4><div class="deeper"> It is run manually by a site admin, though it could be run automatically too. This commandline program searches the database for submissions that match these rules: <ul><li> Status is not published, rejected or canceled </li><li> Has no screenshots yet </li><li> Is for NES, SNES, GB, SGB, GBC, GBA or Genesis </li></ul><p/>For each matching submission, it does the following: <ul><li> Tries to determine which ROM should be used to watch the submission <ul><li> Scans the list of Good* ROM names and selects ~90 ones that most closely match the submitted ROM name or game's name </li><li> If it finds one whose checksum (CRC32, MD5 or other) matches what the submitted movie file has, uses it </li></ul></li><li> Runs the emulator with specific parameters <ul><li> A requirement is that the emulator can run on parameters without need for keyboard or mouse input </li><li> The emulator must also accept the fact that it can't produce sound (server has no soundcard) </li><li> The emulator is run under Xvfb so it can be graphical yet non-interactive </li></ul></li><li> Updates the submission message with <ul><li> links to images produced by the emulator </li><li> notes about the ROM selection </li></ul></li></ul></div><h4><a name="InnerLayerModifiedEmulator"></a>Inner layer: Modified emulator</h4><div class="deeper"> The emulator is slightly patched version of the normal emulator used to run the movies. <ul><li> It is completely uninteractive (movie file, sync options, play length specified on command line) </li><li> It includes a beauty analysis module. </li></ul></div></div><h3><a name="BeautyAnalysisModule"></a>The beauty analysis module</h3><div class="deeper"> The beauty analysis module is a component in the emulator that works like this: <p/><ul><li> At every frame when the movie plays, it observes the emulated screen (in a similar manner as AVI encoder does). </li><li> At every frame, it can also save or load a savestate, or terminate the emulator. </li></ul><h4><a name="DescriptionOfOperation"></a>Description of operation</h4><div class="deeper"> First, it plays the movie normally. <ul><li> Every frame, it analyses the image for beauty, and records it as a numeric value for that particular frame. </li><li> Every 100 frames, it makes a savestate. </li></ul>When the movie ends (actually, 2 frames before but that's a detail): <ul><li> It divides the movie into N evenly long sections <ul><li> Where N is selected so that N <= 50 but that a section length must be at least 1 second and not more than 10 seconds </li><li> And from that section, it determines the most beautiful shot, using the numeric values recorded earlier. </li><li> Then it picks the 20 most beatiful sections, and remembers the frame numbers where the single most beatiful screen occured during that section. </li></ul></li><li> For each selected beauty: <ul><li> It loads the latest savestate that was recorded before that particular frame </li><li> Runs the emulator until the framenumber matches the desired frame </li><li> And saves the screen as a PNG file. </li></ul></li><li> Then it terminates the emulator. </li></ul></div><h4><a name="AnalysisOfBeauty"></a>Analysis of beauty</h4><div class="deeper"> The algorithm used by the emulator for determining the beauty of a particular screenshot is written hoping to meet the following guidelines. A good screenshot contains: <ul><li> distinctly different colours at different sections of the screen (saturated colors preferred over grayish ones) </li><li> a lot of movement at each portion of the screen (less desirable to have it localized in a single corner) </li></ul><p/>The algorithm works by dividing the screen into 12 sections (4 horizontally, 3 vertically), and for each section, determining: <ul><li> The average tint on that section of the screen </li><li> The number of sprites that have changed position on that section of screen compared to the previous frame </li></ul><p/>The score is determined by the sum of sprite jitter in all sections, and matching each diagonal pair of sections and calculating how much the tint differs in them. <p/>The tint is calculated by sampling the pixels in that section (actually, about a third of them in a random gridlike pattern for speed but that's again an implementation detail), converting them to the YIQ colorspace and calculating average of the complex number comprised by the I and Q components, weighted using the Y value, and the final number weighted using a logarithmic transformation. </div></div>
