Posted: 4/11/2008 9:19 AM
(Edited: 5/30/2008 5:20 AM)
Post subject: Multimedia TASing for rerecording-deprived emulators
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
As I have not got access to pSX source code, I have done some plans using my favourite Unix technique, the LD_PRELOAD trick. Two of such outcomes of my project are: TASVideos quality AVI recording This is my classic "injected encoder" trick. Example: http://www.youtube.com/watch?v=sPvrgO_UN80 It requires that the emulator is configured to run in "no frame skipping" mode and "synchronous audio" mode. Multimedia TASing Multimedia TASing is the action of recording your entire playing session on the emulator, quicksaves and quickloads and all, and then postprocessing the video&audio recording such that all your undoes are removed from the recording. With my technique, such postprocessing can be automated. Such method is not allowed on this site, because videos produced by it are unverifiable: It is impossible to verify whether what happens on the video is the exact product of running the game, or whether the game is hacked, video is edited, or such. However, the emulator lacking real TASing support, it is a neat feature to produce TAS-like videos. Here's how it works. Every time the emulator saves or loads a quicksave, the audio&video recorder takes note of the current recording position, and logs those into an event log file. Afterwards, by inspecting the log file, one can stitch together the video&audio such that all of the undone actions are removed, and only the final product is shown, just like in a TAS. Of course, you're still limited to the emulator's available toolset. In the case of pSX, this means you won't have frame advance, for example. But you do have quicksaves, and due to the recording, the emulator will be running slower than usual, so that helps too. Good: * Allows creating TAS-like videos (in fact, they're TASes, just unverifiable) on emulators that do not have TAS support. Bad: * Requires a lot of disk space: the whole recording session must be recorded, including those frames which you undo; and a keyframe must be explicitly generated every time a savestate is made, to allow for stitching. Recording is done at very high bitrate encoding, in order to allow making postprocessed AVIs at various quality levels. * Impossible to validate the resulting video. Source code The source code of this project is available at: http://bisqwit.iki.fi/src/pSX-avi-recording-script.zip Instructions are included inside, in a HOWTO.txt file. Sorry, this will not work on Windows, for Windows lacks the LD_PRELOAD technique. It could possibly be done by the means explained here, but I lack the motivation & expertise to try it out. Oh, and just to clarify, this particular release only works with pSX. Each different emulator has their own library routines that they call and in different manner, so the intercepter must be customized to take account for the differences. It is configured such that it records video at constant 320x240 resolution, using the x264 codec. (Obviously, x264 must be installed.) When the emulator produces video at a different resolution, it is automatically scaled using the lanczos filter. (You'll notice it does when the emulator slows down significantly.)
Posted: 4/11/2008 9:34 AM
Quote
moozooh
Senior Moderator
Joined: 8/4/2005
Posts: 5777
Location: Away
A similar, even an easier approach (through a different process, though) can be taken with the games that have built-in recording engines: many FPS games, certain freeware games, ElastoMania, etc. Kind of like Doom, but without custom engine with TAS tools built in. Now that you've reminded me with this thread, I think I'll try researching "external tool-assited" running some more, and maybe show some results.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Posted: 4/11/2008 10:27 AM
Quote
LagDotCom
Joined: 3/7/2006
Posts: 720
Location: UK
Bear in mind that there are two (or maybe more) types of 'demo recording engines': 1. Engines that record button presses. 2. Engines that record rendered graphics. Obviously making a TAS of a #1 engine is possible, but editing a #2 demo file is nothing to do with the actual player at all, you could make him finish in one frame by placing him at the exit by directly modifying his position coordinates. Of what demo files I have looked at... 1. Doom series/engine, Worms Armageddon 2. Quake series, Serious Sam, Descent
Voted NO for NO reason
Posted: 4/15/2008 7:15 AM
(Edited: 5/30/2008 5:43 AM)
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Yep, the problem with multimedia re-recording is that the resulting product can not be validated: there's no way to verify whether it was actually the product of playing or if some video editing trickery transpired. Still, if you can trust the author, it allows for impressive videos indistinguishable from those TASes that can be validated. I updated the source code package a bit, and added a stitching script in it. Basically, to create a pSX "TAS", you'll do the following: HOWTO.txt included in the package describes the use. Caveats: ä̤) There is no "read-only" movie playback. If you make a mistake and you've got no savestate to backtrack with, you have to either accept the mistake or begin anew. So, you must be careful with where you make savestates and where you don't. b̤̈) Every time you switch windows while the emulator is active, 1 frame of audio is lost. Cumulative, this results in a gradual A/V desync in the video. You can undo the lost frame by loading a savestate that was created before the frame was lost. You can avoid this entirely by only switching windows when the emulator is paused (using the Enter key). c̤̈) The video and audio written in the directory indicated by BASE will be accumulated every time you run a frame on the emulator, regardless of whether it is later undone or not. This mean that even for short movies, you will quickly accumulate large files, because all your failed attempts are still recorded in those files. (Later, a tool can be written to shorten these files.) d̤̈) pSX savestates are good but not perfect: Notably, the sound echo state is not saved in the state. You should avoid loading savestates when something having an echo is playing, because it will still echo when you load the state. In the stitched AVI, it will sound like an echo of something appears from nowhere. It is a cosmetic inconvenience only. ë̤) "TAS"es created with this way cannot be submitted at TASVideos.org, because they cannot be validated. But be sure to post them in your own webspace and provide a link on these forums! --- Footnote: Of course it would be a dozen times more preferrable to have robust re-recording functionality in pSX itself, but unfortunately, the author of pSX seems to be stonewalling me.
Posted: 4/16/2008 4:25 AM
Quote
Sticky
Joined: 2/26/2007
Posts: 1365
Location: Minnesota
I think this is a great step in psx tasing. Good job Bisqwit!! I also think that, until verifiability comes along, this should be more of a forums only thing. Sure, if Aqfaq makes a psx run, I believe him. But what about me making a FF7 run? I have never made a TAS, so who knows if I would cheat it. Publishing should not be available to Aqfaq and not me, for obvious reasons, but it would be cool if the community (or whoever did make the runs) could post them somewhere for the world to see. Of course, then comes the problem of "Why strain to make a run of FF7 when it will not be [immediately] published?" But, then again, labors of love rule the world!
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
Posted: 4/16/2008 4:48 AM
Post subject: Don'tCareAboutNewConsolesThatMuchButTheValidationProblemIsOk
Quote
Aqfaq
Editor, Expert player (2477)
Joined: 4/8/2005
Posts: 1573
Location: Gone for a year, just for varietyyyyyyyyy!!
Sega TASer of 2008 Lucky TAS of 2008
Bisqwit wrote:
Yep, the problem with multimedia re-recording is that the resulting product can not be validated: there's no way to verify whether it was actually the product of playing or if some video editing trickery transpired.
Well, you could always ask the author to explain the parts which look suspicious. Every existing trick can be replicated and people who know the game will easily spot even tiny game modifications or video edits. No validation system could ever be 100% fool-proof, but I think that the best movies would have a higher probability of being valid, because a good TASer needs no video editing trickery to produce movies that look suspicious. :) I suppose one of the easiest ways to cheat would be to fix some random encounters in RPGs. Generally, valid luck-manipulation would be hard to verify. But most of the times the cheater would be caught eventually or the cheated movie (probably featuring sloppy gameplay) improved by someone who can TAS for real. Interestingly, the validation problem goes both ways: If somebody made a completely valid movie, which uses a super rare glitch that nobody could ever replicate, people might not believe it to be valid. The poor TASer ninja would then be very sad. To further make the validation problem seem bigger than it is: Somebody might make an invalid TAS unintenionally by using a bad/hacked ROM without knowing it. This could happen easily, unless some sort of ROM checksum type of solution is used for verifying that everyone uses the same version of the game. In any case, the validation problem will surely not ruin the fun of Multimedia TASing (MTAS?) any more than it ruins the fun of the SDA-style speedrunning.
stickyman05 wrote:
Sure, if Aqfaq makes a psx run, I believe him.
Oh, thanks!
Posted: 4/16/2008 6:26 AM
Post subject: Re: Don'tCareAboutNewConsolesThatMuchButTheValidationProblem
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Also, more insidious methods of cheating with video-only exist: Remove every seventh frame, and you've got a 14% faster video and most people can't notice why. (Though it would be somewhat limited to situations where the music speeding up cannot be noticed.) Well, remove every 20th frame, and you've got a 5% faster video, and that people wouldn't probably notice.
Posted: 4/16/2008 4:29 PM
Quote
amaurea
Former player
Joined: 2/19/2007
Posts: 424
Location: UK
Actually, whether one can trust a multimedia tas is not only a question of whether the author can be trusted. If the savestates are incomplete, one will end up in an unrealistic state after loading one. If the game progresses through states A, B, C, D, E, and you made a savestate at C, loading it at point E, the new version of history will be A, B, C*, where C* is different from C if the savestates are unreliable. The state C* might not even be reachable normally. Inaccuracies like this can accumulate whenever you load a state. As an extreme example of why this is a problem, suppose that the "on the ground" flag of some game is not properly saved in the savestate, and that C was in the air, while E was on the ground. C* could then end up being able to jump, even though it is in the air. Any gains from this would not be legitimate. There is a way to check if this problem exists. Record your resulting inputs, and play them back to the emulator. If this causes a desync, then one of the state loads brought you back to an incorrect state, and the multimedia file cannot be trusted.
Posted: 4/16/2008 4:33 PM
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
True, emulator inaccuracies in savestate loading can also cause peculiarities. According to my perception, pSX savestates seem to be quite robust, though. The only problem I've noticed has been with the sound echo buffer; it is not affected by savestates.
Posted: 4/16/2008 5:01 PM
Quote
amaurea
Former player
Joined: 2/19/2007
Posts: 424
Location: UK
When I stated that any desync during rerecording was proof of savestate problems, I had forgotten that nondeterministic emulation is another likely source of desyncs, that can happen even in the absence of savestates. Bisqwit reminded me of this on IRC, but I thought I'd post it here, so my earlier comments don't mislead people. Nondeterministic emulation is, in a way, just as problematic as savestate problems, as the game might end up in a different state than it would have been in on the actual console, but since it happens even during normal gameplay, any significant deviations would have been noticed and corrected already.
Posted: 4/16/2008 8:12 PM
Quote
ZeXr0
Joined: 8/27/2006
Posts: 883
You can see that the administrators of the forums doesn't fully trust savestate, and they even recommend not using them for a full run because they can left junk in memory and that this junk isn't cleaned between two gameplay.
Posted: 8/15/2008 12:36 AM
Post subject: excellent
Quote
Ersatz
Joined: 8/9/2004
Posts: 139
Location: Washington State
That is amazing work and awesome, but I gotta be the guy who says... Now, can we get it for the dreamcast? I'd love to do Ikaruga.
Posted: 8/19/2008 9:13 AM
Post subject: Re: excellent
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Ersatz wrote:
That is amazing work and awesome, but I gotta be the guy who says... Now, can we get it for the dreamcast?
Is there a Dreamcast emulator? Edit: Or rather, is there a Dreamcast emulator for Linux? (Because this technique is not ready for use on other platforms such as Windows yet.) Edit 2: There is at least one ― lxDream. It even has source code available! Source code makes things so much easier. Now, the next questions to be answered are: ― Can it play games?¹ :) How well? ― Does it output synchronized sound (i.e. 44100 samples of audio per 60 frames of video regardless of host system lag / turbo)? Does it have a no frame skipping option? (It doesn't matter if such option makes the emulator run slowly.) If it does not have those features, then it becomes difficult (or impossible) to record a smooth video. ― Does it have savestates?² How robust? After satisfying answers to those questions, adding a multimedia rerecording to the emulator is just a matter of some coding, which is best done by someone who is motivated with working with that console. Right at this moment, that excludes me. (But I can help with information concerning the code I already have written.) Edit 3: ¹ Seems like the game compatibility is even worse with lxdream than it is with Mupen64. Hopes for playing <insert game name here> are rather low. ² Seems to have. Robustness is an unanswered question.