Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Having a single instance with spu switch will remove the need for sleep().
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Having a single instance with spu switch will remove the need for sleep().
Yes, make one and only one script would be great.
Ok, actually here's how the current workflow work for those who wonder.
Part1Part2(The app2 is the interesting part)
Yeah I know, I could lower the value of the tolerable desync.
For instance with the Azure Dream movie, I keep getting this:
frame 2600 Hash ok
frame 2601 Desync
frame 2602 Hash ok
frame 2603 Desync
Almost always. And there so many loading screen, that make it even more problematic.
As you know, if we load a savestate at every let's say 50 frame. In the end you get almost the same result as using the TAS sound plugin. You see what I mean?
At some point, it's up the encoder to choose the right value for the tolerable desync.
Yeah I know, I could lower the value of the tolerable desync.
For instance with the Azure Dream movie, I keep getting this:
frame 2600 Hash ok
frame 2601 Desync
frame 2602 Hash ok
frame 2603 Desync
Almost always. And there so many loading screen, that make it even more problematic.
As you know, if we load a savestate at every let's say 50 frame. In the end you get almost the same result as using the TAS sound plugin. You see what I mean?
At some point, it's up the encoder to choose the right value for the tolerable desync.
In your example, the save state for frame 2601 was where though? I think that matters. If it is 50 frames back, you need to get a save closer to frame 2601.
In your example, the save state for frame 2601 was where though? I think that matters. If it is 50 frames back, you need to get a save closer to frame 2601.
Usually that sync/desync pattern was on a loading screen.
So when I saw your post talking about this issue on loading screen, I knew tolerable desync feature was something necessary.
edit: At some point here an another crazy idea: We might need a "formula guideline" for checking the amount minor desync VS major desync in order to determinate what is the amount of tolerable frame desync in order to keep a right level of sanity for "sound memory" (using eternal plugin).
Someone understand what I mean? :D
Joke aside... I'll post some more progress soon.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
How about moving "save to xml" to sync script, while detect makes a save?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
While patrolling the detect Script read the XML. Same for the the sync script.
The other way around would have been to use socket, much easier. Shame on me.
In your example, the save state for frame 2601 was where though? I think that matters. If it is 50 frames back, you need to get a save closer to frame 2601.
Usually that sync/desync pattern was on a loading screen.
So when I saw your post talking about this issue on loading screen, I knew tolerable desync feature was something necessary.
edit: At some point here an another crazy idea: We might need a "formula guideline" for checking the amount minor desync VS major desync in order to determinate what is the amount of tolerable frame desync in order to keep a right level of sanity for "sound memory" (using eternal plugin).
Someone understand what I mean? :D
Joke aside... I'll post some more progress soon.
Does lua know what frame you are currently on? The compare should only be on the same frame number. That will fix the missing loading frames problem.
Feos and I did some debugging thought pcsxrr for getting a switchspu function on the fly. I did some test and it seem to work if you proceed that way:
Language: pseudo-code
at frame 2600 : make a savestate(using "spuPlugin#1.dll")
call emu.switchspu("spuPlugin#2.dll")
loadstate to frame 2600
call emu.frameadvance()
So yeah, still need a load savestate in order to get immediate result, but I think this is a pretty good result.
@Aktan: I'll need to look exactly what's up with the loading screen a bit more before answering you. But, I just want you know that the "tolerable desync" is a required concept for now and as we keep testing different game, we might lower the value for those tolerable desync.
Last point, I should probably start thinking on how to redesign the whole script efficiently(in order to make it easier to maintain the script) now that I don't have to bother with some basic issue FROM pcsxrr.
edit: using only using SPU_freeze while switching plugin seem to make the sound working between TAS sound plugin to eternal and vice versa. We will need one more fix to make sure switching plugin doesn't force lua to exit directly his current script.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Meanwhile...
http://code.google.com/p/feos-tas/source/listFri, May 10 2013 9:08pm pcsx: Calling menu now resets movie mode.
Fri, May 10 2013 8:01pm now it's possible to switch spu plugin multiple time in a single script
Fri, May 10 2013 3:53pm pcsx: SPU switch looks implemented. State gets passed between pluings. Sound works.
Fri, May 10 2013 5:02am Added string as param for pcsx.switchspu for dll change
Fri, May 10 2013 2:35am pcsx: lua function to switch spu almost done. It loads eternal spu, but it has no sound.
One can use a test script like this:
Download SwitchSPU.lua
Language: lua
local key = {"V", "B"}
local now = {};
local lastframe = {}
function KeyEvent()
for k, v in pairs(key) do
now[k] = input.get()[v]
now[v] = now[k];
if now[k] and not lastframe[k] then
print(v)
if (v=="V") then
print("load spuTAS")
pcsx.switchspu("spuTAS.dll");
elseif (v=="B") then
print("load spuEternal")
pcsx.switchspu("spuEternal.dll");
end
end
lastframe[k] = now[k]
lastframe[v] = now[k]
end
end
while true do
KeyEvent()
emu.frameadvance()
end
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Ok, let's say all my previous script was only a "proof of concept". The double instance thing seem to be a failure to fully automate the process since it share savestate and it's almost unmaintainable, but I like the new direction we're are getting now.
By the fact that we're starting to get more and more confident with pcsxrr source code, thing should be a lot easier now. I can already say that next version is gonna be at least 300% faster(from the screenshot hash processing) and far less overhead on the lua side.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
gui.hashframe() is now added in r582. This new function is a bit faster than using gui.gdscreenshot(), but somehow still not as fast as the the snapshot from the plugin feature, that I tested the other day. While makeNormalSnapshotPNG() and loading the snap in lua.
Also, a nice addition that I am thinking would be a movie.load("movie.pxm"). Such function would help a lot when testing special case where a desync might happen after the movie.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Since openGL2 won't give screencontents to lua (callback isn't present), and GPUfreeze would be required to use for it (which is slow), maybe really we shall create snapshots, hash the files and remove them, as you suggested earlier. Otherwise it will stay incomplete.
Dunno if it shall be done only for opengl2 case, or for all cases.
EDIT: Well, pcsx must also be able to hash the newly created snapshot file (even is it does not create it itself) and delete it, right? Still may be faster than lua.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
My vision of the future workflow so far.
1. Pass one - TAS SPU. Playback the movie, dump every frame hashes. Also, take savestates for every 100th (?) frame.
2. Pass two - Eternal SPU. Playback the movie, hash every frame again, compare the values with the ones from the first pass.
a) if mismatch, first mismatch frame gets stored in memory.
b) if constant mismatch lasts 15 frames, switch to TAS SPU, load the last backup state, run to the first mismatch frame. Save state. Switch to Eternal SPU. Load state. Resume playback.
c) in this case the first mismatch frame gets dumped to a file. Otherwise, the value gets replaced by the new one at the next mismatch.
3. Pass tree - Eternal SPU. Start capture, let the states from the framelist be loaded on playback.
Notes about implementation and usage.
First of all, one must use the same GPU plugin for hashing and final capturing. If you do antialiasing, use such a plugin right on the first pass. This ensures that you save control states with the same picture you will be using at dumping and use them without any visual difference then.
Second, in pcsx, all but client and core is done by plugins completely. They output sound, video, dump them, make screenshots, etc. Regular plugins also provide lua access, for example, most GPUs have a callback to give lua the framebuffer. So, we can order it via lua and use as we wish. Right now it is how lua framehashing is implemented.
But modern OpenGL2 plugin by Pete that supports antialiasing does not return any frame bitmap to lua and seems to ignore its callbacks completely. So we just can't order bitmap the fast way.
There are 3 ways to grab bitmap though.
Call GetScreenPic that is used by savesates and can be very slow.
Call GPUfreeze that returns VRAM contents in one of the parameters.
Call GPUmakeSnapshot and the delete the file to prevent overload.
Each of these returns framebuffer of some kind and can be hashed. It wasn't just fully tested which would work faster.
It must be decided, will one of the hashing ways be used for all cases (any GPU), or it will be made dependent on plugin (luahashing for GPUs that allow lua interacition), one of the 3 above for OpenGL2.
Finally, we must work out the savestate usage: how frequently we can afford it. With no compression each is 4MB and isn't too managable. Compression 9 makes it a bit over 1MB, compression 1 gives about 2MB. If we save every 100th frame, for an hour movie we get 2000 savestates that are about 4GB in total. Recalling how huge our antialiased encodes (up to 100GB and more), we can afford such a thing.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
New edition.
http://code.google.com/p/feos-tas/source/detail?r=582
Now we have lua hash implemented in pcsx. It uses the framebuffer provided by lua loop that co-operates with GPU (also used by dgscreenshot for example).
It was proven, that it never matters what GPU was used to make a state, when you load it, you get the picture you need at proper resolution. It means, we can do all passes with lowres TAS GPU and move to hires antialiasing when capturin, it will look proper. So we can rely on lua hashing pretty much.
If we could keep 2 instances in sync, we won't ever need frequent backup savestates. If framecount of one instance overruns another, it must be set to sleep. Once check instance (eternal) finds the first desync, partol instance (tas) makes a state and freezes. If desync in eternal lasts 15 frames, it loads the state tas instance just created.
If the state was LOADED by app2, that state is reserved. If desync disappeared, both resume running, and at the next desync this state gets replaced somehow.
At the end, we will only have the necessary states. They are to be loaded at capturing. And all that can be done in lua with small expect of OOM.
BUT: we can't control 2 instances with one script. Only through sockets. So, we either keep using sockets, or add some kind of interaction to pcsx, or build the external app to read the returned hashes and framecounts, make and load states, sleep instances. Probably not as hard as it can sound. But some experienced hints are needed.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
That new workflow seem better than my previous one. Should be good.
As for implements this, it seem there still several way to do it. Right now, Nach told me about adding IPC interface to pcsxrr and I'm very interessed, so I can basicly manage the WorkFlow without Lua assistance and from an external app(in Java or C#). So right now, I'll be looking foward to his next if he can add some function like:
frame_num()frame_hash(), (note: mine from lua could probably get optimised to run faster...)
state_save(path), (note: path is required, so we can store desync elsewhere)
state_load(index)sleep(seconds) (note: later I'll investigate if there a way to "yield" without completly freeze the app, but for now sleep is good enough)
Meanwhile, I'll continue to add some Lua function, so if for wathever reason I dislike IPC, I can go back to Lua with a little bit less of overhead. I'm also thinking about adding a easier feature to show frame input, for game that doesn't support the GPU TAS and must use OpenGL2 and have to use weird way to show input like this.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
You won't need path for savestates. State is stored in RAM as it is created. If it is not needed (desync lasted less than 15 frames), it will be removed from RAM. If it was used by app2, it gets dumped to disc.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Ok, after a small talk with feos we both agreed on what function we needed, so I edited my previous post.
As discussed, we also think that state_save/state_load IPC function need to store the savestate in RAM.
I'm still not sure for the reason of tolerable desync. Only thing I can think of is the loading screens in AD are just slightly different a few times, but I think I fix those to be exact too (hence the pain). Here is what I envision as the steps.
All done with TAS GPU except step 3 (unless it can't sync with TAS GPU then we have a different problem)
1. Pass one - TAS SPU. Playback movie. Dump every frame hash. Savestate to disk every x (100?) frames.
2. Pass two - Eternal SPU. Playback movie, hash every frame, compare hash to the same frame number as TAS SPU (will fix loading problems?). If mismatch, load the closest previous savestate that is before 15 previous frames of the mismatched frame, playback movie until 15 frames before the hash mismatch frame, save state there, continue playback to see if the mismatch has been fixed if not, repeat with loading the savestate at 15 frames before mismatch frame, save state at 14 frames before hash mismatch, if still does not fix desync, then try 13, then try 12, etc..., until the mismatch is fixed, save the frame number difference from savestate that fix the desync to the hash mismatch (to be used later in Avisynth), and then delete all unneeded savestates. At the end of this step, most or all of the savestates from pass one should be gone as only the savestate from each desync made from pass 2 is left.
3. Pass three - Eternal SPU. Capture all the desync parts to one AVI sequence, loading each state at certain points after a desync and write the trim points to file.
Now steps 2 and 3 are probably not explained well enough, and some of step 3 may be impossible to do, so let me know and I can clarify as best as I can.
Again, I really do not think tolerable desync is needed at all, and if it causes it to have 100 more desyncs, I say meh since it is now all automated.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Except the final tool still doesn't exist yet.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
The final tool for the extra 100 desyncs? Of course not, but that's what what we are aiming for right? I don't think we should compromise accuracy for automation. It be better off the manual way then since it be more accurate.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
If we keep the backup application 15 frames behind the check application, we could apply your request easily. But now we need the program to control 2 instances, Nach promised some IPC help.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
If we keep the backup application 15 frames behind the check application, we could apply your request easily. But now we need the program to control 2 instances, Nach promised some IPC help.
You don't need a backup application, you just use the savestates made from step 1 which is a savestate every x (100?) frames.
For example. Desync on frame 1234. 1234 - 15 = 1219. Load savestate at frame 1200, aka the 12th savestate.
