Wohw, NES games in 10-bit just look spectacular! I have always refuse to download a speedrun, prefering the full quality of the emulator but now I may have just change my mind after seeing the 10-bit encodes.
I am confused about something though, the colorspace of our games is RGB, but people call it 8-bit because its 8 bits per channel, but isnt RGB what our quality games are? Is 10-bit 1024 shades for RGB? I am so lost.
Also, how do I make a 10-bit encode? I have captured from FCEU in FFV1 video codec but opening it with avisynth gives me low quality YV12 output. Opening it direcly with virtualdub gives the high quality RGB output. Is there another, better lossless codec I can use besides FFV1? Is there a x264vfw that can do RGB lossless?
Joined: 4/17/2010
Posts: 11479
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
The best compressing+capable lossless codec is CamStudio. Second place if after Lagarith.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator output is (usually) 8 bit per channel RGB. 10 bit encodes are 10 bit per channel (1024 shades). In this case the colorspace is also different; it's YUV. Why 10 bit YUV compresses better than 8 bit YUV in x264 is complex and I don't understand it. The big difference you're probably seeing is the lack of chroma subsampling, which is also used in our 10 bit encodes.
There are a variety of ways to manage your video workflow. What I would recommend is to capture lossless RGB from the emulator, import lossless RGB to avisynth, and do the entire avisynth process in lossless RGB. Then send RGB (using avs2pipemod or similar) to x264 (10 bit build) and have it convert to YUV 4:4:4 10 bit internally.
This post has some RGB lossless codecs in it:
http://tasvideos.org/forum/viewtopic.php?t=12390
As far as your particular FFV1 question, I'm not familiar with it. I can only say that these sorts of problems (colorspace issues with import) are why planning out your workflow is difficult.
At the moment, x264 RGB lossless is not recommended. The only way to get it as output from most emulators under windows at a moment is to use a custom codec tool I made, and you don't want to worry about that (it's rather horrid). There's also no convenient way I know of to import it into avisynth. (Both FFMS2 and ffdshow-tryouts give a colorspace problem).
These are the intermediate codecs I use:
Camstudio - kind of slow, VFW codec is 32 bit only. Supported directly by libav* if that's important to you.
Lagarith - much faster than Camstudio, rather large files. Null frames gives compression increase in deduppable games (eg: gens prince of persia), but chokes up ffmpeg command line (works fine in vfw+avs).
MLC - slow as balls, smaller than either Camstudio or Lagarith. Gives exceptions in Vdub when trying to repeatedly reload a script, pain in the ass. Codec of choice for getting longer Gens dumps under 2GB (as Gens always screws up the 2GB split).
ZMBV - no 24 bit input option, only 32 or 16, so mostly only for psxjin. Can be slow or fast depending; small.
Uncompressed - I actually did this once for the fun. Can be slower than Lagarith because of i/o bound (especially if you try to rapidly seek the avs script in vdub). Monstrously huge files.
LCL zlib - available from commandline ffmpeg, and can be loaded with FFMS2. Speed&performance somewhere around lagarith. Doesn't work with standard tools.
Joined: 4/17/2010
Posts: 11479
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Funny thing that some codecs may provide output of different size that become the same after regular x264 encoding. Happened for me with CamStudio and MLC.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Funny thing that some codecs may provide output of different size that become the same after regular x264 encoding. Happened for me with CamStudio and MLC.
That's completely expected, no? All of these codecs are exactly lossless, so that what they give avisynth will be exactly the same in every case. How they store it internally (which is their own problem) though, is what determines their filesizes.
No, video compression usually uses YCbCr instead of RGB.
YCbCr represents colors as follows
Convert the image to greyscale. This is the Y channel (called luma).
Then compute difference image between the original and greyscale version.
The blue channel of this difference image is the Cb channel and the red channel is the Cr channel (both are collectively known as chroma).
Now, 10-bit YCbCr depending on the variant has:
* 1024 shades for luma, 1024 for each chroma channel (known as full or PC range).
* 880 shades for luma, 896 shades for each chroma channel (known as TV range).
However, this isn't the reason why 10bit encodes look so much better. Yes, 8-bit YCbCr has some color loss, but it is insignificant (however, 10-bit improves compression).
The true reason is follows: Normally every pixel has luma, but only each 2x2 pixel block has chroma (this is chroma subsampling, done since human eye has better resolution for luma than chroma and it improves compression).
This of course causes colors to bleed. And on low resolutions, this is very noticeable.
However, 10bit444 encodes don't subsample chroma, so every pixel has its own chroma and thus the color is independent from all other pixels.
One way to see this: Grab some DOS encode, as VGA pixeldoubles/doublescans, pretty much eliminating chroma subsampling artifacts (but the encodes still are 8-bit).
Santiago wrote:
Also, how do I make a 10-bit encode? I have captured from FCEU in FFV1 video codec but opening it with avisynth gives me low quality YV12 output.
IIRC, there was a bug in AVIsynth that caused FFV1 to be decoded as YV12 unless you overrode this (some AVIsource option I don't remember offhand).
IIRC, there was a bug in AVIsynth that caused FFV1 to be decoded as YV12 unless you overrode this (some AVIsource option I don't remember offhand).
Now that you say that, I know what it is. The problem is that VFW gives no direct way of knowing what the "compressed" colorspace is, other than what the decompressor supports as output. (IE there's no way to support multiple colorspaces as decompression output while marking one as "preferred"). Because of the way it's constructed, the ffdshow vfw FFV1 decompressor supports YV12 output unless you change one of those 15 billion settings it has. And AviSynth checks for YUV output possibilities from the decompressor first (this is actually a good idea for certain other codecs, but not helpful here).
The preferred way to fix this would be:
AviSource ("foo.avi", pixel_type="RGB24")
# if that doesn't work
AviSource ("foo.avi", pixel_type="RGB32")
Joined: 4/17/2010
Posts: 11479
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Was screwing up blue color fixed in FFV1? I wonder why this codec is still mentioned in our guide at all, after having such non-intuitive issues and being TOTALLY obsoleted by LAGS and CStudio.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
The best compressing+capable lossless codec is CamStudio. Second place if after Lagarith.
ThatGugaWhoPlay wrote:
Lagarith Lossless, no doubt about it!
You should also mention what your criterium is.
CamStudio is very fast and achieves a nice compression for the little time it spends on it.
Lagarith compresses more but when you're editing all that usually matters is speed, so I don't see it as that useful.
natt wrote:
At the moment, x264 RGB lossless is not recommended.
Did you mean h.264?
natt wrote:
Camstudio - kind of slow, VFW codec is 32 bit only.
There's a faster one (that involves compression and a harddisk)?
natt wrote:
Lagarith - much faster than Camstudio
Only if you have enough cores, afaik. On a single-core machine it'd probably be slow.
natt wrote:
ZMBV - no 24 bit input option, only 32 or 16, so mostly only for psxjin.
What would be interesting is a "fake" codec that converts everything to 32-bit (or 16-bit, if the end result is the same) and passes it over to ZMBV.
Joined: 4/17/2010
Posts: 11479
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I mean that Camstudio gives the highest compression than what I've had tried besides, when using at GZip, Level 9. But so it works way slower than LAGS. And CamStudio encodes are easy to seek through (not if LZO method is used).
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
You should also mention what your criterium is.
CamStudio is very fast and achieves a nice compression for the little time it spends on it.
natt wrote:
ZMBV - no 24 bit input option, only 32 or 16, so mostly only for psxjin.
What would be interesting is a "fake" codec that converts everything to 32-bit (or 16-bit, if the end result is the same) and passes it over to ZMBV.
I have played with the following lossless codecs:
* CSCD (Camstudio, gzip mode)
* TSCC (Camtasia)
* ZMBV (DOSBox)
TSCC wins over CSCD in both compression and speed (unfortunately, it has other issues).
ZMBV (DOSBox reference implementation)... Settings that get filesizes that are comparable to TSCC/CSCD (I got it between TSCC and CSCD in compression) are much slower than what either CSCD or TSCC can do (less than half of the speed).
But then, it could have been the test material I was using... But it looked like the kind of material where ZMBV would really rule (motion estimation!)...
For these reasons, I decided not to include ZMBV codec in lsnes (but I still have the fully functional codec code for ZMBV).
What would be interesting is a "fake" codec that converts everything to 32-bit (or 16-bit, if the end result is the same) and passes it over to ZMBV.
Tried it, two ways:
1) Actually change zmbv to support 24 bit directly
Failed because of some weird compilation issue, some sort of C++ crash, never figured it out. The code was right (the core work was done by Bisqwit a while back), it was just a problem with my compiling environment?
2) Construct a "proxy codec" that sits inbetween AVIFIL32 and the ZMBV implementation and converts 24->32 (which can be completely lossless).
Worked, but output files were ungodly huge, never figured out why.
I'm not the best coder around =/
Why 10 bit YUV compresses better than 8 bit YUV in x264 is complex and I don't understand it.
It is easiest for me to think of it in terms of banding.
A color gradient in 8 bit has banding because we can see the difference from one color to the next.
To fix this in 8 bit you can apply dithering, a checkerboard pattern which gives an appearance of a greater number of colors.
But the pattern is hard to compress, so you lose it if you don't up the bitrate.
10 bit has more colors, so it doesn't need dithering, so you can produce the same visual effect in less bits.
(you don't have a 10bit monitor so the dithering is still there, but it is added after decompression by the codec or your graphics card)
Chroma Subsampling
Dropping most of the color data cuts the raw file size/bandwidth in half.
This works ok because color data is mostly redundant.
But video compression is great at reducing exactly the type of redundancy present in chroma data.
So for compressed video you can have full chroma at very little increase to bitrate.
Ok I now use Lagarith and avisynth recognized it as RGB. However, x264 10-bit still outputs YV12 video, wtf? I did add that --outputt 444 or whatever it is called in case you ask.
Ok I now use Lagarith and avisynth recognized it as RGB. However, x264 10-bit still outputs YV12 video, wtf? I did add that --outputt 444 or whatever it is called in case you ask.
Does the Avisynth script contain those ConvertToYV24/ConvertToYV12 lines? Don't do that when encoding 10bit444.
And the option is '--output-csp i444'.
Does the Avisynth script contain those ConvertToYV24/ConvertToYV12 lines?
No.
Ilari wrote:
And the option is '--output-csp i444'.
Yes I know that and I did add it.
Then x264 outputted 4:4:4 (and if it was a 10 bit encode, 10 bit). What are you checking with? MediaInfo is always a good place to start. MadVR's extra info display (CTRL+J) can tell you some useful statistics about a video as well.
Then x264 outputted 4:4:4 (and if it was a 10 bit encode, 10 bit). What are you checking with? MediaInfo is always a good place to start. MadVR's extra info display (CTRL+J) can tell you some useful statistics about a video as well.
I'm checking it with my eyes and seeing obvious YV12 garbage, I do not care what mediainfo or anything says it is.
Edit: opening the AVS script with Virtualdub outputs colorspace fine, it's only when I encode it that it looks like YV12 crap.
Then x264 outputted 4:4:4 (and if it was a 10 bit encode, 10 bit). What are you checking with? MediaInfo is always a good place to start. MadVR's extra info display (CTRL+J) can tell you some useful statistics about a video as well.
I'm checking it with my eyes and seeing obvious YV12 garbage, I do not care what mediainfo or anything says it is.
That's useful information, but it's not the same as what you said before. A problem in the workflow can cause data to pass through 4:2:0 through an inappropriate conversion, even if it ends up as 4:4:4.
How are you passing the avs script to x264?
That's useful information, but it's not the same as what you said before.
What?
natt wrote:
A problem in the workflow can cause data to pass through 4:2:0 through an inappropriate conversion, even if it ends up as 4:4:4.
How are you passing the avs script to x264?
With x264 10-bit.exe I use a .bat.
Nanogyth, no. I see YV12 output, it is obvious because colors are blured like they normally are when they are YV12 and not RGB. Mediainfo says it is 4:4:4 but it is not.
So x264 itself is opening the avs file?
Hmm... could you maybe provide a short example clip? Both encoded to some RGB lossless (say, camstudio or lagarith) and encoded through x264?
Nanogyth, no. I see YV12 output, it is obvious because colors are blured like they normally are when they are YV12 and not RGB. Mediainfo says it is 4:4:4 but it is not.
I have seen media player that will convert 4:4:4 videos to YV12 when playing back (at least with some configs), so one better play the video point-scaled or it will seem to have chroma subsampling artifacts...
Of course, if 10bit444s in the site play fine at 1x but your encodes don't, that isn't the problem...
