Posted: 11/1/2009 4:15 PM
Post subject: Encoding challenge
Quote
moozooh
Senior Moderator
Joined: 8/4/2005
Posts: 5777
Location: Away
Inspired by the difficulties with encoding the recent Shadow of the Beast submission, I decided to run a few tests of my own. After making 20 excruciatingly slow test encodes on a short (26 second long) sample I realized that doing this alone would be much slower than otherwise. Since I originally aimed to use a bitrate of 500 kbps I decided to freeze this value and try to optimize quality by tuning other x264 settings. I use SSIM (append --ssim to your x264 commandline) for quality measurements since it's the most accurate quality measurement algorithm available to me. So, here's the challenge. Get a SSIM value of 0.92+ on the following sample: http://www.sendspace.com/file/slab97 (249 MB). So far my best attempt got 0.9151, but I didn't max out a couple of potentially useful parameters due to lack of available waiting time. Restrictions: use --fullrange on, don't use --deldup (for this sample). Target bitrate is 500±5 kbps. Post your best encodes along with their commandlines, and maybe we can improve something in the current encoding standards! Or maybe you will find some settings in your usual commandlines that decrease the quality in some way. My current commandline: 
--bitrate 500 --ref 16 --no-fast-pskip --bframes 10 --b-adapt 2 --direct auto --deblock -4:-4 --subme 10 --trellis 2 --partitions all --me tesa --merange 64 --aq-strength 2 --no-dct-decimate --ssim --fullrange on --rc-lookahead 250 --qpmin 12 --qblur 0.1 --keyint=1800 --weightb --8x8dct --mixed-refs
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Posted: 11/1/2009 4:20 PM
Quote
zefiris
Joined: 4/11/2006
Posts: 487
Location: North of Russia :[
would not optimal setting vary for different videos? I mean... Why try to get hishes possible quality on sample movie if you can't be sure results will be as good with other movie?
Posted: 11/1/2009 4:33 PM
Quote
moozooh
Senior Moderator
Joined: 8/4/2005
Posts: 5777
Location: Away
They will more likely than not give results at least as good. …And because it's a challenge. :D Realistically, the only setting that varies between all movies is bitrate, the rest is more-or-less the same, sometimes lightened to save time. Pretty much every other setting in my commandline is quality-for-time tradeoff.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Posted: 11/1/2009 6:46 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
1-pass line: 
--preset placebo --tune ssim --pass 1 --slow-firstpass --bitrate 500 --stats "" --ssim --fullrange on --rc-lookahead 600 --output "" ""
2-pass line: 
--preset placebo --tune ssim --pass 3 --bitrate 500 --stats "" --ssim --fullrange on --rc-lookahead 600 --output "" ""
SSIM Mean Y: 0.9203750 bitrate: 501.09 link to file: http://www.mediafire.com/download.php?5tlyj3ynojt note: --rc-lookahead seem to be capped at 250 note 2: personally, I don't see why one would want to limit themselves to a certain bitrate. Just set it to crf <20 and be done with it =p
Posted: 11/1/2009 9:54 PM
Post subject: SSIM 0.9238759
Quote
Raiscan
Joined: 11/11/2006
Posts: 1235
Location: United Kingdom 
..\common\x264.exe "sotb-outdoorsample.avi" --stats "./sample.stats" --pass 1 --slow-firstpass --ssim --no-psy --bitrate 500 --min-keyint 60 --keyint 3000 --ref 16 --deblock -3:-3 --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree  --weightb --direct auto --subme 10 --trellis 2 --rc-lookahead 600 --partitions all --fullrange on --aq-mode 2 --me tesa --merange 64 --threads 4 --8x8dct --no-dct-decimate --output "test.mp4"> outputpass1.txt 2>&1

..\common\x264.exe "sotb-outdoorsample.avi" --stats "./sample.stats" --pass 3 --ssim --no-psy --bitrate 500 --min-keyint 60 --keyint 3000 --ref 16 --deblock -3:-3 --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree  --weightb --direct auto --subme 10 --trellis 2 --rc-lookahead 600 --partitions all --fullrange on --aq-mode 2 --me tesa --merange 64 --threads 4 --8x8dct --no-dct-decimate --output "test.mp4"> outputpass2.txt 2>&1

..\common\x264.exe "sotb-outdoorsample.avi" --stats "./sample.stats" --pass 3 --ssim --no-psy --bitrate 500 --min-keyint 60 --keyint 3000 --ref 16 --deblock -3:-3 --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree  --weightb --direct auto --subme 10 --trellis 2 --rc-lookahead 600 --partitions all --fullrange on --aq-mode 2 --me tesa --merange 64 --threads 4 --8x8dct --no-dct-decimate --output "test.mp4"> outputpass3.txt 2>&1
Produces: 
x264 [info]: SSIM Mean Y:0.9238759
Extra stats which I thought had some interesting information: 
x264 [info]: frame P:999   Avg QP:34.76  size:  1488
x264 [info]: frame B:559   Avg QP:38.60  size:   257
x264 [info]: consecutive B-frames: 33.2% 51.9% 14.6%  0.3%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 52.9% 47.1%  0.0%
x264 [info]: mb P  I16..4:  0.7%  0.2%  0.2%  P16..4: 32.4% 14.1% 13.9%  1.5%  0.9%    skip:36.1%
x264 [info]: mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 29.4%  3.7%  2.8%  direct: 2.4%  skip:61.6%  L0:38.4% L1:46.1% BI:15.6%
x264 [info]: 8x8 transform intra:26.6% inter:10.5%
x264 [info]: direct mvs  spatial:52.2% temporal:47.8%
x264 [info]: coded y,uvDC,uvAC intra: 16.0% 18.0% 11.4% inter: 11.1% 12.0% 8.9%
x264 [info]: i16 v,h,dc,p: 70% 23%  6%  1%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  5% 30% 56%  0%  1%  1%  1%  1%  5%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  9% 17% 28%  5%  8%  6%  7%  8% 14%
x264 [info]: ref P L0: 70.1% 10.2%  6.2%  2.7%  2.2%  1.9%  1.7%  0.8%  0.8%  0.6%  0.6%  0.6%  0.5%  0.4%  0.4%  0.2%
x264 [info]: ref B L0: 85.4%  6.2%  2.9%  1.3%  0.9%  0.8%  0.6%  0.3%  0.3%  0.3%  0.2%  0.2%  0.2%  0.2%  0.3%
x264 [info]: SSIM Mean Y:0.9238759
x264 [info]: kb/s:501.58
I can upload the mp4 if required. I'd also like to express my concerns with the rules; the option --no-psy should be banned as it's a feature that trades off visual quality for SSIM points. --no-psy is also part of the psnr and ssim tweaks so those would have to be banned also. Oh also: this could probably be beaten by changing threads to 1. Have fun with waiting an hour for 25 seconds of movie though :)
<adelikat> I am annoyed at my irc statements ending up in forums & sigs
Posted: 11/1/2009 10:47 PM
(Edited: 11/1/2009 10:59 PM)
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
I'd also like to express my concerns with the rules; the option --no-psy should be banned as it's a feature that trades off visual quality for SSIM points. --no-psy is also part of the psnr and ssim tweaks so those would have to be banned also.
Yep, this is probably what really made it jump over .92 easy. though in terms of visual quality, my eye can't really tell a difference with no psy =p. This as usual shows that SSIM can't measure perceived quality.
Oh also: this could probably be beaten by changing threads to 1. Have fun with waiting an hour for 25 seconds of movie though :)
I might be crazy enough to do that =D
Posted: 11/1/2009 10:51 PM
Post subject: SSIM: 0.9232806 /w multiple passes
Quote
Raiscan
Joined: 11/11/2006
Posts: 1235
Location: United Kingdom
Not if I beat you to it. ... TO THE ENCODING CAVE! Edit: Interesting result: 
x264 [info]: SSIM Mean Y:0.9232806
x264 [info]: kb/s:499.47
With one core, carrying on from previous encodes. As you can see the bitrate is also lower.
<adelikat> I am annoyed at my irc statements ending up in forums & sigs
Posted: 11/2/2009 3:46 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
Since I had the time, here is a comparison to crf mode. Note --no-psy is not used. 
--crf 30.85 --keyint 600 --min-keyint 1 --ref 16 --no-fast-pskip --bframes 16 --b-adapt 2 --direct auto --subme 10 --trellis 2 --partitions all --me tesa --merange 64 --threads 1 --thread-input --no-dct-decimate --ssim --fullrange on --rc-lookahead 250 --output "" ""
x264 [info]: SSIM Mean Y:0.9107112
x264 [info]: kb/s:500.83
Link to file: http://www.mediafire.com/download.php?yndnwmrzmjz