Post by feos

Posted: 8/2/2017 6:06 PM

Post subject: Best aspect ratio correction for HD

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

We know we need to do it via avisynth. But in what way? Here's a playlist with some methods, point x6 + lanczos ARC is what is mistakenly used right now, and we need to move away from that. https://www.youtube.com/playlist?list=PLR_HUaPVlquM-IsKOH4hhAr9_obTt_pIT So tell me, which one looks best at various resolutions? When I have time, I'll just download them all, but if someone can do it right now and post the pics, please do!

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 8/5/2017 9:35 AM
(Edited: 2/15/2024 9:36 PM)

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

Here's the full list of screenshots. Please tell which one looks best for each resolution (at 100% zoom). https://disk.yandex.ru/d/O9856Mrr-ByUKQ Below 4K they all look about the same to me. At 4K the best one is point8x+pointARC.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 8/5/2017 10:48 AM

Quote

fsvgm777

She/Her

Senior Publisher, Player (226)

Joined: 5/28/2009
Posts: 1215
Location: Luxembourg

feos wrote:

At 4K the best one is point8x+pointARC.

You sure? The left eye and the left tooth of the snake are a bit wider than the right ones... unless that's just my impression. That said, I'd like to see point8x+lanczosARC to compare how it fares at 4k.

Steam Community page - Bluesky profile Oh, I'm just a concerned observer.

Posted: 8/5/2017 10:52 AM

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

They are different size, otherwise there will be bleeding. Can make 8x+lanczos too.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 8/5/2017 3:36 PM
(Edited: 8/6/2017 11:57 AM)

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

https://www.youtube.com/watch?v=QXPJww0nwzs https://files.tasvideos.org/2095/ARC/point-1440x1080-point-4K.png

Language: avisynth
PointResize(1440, 1080)
PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")

https://www.youtube.com/watch?v=wB6E07OhzWM http://files.tasvideos.org/2095/ARC/point_straight_to_3584x2688_256x14_224x12.png Cropped to 256x224 manually

Language: avisynth
PointResize(3584, 2688) # (256*14 x 224*12)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")

https://www.youtube.com/watch?v=iHiwfa_0tH0 http://files.tasvideos.org/2095/ARC/mergechroma-point.png

Language: avisynth
AVISource("movie.avi")
chroma = PointResize(1440, 1080).PointResize(2880, 2160)
luma = PointResize(2880, 2160)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)
return last

https://www.youtube.com/watch?v=60jB3tchNuA http://files.tasvideos.org/2095/ARC/mergechroma-lanczos.png

Language: avisynth
AVISource("movie.avi")
chroma = LanczosResize(1440, 1080, taps=2).PointResize(2880, 2160)
luma = PointResize(2880, 2160)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)
return last

http://files.tasvideos.org/2095/ARC/nnedi3_rpow2.png

Language: avisynth
AVISource("movie.avi")
ConvertToRGB24()
nnedi3_rpow2(rfactor=2, nsize=0, nns=4, qual=2, cshift="lanczosresize", ep0=2, pscrn=4, fwidth=568, fheight=488)
AreaResize(360, 270)
PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")

http://files.tasvideos.org/2095/ARC/lanczos-360x270-point-4K.png

Language: avisynth
AVISource("movie.avi")
LanczosResize(360, 270).PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 8/6/2017 11:46 AM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

luma upscale 12x with point + downscale with lanczos chroma double upscale with point http://i.imgur.com/Nfwpulj.png

Language: avisynth
AVISource("movie.avi")
chroma = PointResize(1440, 1080).PointResize(2880, 2160)
luma = PointResize(3408, 2928).LanczosResize(2880, 2160, taps=2)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)

luma upscale 12x with point + downscale with lanczos chroma upscale 12x with point + downscale with lanczos to half target res + upscale with point (note: easier way to do this with taps=3 is to just subsample with lanczos though it be double lanczos downscale) http://i.imgur.com/uPJE6YE.png

Language: avisynth
AVISource("movie.avi")
chroma = PointResize(3408, 2928).LanczosResize(1440, 1080, taps=2).PointResize(2880, 2160)
luma = PointResize(3408, 2928).LanczosResize(2880, 2160, taps=2)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)

Posted: 8/6/2017 12:24 PM

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

https://files.tasvideos.org/2095/ARC/point8x-lanczosARC.png https://www.youtube.com/watch?v=EUCc-_T4UUc

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 8/6/2017 1:16 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

To me all it comes down to which tradeoff is not as bad. Point + Point is sharp but the pixel sizes are inconsistent. Point + Lanczos has consistent pixel sizes but is slightly blurry. Personally, I don't like the blocky look, but since we are stuck on that, I guess I would go with Point + Lanczos. Honestly, the best way to go with Point + Lanczos IMO is:

Language: avisynth
AVISource("movie.avi")
chroma = PointResize(3408, 2928).LanczosResize(1440, 1080, taps=2).PointResize(2880, 2160)
luma = PointResize(3408, 2928).LanczosResize(2880, 2160, taps=2)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)

Which is shown here: http://i.imgur.com/uPJE6YE.png But a simpler to implement way with not much difference is:

Language: avisynth
AVISource("movie.avi")
PointResize(3408, 2928).LanczosResize(2880, 2160, taps=2)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="lanczos") # Notice the change in chromaresample

Which is somewhat shown in here: https://files.tasvideos.org/2095/ARC/2160p_pointx12_lanczosdownscale_2880x2160_8,158MB.png (minus the correct subsampling) Either way is fine with me. A close second (third?) would be:

Language: avisynth
AVISource("movie.avi")
PointResize(1440, 1080)
PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="point")

Which is shown here: http://files.tasvideos.org/2095/ARC/point-1440x1080-point-4K.png Though I've noticed a slight shift in image due to the nature of point resizing twice.

Posted: 8/6/2017 1:49 PM

Quote

fsvgm777

She/Her

Senior Publisher, Player (226)

Joined: 5/28/2009
Posts: 1215
Location: Luxembourg

As it is, I'm torn between Lanczos downscale, MergeChroma-Point, Luma upscale 12x with point + downscale with Lanczos and PointResize to a factor of 8+Lanczos ARC.

Steam Community page - Bluesky profile Oh, I'm just a concerned observer.

Posted: 8/6/2017 2:06 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

Straight point with lanczos subsampling: http://i.imgur.com/Tc7Iwp8.png

Language: avisynth
AVISource("movie.avi")
PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="lanczos")

MergeChroma with straight point luma and more direct lanczos subsampling: http://i.imgur.com/2eyBWNF.png

Language: avisynth
AVISource("movie.avi")
chroma = PointResize(3408, 2928).LanczosResize(1440, 1080, taps=2).PointResize(2880, 2160)
luma = PointResize(2880, 2160)
luma = luma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
chroma = chroma.ConvertToYV24(matrix="Rec709", chromaresample="point").ConvertToYV12(matrix="Rec709", chromaresample="point")
MergeChroma(luma, chroma)

Posted: 12/26/2017 7:57 PM

Quote

fsvgm777

She/Her

Senior Publisher, Player (226)

Joined: 5/28/2009
Posts: 1215
Location: Luxembourg

Aktan wrote:

Straight point with lanczos subsampling: http://i.imgur.com/Tc7Iwp8.png

Language: avisynth
AVISource("movie.avi")
PointResize(2880, 2160)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="lanczos")

I seem to be experiencing inconsistent pixel sizes with this method, so now I have to resort to this method:

Aktan wrote:

Language: avisynth
AVISource("movie.avi")
PointResize(3408, 2928).LanczosResize(2880, 2160, taps=2)
ConvertToYV24(matrix="Rec709", chromaresample="point")
ConvertToYV12(matrix="Rec709", chromaresample="lanczos") # Notice the change in chromaresample

My script now looks like this (only showing the relevant parts) and looked like this before. Here's an image, for comparison's sake. Look closely around the EXIT sign. However, I should point out that the former method is a lot slower than the latter (it's encoding at 6 FPS at best).

Steam Community page - Bluesky profile Oh, I'm just a concerned observer.

Posted: 12/27/2017 5:12 AM
(Edited: 12/28/2017 1:24 AM)

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

The inconsistencies are caused by fact the scaling factor is non-integer in most cases (as evidenced by the fact that an integer-based resize fixes this in your new script). It could probably be sped up by making the script work out the smallest integer the original clip needs to be multiplied by in order for both the width and height to be larger than required, and then using that for the resize. Another problem with resizing directly to 4K is that Youtube is using this clip for all the resolutions. Whilst we've controlled as much as we can (knowing it's going to be re-processed) in relation to the initial 4K stream, Youtube is going to resize the video we've given it prior to re-encoding for the other resolutions. This means that while any artifacts introduced (such as any chroma bleeding as a result of the subsampling) will be encoded as they are presented for 4K; lower resolutions have the potential to look worse as they'll be basing their calculations off a file that already has some percentage of error. What happens if we give Youtube a clip that is sampled at 4:4:4, and not at a preset resolution? If we resize a 256x224 video clip by width x 14 and height x 12, we get 3584x2688 - a resolution that can be point sampled via integers, is of the correct aspect ratio, is beyond 4K, and could be a reference for all of the destination resolutions Youtube provides without having any point of error to begin with.

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 12/28/2017 12:54 AM
(Edited: 12/28/2017 1:27 AM)

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

If we send 4:4:4, YouTube would resample to 4:2:0 for us. That gives us less control for the max res encode on chroma. I can see your argument that it will give a better source to do the resize and resample for the other resolutions, but the chroma bleeding at max res is minimal and it should be fine to resize off of that. For example, a max res of 2880x2160 would have chroma res be 1440x1080. Resizing to 1080p, the luma would resize to 1440x1080 and the chroma will be resized to 720x540. Basically 1/4th for both luma and chroma. If the file was 4:4:4, the resize would be from 2880x2160 to 720x540 for chroma. 1/8th the size. I don't think there be much difference. Bottom line is, I don't think there be any potential to look worse as the resize ratio using a 4:2:0 source would always be the same. Problem with sending 3584x2688 is that we know YT will resize it to it's standard resolutions and I rather have more control. Plus the fact of straight resize to 4K is a bit sharper (at the cost of inconsistent pixel sizes). If you want, you can do a comparison. We did send that res in the past, and I thought it was the best until I realized we have less control.

Posted: 12/28/2017 1:14 AM

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

Ah, that makes sense. I didn't stop to think of what it would be like in comparison to the current method and when you put it like that, the control we currently have definitely outweighs my suggestion (honestly, there's a chance my idea may even look worse depending on their handling).

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 12/28/2017 1:17 AM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

I would still say test, if you got time, as maybe YouTube does better than we think.

Posted: 12/28/2017 1:42 AM

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

No problem. Here are three tests: Point resized directly to 4K Point resized by integers Point resized up, Lanczos resized down Each video was encoded at QP0 to avoid inconsistencies, further information can be found in the description of each video. Here are the source videos for anyone interested and here is a still image comparison ~~I'm yet to draw any conclusions myself as Youtube is still processing these videos at the time of this post, and 4K is not yet available.~~ 4K's now available, so let's do this: It looks like the Lanczos method and integer method give very similar results. There are noticeable differences (see the green pixels above Mario's head), but for the most part they're near identical, leading me to think that perhaps the integer method may be superior in comparison (as it yields a much faster encode and a much smaller file size). It's also worth noting that while the integer & lanczos methods give more accurate pixel sizes and locations, the pixel sizes are still not consistent across the frame (although this is to a significantly smaller degree that when resizing directly to 4K).

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 12/28/2017 1:29 PM

Quote

feos

Site Admin, Skilled player (1255)

Joined: 4/17/2010
Posts: 11486
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg

I always prefer sharpness over absolute pixel consistency, unless it results in worse look at anything lower than 4K. No one is zooming 4K as much as we do for comparisons, and it's impossible to see isolated pixels being different. Does the whole picture look different or worse at 4K? Even if one has a 4K monitor and watches such vids fullscreen, there's still little point in watching them that close to the monitor so you start seeing pixel inconsistency. You're either moving farther from the screen, or you're shrinking the resolution. I think all the previous tests show that at 720p everything looks equally shitty. And imo, at 4K everything looks equally good. The question is, is one of the methods better when you actually watch the TAS instead of zooming 4 times to stare at gigantic static pixels?

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 12/28/2017 1:34 PM

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

Realistically, the hard cap at 60fps is going to drop frames in many situations, so the Youtube encodes already have that as an element of inaccuracy. I always see the on-site encodes (and potentially extra hq encodes) as available to fulfill the accuracy requirement and the Youtube / streaming media encodes available more for casual viewing. The inconsistency is something I'd never noticed on playback (even though I had suspected as much, I'd never investigated) and so in that case,for streaming media, if I had the choice between the two I'd probably end up watching the one with the sharper output.

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 8/4/2018 11:03 AM

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

I've been thinking about how to potentially improve this for a while and it occurred to me: when youtube moved to 60fps, some of the older videos that were uploaded at 60fps (but which displayed at the previous limit of 30fps) were then re-encoded by Youtube and presented at 60fps. A similar thing happened when Youtube switched to the DASH formats, with videos being re-encoded to meet their new standards. There are plenty of other examples exhibiting this same pattern, showing that when Youtube update or change their own formats, they make new copies from the uploaded source video. This made me think about how we handle Youtube encodes, specifically our resizing methods and our frame limiting. Regarding resizing methods: We currently employ a direct resize to 2160p. This has an advantage of being sharper than the alternatives and allows us to directly control how the video will be presented at that specific resolution. It's important to note that all the other resolutions are going to be derived from this same source clip, although the differences are probably insignificant at those resolutions. The pixels sizes are also inconsistent when using this method. The alternative of point-resizing up by integers and then lanczos resizing down to the destination resolution of 2160p has also been mentioned. It fixes the issue of pixel sizes, but at a compromise of sharpness. The encoding times and file sizes are also significantly worsened. My thoughts at the moment are: why are we not just integer resizing to beyond the resolution? To be clear, I'm talking about taking a 256x224 clip as an resizing it by factors of 14 and 12, to 3854x2688. The first consideration was the old case of Youtube not generating 60fps encodes for non-standard resolutions. I did a test upload to see and 2000x1220 generated 1080p60, so this issue appears to have been resolved. The other consideration is quality, as we will have less control of how the final video will look. Having said that, currently we let Youtube decide how resolutions 1440p and below are going to look anyway. Additionally, going back to my first point, when Youtube change their formats they make new copies from the uploaded source. This makes me think that we should be focusing on giving them the best quality input video we can, rather than trying to manipulate the output (we already give them QP 5 encodes for this reason). If we give them an integer point-sized encode, it will be sharp, have correct pixel sizes and encode faster than lanczos resizing that down to 2160p would after the fact, and if they ever decide to allow exact-resolution playback (or even some other new situation, who knows) we won't have to reupload anything. Before moving on, it's worth noting that there will be some uncommon resolutions requiring ridiculous integers to achieve the correct aspect ratio. In these cases, point-resizing up and lanczos resizing down would be impractical as well. Regarding frame limiting: Before I start, this isn't nearly as relevant or important as the resolution topic above. This is just something to round off that discussion and leave it more complete. Currently we perform two Youtube frame rate limitations: a 60fps cap, and half-fps for games that lag every other frame. The 60fps limit is done in AVISynth via ChangeFPS. As far as I know, this just discards however many frames to ensure the rate specified is achieved. I don't see Youtube doing anything vastly different from this and we're already losing frames from this process. Is there any harm in uploading directly at 70.086~ for DOS? Youtube will just throw away frames the same as ChangeFPS. The advantage here would be that our input files would now be of the game's correct rate (which combined with QP 0 and integer point resizing would mean an entirely lossless input file would be ready for re-encoding should something change). We'll have less control over it again, but do we need control over this? That being said, I have more hope Youtube will support specific resolutions at some point than unique frame rates and so a change here probably isn't necessary. That's probably enough typing for now. There's probably a few typos and a dozen things I've overlooked, but right now I think resizing by integers is the strongest of the options. I'm interested in what everyone else thinks!

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 8/4/2018 10:12 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

The problem with your suggestion is that there is no guarantee that YouTube will indeed re-encode all the source files nor keep all those files. Imagine if YouTube has to do that every time they change. That doesn't mean just our files, but all files sent to YT. I believe what you saw was YouTube only re-encoding videos uploaded after a certain point, but not all videos.

Posted: 8/5/2018 3:04 AM

Quote

TheCoreyBurton

He/Him

Joined: 10/14/2013
Posts: 335
Location: Australia

Aktan wrote:

I believe what you saw was YouTube only re-encoding videos uploaded after a certain point, but not all videos.

You're probably right. I was basing my assumption that it was all videos on the combination of the information I mentioned above and that at one point, Youtube introduced a method to download your own input videos (as I had to do at one point). I tried this again now and was only able to download the video in 720p30 (despite it being available in 1080p60), which leads me to believe this feature has been replaced with a new method that allows you to download the best quality combined stream Youtube has.

I'm not as active as I once was, but I can be reached here if I should be needed.

Posted: 8/5/2018 4:21 PM

Quote