Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
It has come to my attention that publishers have been mishandling the encodes used for Archive.org playback.
Since we figured out what we were doing a couple of years back, there have been two important encodes for Archive.org. The first is our primary downloadable encode, which uses the best technology and aims for the smallest file size. That currently means 10-bit H.264 yuv444 with opus in an MKV. The other (the "512kb" encode) is meant for playback via our embedded archive.org player (or HTML5 player once I get it some new tweaks for the site), and must be in 8-bit H.264 yuv420 MP3/AAC in MP4. This latter format is also meant to be highly compatible with various players people hook into our site, or for those with really aging software on their computer.
While doing a status check to see about improving playback for the site, I noticed that many of our publishers are now sometimes uploading some kind of MKV for the 512kb encode, and allowing Archive.org to derive their own MP4 version from it. This is utterly unacceptable. As an example check out the current archive.org on site playback for [3050] NES Super Mario Bros. 3 "arbitrary code execution" by Lord_Tom in 08:16.23, the encode played back in the browser is absolute trash. For a top-rated newcomer-recommended movie, we have seriously failed here.
If encoders/publishers want to make their own MKV for the 512kb too for various reasons, that's fine, but they must ensure to provide their own MP4 of it and not allow Archive.org to create an MP4 itself.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Nach wrote:
If encoders/publishers want to make their own MKV for the 512kb too for various reasons, that's fine, but they must ensure to provide their own MP4 of it and not allow Archive.org to create an MP4 itself.
Right, and then I once again don't see the benefit of even having 512kb.mkv alongside 512kb.mp4. The initial goal was to have a 512kb file that has proper subtitles, but then we end up with mp4 file with bad subtitles anyway, wether we make it or Archive does.
So I think the "least bad" solution would be to drop subtitles from mp4s if they don't look decent right away, and have only 2 traditional downloadables again.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Right, and then I once again don't see the benefit of even having 512kb.mkv alongside 512kb.mp4.
I don't see necessarily the benefit either, but if some encoders want to do both, that is their prerogative.
feos wrote:
The initial goal was to have a 512kb file that has proper subtitles, but then we end up with mp4 file with bad subtitles anyway, wether we make it or Archive does.
Incorrect, the "initial goal" of 512kb is to offer a streaming MP4 which can be played back directly on the site. If archive makes it, the file is huge and looks terrible. This is unacceptable.
feos wrote:
So I think the "least bad" solution would be to drop subtitles from mp4s if they don't look decent right away, and have only 2 traditional downloadables again.
I don't know what this means or how it's relavent to encoders/publishers incorrectly allowing Archive.org to derive its own MP4s for 512kb.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Nach wrote:
feos wrote:
The initial goal was to have a 512kb file that has proper subtitles, but then we end up with mp4 file with bad subtitles anyway, wether we make it or Archive does.
Incorrect, the "initial goal" of 512kb is to offer a streaming MP4 which can be played back directly on the site. If archive makes it, the file is huge and looks terrible. This is unacceptable.
I meant the goal of making 512kb.mkv in the first place. They were made because subtitles suck in mp4.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Nach wrote:
feos wrote:
The initial goal was to have a 512kb file that has proper subtitles, but then we end up with mp4 file with bad subtitles anyway, wether we make it or Archive does.
Incorrect, the "initial goal" of 512kb is to offer a streaming MP4 which can be played back directly on the site. If archive makes it, the file is huge and looks terrible. This is unacceptable.
I meant the goal of making 512kb.mkv in the first place. They were made because subtitles suck in mp4.
I'm not familiar with what the support is like for subtitles embedded in MP4 files. If encoders/publishers want to offer 512kb MKVs for embedded subtitles, that's fine. However, this does not free the encoder/publisher up from the requirement for ensuring we have a good 512kb MP4 for site playback. Regarding subtitles with site playback, the site can handle it fine if you upload an SRT to archive.org for the 512kb. [1443] NES Metroid "low%" by Lord_Tom in 11:08.78 is an example of a publication which has done this properly.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I personally was fine with it mp4 subs, but in some games you want special placement for them, otherwise they may cover all the ingame text. Standalone players can override subtitle placement, but for streams there's little to no point in such subs if they cover important gameplay elements. In mp4 you can't change their placement or embed any other subtitle format. I think it was Spike's idea to use mkv for 512kb.
And as I said, if 512kb encode is anyway done as mp4, there's no point in having it as mkv as well. For proper subtitles one needs mkv, and I see no benefit in having 2 mkvs. So I'd only leave primary mkv, and if subtitles don't look right in a given game using mp4+srt, then only mkv should have the subtitles.
See [2719] Genesis Langrisser II "all levels" by ars4326 in 1:56:46.59 as a great example.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 10/12/2011
Posts: 6449
Location: The land down under.
Screw it stepping in for a second, going to talk about, well, 10bit444.mkv and a question which never got answered by feos.
feos wrote:
So I'd only leave primary mkv
feos I told you straight up in a private message that there was an issue with .ass (or .ssa "SubStation Alpha") styling for playing on 10bit444, you didn't even provide a fix to solve or overcome this issue. I told you this in a message, when you were asking why am I doing 512kb.mkv, since I was asking for a 512kb.mkv to be placed up since Publishers can only do 1 MKV file no matter how the format is. But you didn't provide a solution to this issue, so that's the main reason why I place .ass into 512kb downloadables.
I'll explain the issue for those at home.
Say in the subtitles, the author put information which occurs during a still loading screen. So the loading screen would have a still graphic, or is just generic black.
Well, in 512kb.mkv the (.ass) subtitles show up during that, but what about 10bit? Well, the subtitles fail with the settings we currently have in the guide. The subtitles will either not appear depending on how long the author has placed it in for. Or show up for a split second after the loading and disappear.
So feos, how do you fix it? Actually I'll make it an open question.
So using x264 (within the batch) how do we fix the 10bit444 to prevent it from missing subtitles or chopping them up, and without making it the size of the 512kb?
Nach mentioned that there will be a switch on the downloadable links to archive, but not the torrents, we'll still provide the .ass styling on the 512kb, as it performs better and doesn't have this issue.
I will point out, I have to redo 5 512kb encodes, without the styling subs, only with generic .srt subs, but this will be for archive only specifically for streaming on browser. The downlaodables will still be both MKV, as they're both a file which is being downloaded.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account.Something better for yourself and also others.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Thanks for blaming me a hundred times in a row for not making nonexistent (dedupped) frames be considered in a format that I didn't design. Nice ignorance, or just stupidity?
Maybe you didn't get it back then, but timestamps in the subtitles no longer match the timestamps of a dedupped encode, it seems mkv just can't spawn a subtitle in the middle of a frame, that it makes last as long as its timecodes tell it to.
As for distinguishing between 512kb downloadable and 512kb stream, good point.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11495
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
BTW, the solution to this was to simply include the extra subtitles into the encode at the dup detection step.
Language: avisynth
pass == 1 ? assrender("subs.ass") : 0
Then every change in the subtitles will have its undedupped frame as an anchor, preserving the timing.
Font style and positioning is still only editable through the player for mp4.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
