So apparently there's not much discussion going on surrounding this movie and the policy it might affect. So I'll just summarize my thoughts and see if there is disagreement. I'll also use this post to gather opinions of people I'll be asking.
Two fundamental entities tasvideos has always cared about the most are optimality and entertainment
Each of them can be
measured and
compared. We host TAS movies, and these two entities are qualities of a TAS, so by measuring and comparing them, we compare TAS movies, that's how we decide on obsoletion.
Obsoletion is an important part of hosting TAS movies
All the time optimality and entertainment get improved, and
we only want to primarily showcase the most optimal and the most entertaining movies. Because this is where we set the quality bar, the level of TAS craft. By keeping it high enough, we encourage people to strive for it and improve the overall quality of our content, as well as their skills.
Emulator accuracy is not a quality of a movie
You can't TAS emulation accuracy, you can't make it more entertaining, more optimal.
You can't objectively measure it either! Because what appears as emulation accuracy might in fact be a coincidence. One could argue that test ROMs are our way to assess accuracy, and we can measure how many tests an emulator passes. While this is true, and eventually the emulator core gets closer to the real CPU in logical behavior, there are a few problems.
Consoles may be different in hardware behavior
Talk about known knowns, known unknowns, and unknown unknowns. Test ROMs that evaluate an aspect of some set of machines, might completely miss this same aspect of another set of machines, for whom it works differently. And there can be infinite amount of such different aspects we don't know about. So it might look as being accurately emulated in general, but it only considers a fraction of all the real machines.
There's always human factor to test ROMs
They can simply miss an aspect a real machine has, resulting in all sorts of behavior mismatch, discovered or undiscovered.
https://en.wikipedia.org/wiki/Heisenbug#Related_terms
Testing emulation accuracy is actually very hard
It takes a person with a real machine, a person writing a test ROM, a person developing and setting up a replay device if we're talking about verifications. It's also barely possible to verify, because all those factors use to only work together for volunteers, and we can't guarantee that some of them isn't lying or mistaken.
Setting up an emulator and running a movie is trivial and reliable
Sometimes emulators aren't deterministic enough to work the same for everyone, and some emulators are just hard to set up. But effectively
almost anyone can replay a movie without fiddling with any hardware at all, by simply following known software steps. And most importantly, we don't need too much coverage here, unlike with testing emulator accuracy: we only need to confirm sync a few times, and we approve.
Accuracy is still an important factor for movies
There are cases when a trick used is outright impossible on the real hardware. This matter is a bit of gray area, because there's no strict borderline to what measure of inaccuracy we allow. In general we just require that a bug works on console in principle, instead of completely relying on emulation bugs, being impossible on console in principle.
In such cases, it indeed makes perfect sense to obsolete an old movie: if the trick used was found to be impossible without emulation bugs. But if all we're talking about is just a bit more accurate general event, there's no certainty.
Imagine there's a list of 10 possible drops from a certain enemy type. The old movie gets drop #1 just because randomness aligned that way. Such a drop is legit and possible. A new movie, done on a more accurate emulator, gets drop #6 there. Also legit and possible.
If optimization and entertainment are the same, we wouldn't consider this an improvement
But what if it also syncs on console? See above about how coincidental console sync can be. This is an achievement, and it's directly related to TASing.
But it's not a TASing achievement per se. Strictly speaking, one can theoretically just resync an old movie without improving anything at all. It'd be more accurate to the console. But it's not a
TAS improvement.
Considering the above, my opinion is that
treating non-TAS improvements as TAS improvements in terms of Movie Rules and judgment is moot. It relies on unreliable things and opens up a can of exception worms, where exceptions are allowed despite of unreliability, while in other unreliable areas they aren't allowed.
Solution: we can use Wiki in all sorts of ways to document emulator accuracy and how relevant movies are.
Thoughts?
