There is no way to objectively gauge technical rating except for (rarely) one or two people per game, and even then it can be based on incorrect ideas or complete guesses.
After all, if you objectively rate a movie anything under 10, you should be announcing why the movie could be done faster, or even just making a new version yourself immediately.
Exactly. That's what I've been trying to say.
Xkeeper wrote:
Hell, let's remove the whole idea of objectivism by saying that if it was an objective matter, there wouldn't be an option to rate it in the first place. After all, objective would mean that it would be based on something other than an individual viewer's knowledge and ideals, so it might as well be calculated by a robot.
You got it all right.
The big mistake here is to believe that this "how close to perfection" thing can be subjective. But I've been told all people here love these messages: "I think you could maybe shave a frame somewhere, so I will give you only a 9 in tech", so I don't care anymore. I'll try to like those messages too from now on.
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
After all, objective would mean that it would be based on something other than an individual viewer's knowledge and ideals, so it might as well be calculated by a robot.
And robots have their references programmed by… right. Nowhere to run, huh? :)
I don't understand why we should even try to be objective where it's not only unneeded, but also impossible. Similarly, the votes aren't getting any more objective due of their amount, and thus we shouldn't treat them as such in the first place as well.
mz wrote:
But I've been told all people here love these messages: "I think you could maybe shave a frame somewhere, so I will give you only a 9 in tech", so I don't care anymore. I'll try to like those messages too from now on.
Mind if I ask for examples?
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
That was only an exaggerated made-up example; I thought you may know what I meant...
For example, these kind of recent messages:
From here.
stickyman05 wrote:
Don't take the technical vote wrong, I think it looks great. However, considering this is the first possibly published TAS of this game, I feel there is more to improve (such as a glitch or something)
"I will rate lowly this movie because maybe, who knows, there is a glitch to be found..."
And, of course, judges always write them, as seen here, for is their job to do this (but I'm only complaining against using a totally wrong number to indicate if a movie is good or not).
mmbossman wrote:
I'll give it a 4 (tech) /7 (entertainment), but because I only have theories about improvements
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
Don't take the technical vote wrong, I think it looks great. However, considering this is the first possibly published TAS of this game, I feel there is more to improve (such as a glitch or something)
"I will rate lowly this movie because maybe, who knows, there is a glitch to be found..."
While I won't discuss stickyman's reasoning for the vote, technically, he is correct. First generation TASes of even remotely complex games have always, always been improvable because it's quite practically impossible to find everything that will be useful the first time around. Whether that should be taken into account when voting, I'll leave that on his own conscience.
mz wrote:
mmbossman wrote:
I'll give it a 4 (tech) /7 (entertainment), but because I only have theories about improvements
The full quote was as such (emphasis mine):
mmbossman wrote:
I'll give it a 4 (tech) /7 (entertainment), but because I only have theories about improvements, I don't see any reason to not accept this for publication.
I should also point out that mmbossman's qualms were quite valid. Don't forget that judges are judges because they are supposed to be able to appraise the quality of optimization without having to check everything for themselves (although that's always welcome).
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
While I won't discuss stickyman's reasoning for the vote, technically, he is correct. First generation TASes of even remotely complex games have always, always been improvable because it's quite practically impossible to find everything that will be useful the first time around.
I know that. I only don't like the part where movies have this based-on-assumption number in their tables telling new viewers how good that movie is.
moozooh wrote:
I should also point out that mmbossman's qualms were quite valid. Don't forget that judges are judges because they are supposed to be able to appraise the quality of optimization without having to check everything for themselves (although that's always welcome).
I know, and I like that. What I don't like is having that hypothetic number there telling other people if a movie is good or not.
I repeat: I'm against having a subjective number rating something that's not subjective. You can't really tell how perfect a movie is, you can only assume. That number is only useful to misguide new viewers.
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
The thing is, I apparently don't rate like everyone else does here. I don't feel that a 10 is a yes vote, 5 a meh vote, etc. I feel 10 is a perfect movie, 5 is acceptable, and 1 is get this shit outta here.
I don't see how people can vote 10s on technical, when in some instances (See my old movie rating for mario 64) a month later the game is 10 seconds faster, and more frame perfect. Yes, an extreme example, but that is how I view it... and I vote accordingly.
On another note, I make sure the judges, and authors, know the reason for my lower vote numbers. For ferret, I PM'd him, to let him know what my reasoning was even further, and to make sure he was OK with it.
If any player has a problem with my voting, I encourage them to let me know so I can explain how I voted. I do not appreciate being brought into the conversation without a direct question as to why I voted that way.
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
I don't see how people can vote 10s on technical, when in some instances (See my old movie rating for mario 64) a month later the game is 10 seconds faster, and more frame perfect.
So why do we even have the option to vote 10 if we must not do that? Everything on the tech quality rating as we use it is flawed...
stickyman05 wrote:
I do not appreciate being brought into the conversation without a direct question as to why I voted that way.
Sorry about quoting you, it was the first example I found. :P
I know we've all voted that way sometime, even I did. I just don't feel like it's right to have this rating anymore...
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
Joined: 10/27/2004
Posts: 1978
Location: Making an escape
I'm going to back up sticky here by saying that in no way did I feel slighted by his rating. He said he enjoyed watching it, so why should I give a carp about some silly little numbers? As I've pretty much said earlier this topic, posts trump numbers. Always.
Also, I'm going to put on the appearance of back-peddling by saying that I'm actually impressed by how little this place has changed. Remember, I did church service for two years that kept me, for the most part, off the internet (I was allowed to do limited e-mail once a week). Many places on the internet have changed quite drastically during the interim, yet I come here to find that the name was changed from NESVideos to TASVideos, the forum index was changed to double column, the gruefood forum was created, and variable avatars were introduced (which thing I have obviously taken to perhaps too well), but other than that, this place was very familiar. My own house changed more than this place.
Heck, most of the changes have occurred after my return. The recommended list was removed (which, apparently, is making a comeback, yay!) and the voting system was overhauled, which has left me unaffected for reasons previously stated.
Some people may mention a greater idolization of "perfection" and more drama, but when I think about it, neither of those have changed. I remember being somewhat disturbed by the intense focus on perfecting movies before I left, and I haven't seen much of an increase in that focus. I also remember being annoyed by the rampant anger around here... and people are still as angry as before! So yes, I think these problems have always been around.
About the only change that I either haven't come to like or been apathetic towards is the greater insistence on using more and more advanced tools to create these movies. Before I left, it seemed memory watching was rarely used; now it's a standard. Also sneaking in are LUA scripting and dissembling. And the only reason why this leaves me concerned is because it would seem to make it harder for newcomers to come in and show their hands, but since I'm not a newcomer in these times, I lack that perspective and could easily be completely off base here.
So, in all, thank you everyone for being so stable! At least I still know my way around here.
A hundred years from now, they will gaze upon my work and marvel at my skills but never know my name. And that will be good enough for me.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
mz wrote:
And, of course, judges always write them, as seen here, for is their job to do this (but I'm only complaining against using a totally wrong number to indicate if a movie is good or not).
mmbossman wrote:
I'll give it a 4 (tech) /7 (entertainment), but because I only have theories about improvements
I guess since I got somewhat called out, I'll respond.
Yes, tech rating is subjective. So is entertainment. And they're both there because they're a somewhat interesting and fun addition to the website to make it more interesting and engaging. I understand that you don't like it. But, you also can't convince everyone that it isn't wanted as a feature, because I personally enjoy rating movies, and seeing how others rate mine. And I also realize that people view things differently, and will probably not rate exactly as I do. Zurreco, for example, feels that everyone's ratings should always average out to the median (5), and while I don't agree with him, I can see his point. But I also realize that I won't change his idea of what a technical (or entertainment) rating should be.
In the comment in question, I took into account several things, most of which I laid out point by point. Yes, I can't prove that my theories are valid without actively TASing that particular game. But I've been around for over two years, have watched over a thousand movies and WIPs, and have produced over 15 finished TASes of my own. My experience is one of the reasons I was made a judge. It is also one of the reasons that I can assume that many of the strategies that I have used to save time in TASes can also be applied to other runs.
I also take into account author when I think about technical quality. If the above mentioned run had been produced by a member of the site with several published movies who had consistently produced high quality work, I would assume that the same level of care and precision had been used to create that run. In this case, however, I knew that max had produced and submitted several runs that were of very poor quality, which I took into account when I gave it a rank. However, I also know that he is improving as a TASer, which is why I accepted that run, and am not above changing my rating if the improvements I suggested are shown to be incorrect.
Some people will probably read this and piss and moan about bias and whatnot. But I'll restate what I said in the beginning: TASes, and the ratings associated with them on this site, are for fun. So there's no reason to take them so seriously.
I repeat: I'm against having a subjective number rating something that's not subjective.
You keep insisting that "technical rating" cannot be subjective. Who says that? I don't say that. I don't know anyone else saying that. Well, except you.
You say that newcomers will come to the site and see "technical quality" and immediately jump to the conclusion that this must be a mathematical formula with one single correct answer, and that the number represents this correct answer, and because the number is really not the mathematically correct answer, this misleads the newcomers. This is the most ridiculous thing I have heard in a long time.
What newcomers see when they watch the info for a movie is a number titled "rating", which additionally tells how many people have voted. They don't even see a "technical quality" anywhere, unless they go to the rating details page. And even there it says nothing about "perfection" or anything of the sorts. Rating, votes... how the heck could they get the impression that this number is somehow an objective absolute measure of perfection? You are seeing something which simply is not there.
I repeat: If you don't like the technical quality rating, just ignore it. It's that simple.
About the forum especifically, the rating system sounds great but maybe it's overly complicated for something that's very subjective. I think the "technical rating" should be removed altogether and unify it into 1 rating system: "what did you think of this movie?" going from "boring" to "awesome!". It's (should be) all about entertaining, right? Just my suggestion.
The only problem with that is that people will abuse that feature to fit their own interest even more so than the previous yes/no/meh vote system. Also having a technical rating makes it easier to determine which obscure games need to be updated in order to have a run that meets current site standards.
You keep insisting that "technical rating" cannot be subjective. Who says that? I don't say that. I don't know anyone else saying that. Well, except you.
No one says that. That's the problem. You're all very convinced that "how close to perfection" is only subjective, that every people have a different sense of perfection. Like, I say the perfect time for Super Mario is 4:00 and Bisqwit says it's 3:00, and we both are right.
Warp wrote:
What newcomers see when they watch the info for a movie is a number titled "rating", which additionally tells how many people have voted. They don't even see a "technical quality" anywhere, unless they go to the rating details page. And even there it says nothing about "perfection" or anything of the sorts. Rating, votes... how the heck could they get the impression that this number is somehow an objective absolute measure of perfection? You are seeing something which simply is not there.
That's exactly the problem. They go the NES page and see this number and think "oh, cool, this Dragon's Lair movie must be more entertaining than Gone with the Wind". Little do they know that most people have given high ratings to that boring game only because they assumed the movie was almost frame-perfect.
Warp wrote:
I repeat: If you don't like the technical quality rating, just ignore it. It's that simple.
I repeat: that doesn't solve my problem, at all. :)
I understand that what I want will never happen anyway, because you've all been here since a long time ago, and probably you all love the old "tech quality" rating and would miss it. Also, as mmbossman pointed out, this is just for fun; I shouldn't take these wasted hours of work seriously anymore.
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
You keep using that expression over and over. Just drop it. It doesn't exist anymore anywhere. You seem to have some kind of compulsive obsession about it.
That's exactly the problem. They go the NES page and see this number and think "oh, cool, this Dragon's Lair movie must be more entertaining than Gone with the Wind". Little do they know that most people have given high ratings to that boring game only because they assumed the movie was almost frame-perfect.
By the same logic the entertainment rating should be dropped as well, because, after all, it makes two thirds of the average score visitors see in the movie info page.
Why visitors would interpret some semi-hidden (as in: it doesn't show explicitly in the movie info page) "technical quality rating" differently from the equally semi-hidden "entertainment rating" is beyond me.
You keep using that expression over and over. Just drop it. It doesn't exist anymore anywhere. You seem to have some kind of compulsive obsession about it.
I know; I don't use it myself since some time ago. You should now start telling the other 2242 users to stop using that system, starting from our judges.
Warp wrote:
By the same logic the entertainment rating should be dropped as well, because, after all, it makes two thirds of the average score visitors see in the movie info page.
Why visitors would interpret some semi-hidden (as in: it doesn't show explicitly in the movie info page) "technical quality rating" differently from the equally semi-hidden "entertainment rating" is beyond me.
The "entertainment rating" is accurate, the "technical quality rating" is not. That's what I've been trying to tell you from the beginning.
I don't want to keep this discussion even longer. So, thanks for reading my concerns and giving me your opinions about them. I'll see what can I do with all this.
You're just fucking stupid, everyone hates you, sorry to tell you the truth. no one likes you, you're someone pretentious and TASes only to be on speed game, but don't have any hope, you won't get there.
It's not beyond me. A newcomer to the site is looking for entertaining runs, and could care less about technical perfection except insomuch as it affects how entertaining the video is. It's reasonable to assume that something that most people find entertaining will also be entertaining to a newcomer; yes, there are specific exceptions, and yes, because we're experts on the subject, our tastes are different, but there is at least a correlation.
But then you throw that "technical quality" into the mix, and suddenly you have a number that bears much less correlation to entertainment. You can vary a movie's overall rating by 3 points just because a terrible game was TASed with a high degree of precision or a highly entertaining game was sloppily-run. This makes it nigh-impossible for a newcomer to use the ratings to find entertaining runs.
I'd be fine with splitting out the listed ratings on the movie display so that technical quality is distinct from entertainment.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
mz wrote:
Warp wrote:
You keep using that expression over and over. Just drop it. It doesn't exist anymore anywhere. You seem to have some kind of compulsive obsession about it.
I know; I don't use it myself since some time ago. You should now start telling the other 2242 users to stop using that system, starting from our judges.
Since you seem to keep singling us out, I'll just keep responding.
I do tend to think of a tech rating of 10 as being a rating that conveys frame perfect-ness. I do believe that there is one theoretical fastest time for each game, and that a perfectly done run would reach that time. However, when I give a tech score, that score is just based on my judgment, in exactly the same way entertainment is. Am I positive my scores will be exactly perfect if you compare them to the gold standard of a theoretically perfect run? Of course not. I don't have all the information. Neither does anyone else in this universe, and no one will ever have all the information until each combination of button presses is tried (the famous brute-force problem). So when you mention that someone may think that 3:00 is the theoretically fastest time to complete SMB, and you happen to think it's 4:00, neither of you are right because the true answer is likely to never be known. Those figures are based on your judgment, into which many factors (such as experience, perception of technique, perception of author, etc.) go. So you're right. I do think of my tech ratings in terms of how close a movie is to perfection. But since neither you nor I will ever know that perfect value, you have no right to assume that my judgments are any better or worse than yours. And I believe that is very close to the meaning of subjective.
You must surely concede that the technical rating has no inherent bearing on entertainment for some subset of the population. And I argue that newbies have less interest in technical perfection than veterans do, especially given the already high technical bar that must be passed for a video to get accepted at all. So if we accept that the ratings should be useful to newbies, and we accept that newbies in general are uninterested in technical perfection, then the ratings should not take technical perfection into account (or at the very least should split it out).
I'm not trying to take your interest in technical perfection away from you. I'm just saying that you have a perspective that is not shared by newcomers to the art of TASing.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
You must surely concede that the technical rating has no inherent bearing on entertainment for some subset of the population.
And tool-assistance is seen as cheating and a hoax for some subset of the population. So what? Should we remove everything that somebody might perhaps not like?
On a similar note: If you have two movies, one rated 3 for technical quality and another rated 9.8, which one do you think would more probably be more entertaining? (And please don't nitpick. Individual exceptions don't affect the answer to a question about what's the most probable situation.)
And I argue that newbies have less interest in technical perfection than veterans do, especially given the already high technical bar that must be passed for a video to get accepted at all.
You assume that the technical rating is about perfection. It isn't.
So if we accept that the ratings should be useful to newbies, and we accept that newbies in general are uninterested in technical perfection, then the ratings should not take technical perfection into account (or at the very least should split it out).
I for one don't accept the idea that newbies couldn't care less about the technical quality of tool-assisted speedruns. If anything, a low tech score would be indicative of sloppy play, which is not something you would expect in a tool-assisted run (regardless of whether you are a veteran or a noob).
I'm not trying to take your interest in technical perfection away from you. I'm just saying that you have a perspective that is not shared by newcomers to the art of TASing.
How do you know newcomers are not interested in the technical aspects of tool-assistance?
After all, objective would mean that it would be based on something other than an individual viewer's knowledge and ideals, so it might as well be calculated by a robot.
And robots have their references programmed by… right. Nowhere to run, huh? :)
No; it would simply brute-force the game to get to the ending fastest, then assign a technical rating based on how many frames behind the human run is.
Then it would simply not tell anybody where it was :)
Bisqwit wrote:
Actually, my first intention was precisely to have this as a "machinima" website.
In case I guessed the meaning of that weird word wrong, I mean that I originally intended the site to be a collection of all kinds of weird gameplay videos that have been created by utilizing tools, without changing the actual games.
This is still something I would love to see. Hiliarous or game-breaking bugs in games; it's an utter shame that a lot of the funny or weird tricks have to be left out in the name of speed. :\
On a similar note: If you have two movies, one rated 3 for technical quality and another rated 9.8, which one do you think would more probably be more entertaining? (And please don't nitpick. Individual exceptions don't affect the answer to a question about what's the most probable situation.)
Probably the one with a 10 for entertainment.
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
You're not addressing my main point, Warp, which is that given the existence of people who do not care about technical ratings, forcing the technical rating into the only visible rating (which thereby obscures the entertainment rating) is a mistake.
If someone is scrolling through the movie listings looking for an entertaining movie, what are they supposed to do, click through on every one to get to the "discuss this movie" page so they can see the split-out numbers? Hell, if someone's scrolling through the movies looking for a technically strong movie, they're similarly out of luck.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Hell, if someone's scrolling through the movies looking for a technically strong movie, they're similarly out of luck.
But a way does in fact exist, if it's perhaps not too visible:
http://tasvideos.org/MovieStatistics/HighestTechnicalRating.html
I think if you read down the list that you will probably agree that most of these movies are going to be hard to improve upon timewise. It will certainly happen, but it will probably not be trivial. So I think that the tech rating is nice.
Also: if you look at the highest "entertainment" ratings:
http://tasvideos.org/MovieStatistics/HighestEntertainmentRating.html
Especially towards the high end, a large number of these movies are Mario, Sonic, Zelda, Mega Man, Castlevania or Metroid games. People naturally enjoy more familiar games, and popular games are more familiar to more people. That's fine -- I like all of these movies a lot. But it's nice to be able to collect the more obscure, yet highly optimized movies via the technical rating (King's Bounty!).
Oh, and having tech rating combine with entertainment to form the "real" rating is good because it skews the more optimized movies upwards.
