How likely is it that someone who voted a straight 10/10 in the submissions forum just to see his favorite run published will now go back and re-rate it? Clearly such a person doesn't care about the rating system, so why should he bother?
Yeah, obviously. But just because they won't doesn't mean that they shouldn't :)
Joined: 8/26/2006
Posts: 1139
Location: United Kingdom
Although I'm glad the rating system has been removed I'm disappointed that it has been replaced so quickly with the old one. I think that, in the light of the rating system's resounding failure, now would be a good time to consider alternative solutions as the yes/no/meh system obviously isn't perfect.
I agree with mmbossman when the says that the rating system works on paper but not in practice. I remember someone arguing that weighting the ratings may work better i.e. if X's average rating is an 8 and Y's average rating is a 6 they will both be of equal value on the workbench.
This, of course, is just an idea. I'd say that it may be worth considering alternatives, as so many people opposed the voting system a year ago when asked, that it may be foolish to completely revert so suddenly.
Derakon> I think anyone who votes wrong because they didn't read the poll deserves what they get.
We could also randomize the button positions on every page load, that would teach them, eh?
It not just their problem, it also gives the wrong input to us as judges. I don't see why they should be in an illogical order. It doesn't help anyone.
Joined: 5/25/2007
Posts: 399
Location: New England
In terms of pre-publication voting, I think the yes/no/meh system is fine.
For movie ratings, I may have spouted on this before, but here's my ideal rating system (it's similar to Mukki's idea):
If you want to rate a movie, up pops a list of all the currently published movies you've ever rated, from 1 to n. Number 1 is your "favorite" movie. Number n is your "least favorite" movie. You then choose a ranking for the current movie, from 1 to n+1, resulting in a new, ordered list.
Movies' rankings would then be some aggregate of the position of a movie on each rater's lists, perhaps with a minimum 10-rater threshold before a movie's ranking appears.
For interface, one could modify a single movie's ranking with just a text box, or take a page from the netflix queue system for multiple simultaneous adjustments.
This, as I see it, solves a few annoyances with the current ranking system:
1. the 10/10 problem (and its inverse, 0/0): under the current system, individual users can have significant impact on the rankings of their favorite/least favorite movies by rating them at the extremes. Under this system, they'd have to prioritize, because only 1 movie can be their favorite/least favorite.
2. no more arguments about technical vs entertainment
3. no more worry about ratings inflation, the comparison of ratings across different eras of TAS'ing, etc.
Obviously, this would be a radical change, but I think it would end up being beneficial.
I'll drop an idea here that I just had. I'm not sure if it's a good one.
It's that in a rating system from 0-10, the 10s of the voters who give lower votes in average should count more than 10s of others. Now often people refrain from giving low votes to bad movies because they think it isn't going to be published anyway, so why bother. This will force them to vote low so they have more voting power for the movies they like.
I don't know how it would be handled best mathematically, nor am I sure if the idea is any good at all. [ (2*Rating-Average(of that User)) +10] / 3 would be a very basic fomula, I hope. I don't really have much of a clue when it come to mathematics. x_x
Abuse could happen if users change all their previous votes to 0 to vote 10 to get a new movie published, so it wouldn't work for that, only for rating purposes. (I hope)
I agree the new voting system isn't working, although I suspected it would happen but felt it needed trialed anyway. There some good ideas suggested, while some are inpractical or wouldn't work well in practice. Another idea is to keep the rating system, but limit it to one rating with the question being "On a scale of 1 to 10 how publishable do you consider this movie to be?"
Sadly, I doubt it will work. Since it will be bombed with 10's to an even worse extent than what the current rating system is.
So what if I don't like watching crappy movies on obscure, unsuitable games, and hence never have to vote low? Averaging a user's votes is only valid if the user watched an standard distributed set of movies.
It's also doesn't hurt to have our top movies actually display a 9.x on the front page. Promotion and stuff.
If we wish to avoid abuse, we could employ a rating bias metric: Calculate how close a user's ratings were to the public opinion, i.e. the difference between his rating and the total average.
For example, I rated 8 on movie A and 4 on movie B, while the average rating is 6 on A 4 on B. My opinion is on average ((8-6) + (4-4))/2 = 1 above the popular opinion. I'm a high-voter, reduce all my ratings by 1 point.
(there are some corner cases that need to be taken into account to avoid unwanted side-effects, though. Like users with only 1 vote.)
Of course, any form of normalization can be manipulated by placing a few strategic votes to shift your average. But I feel that my suggestion will give better results than just setting the rating's mean to 5.
I prefer this idea that ratings are normalized within each single movie.
<klmz> it reminds me of that people used to keep quoting adelikat's IRC statements in the old good days
<adelikat> no doubt
<adelikat> klmz, they still do
Averaging out individual votes would have the effect of more or less averaging out ratings.
I stopped giving movie ratings a while ago (aside from the occasional oddball) and stopped bothering to deal with submissions as well.
I prefer the yes/no/meh system, but as I said I would upgrade it to have slightly higher resolution as "yes", "lean yes", "meh", "lean no", "no". For display purposes you could average it out that yes, meh, and no are +2 to each of their respective values and the leans are +1 to themselves and meh.
e.g. yes = +2 yes, meh = +2 meh, lean no = +1 meh, +1 no
I used to see a lot of "Weak _____" votes, so this is a good compromise.
Also, I strongly dislike the movement away from "Did you like this movie" to "should this movie be published".
If you want to rate a movie, up pops a list of all the currently published movies you've ever rated, from 1 to n. Number 1 is your "favorite" movie. Number n is your "least favorite" movie. You then choose a ranking for the current movie, from 1 to n+1, resulting in a new, ordered list.
You would be surprised how easy it is to get into a situation with such lists where you later examine your list and see that you really think that movie A is better than movie B, movie B is better than movie C, but movie C is better than movie A, for different reasons.
If all three movies are in consecutive order in the list, then it's not such a big problem: Just choose one ordering which is the best compromise. However, you would also be surprised how easy it is to get into that situation but so that there will be something like 20 movies in-between those three, and deciding on a good ordering is a headache.
Otherwise I think it's a good idea. (Of course its only problem is that it would require more or less rewriting the entire rating system and dropping all existing ratings, which might not be a welcomed change.)
Xkeeper wrote:
Also, I strongly dislike the movement away from "Did you like this movie" to "should this movie be published".
I don't. "Did you like this movie" is asking the wrong question. Even if you don't like a movie, that doesn't necessarily mean it's not worth publishing.
Personally I find many of the published movies quite boring and I don't especially like them too much. However, most of them are worthy of publication: They are well-executed, and there are no good reasons why they shouldn't be published. Just because I find them a bit boring doesn't mean they aren't good.
Conversely: Even if you do like a movie, that doesn't necessarily mean it should be published. Maybe it's entertaining and fun to watch (to you), but could be clearly improved, and the author should make an improved version by removing the sloppy parts and mistakes. Maybe the submission breaks the rules. Maybe the submission is worse than an existing publication. There may be a ton of reasons why something should not be published even if you like it a lot.
Yeah, I'm not even going to bother arguing over it. I just see "is this movie publishable" turning into "is this movie well optimized" since that's all tasvideos ever is any more.
Yeah, I'm not even going to bother arguing over it. I just see "is this movie publishable" turning into "is this movie well optimized" since that's all tasvideos ever is any more.
I'm really getting tired of that claim. It's wrong for two reasons:
1) Tasvideos have always been about completing games as fast as possible with superhuman optimizations. From the very first SMB test runs I remember Bisqwit doing. There was never a time when speed was not the main goal. (The only thing which was different in the first years was that tools were not perfect and thus it was almost impossible to create frame-perfect movies. However, that doesn't mean the goals have changed.)
2) There have been many movies where completing the game as fast as possible is not the main goal because it doesn't make sense in the game in question.
"Should this be published?" is the correct question. "Did you like it?" is not. If someone makes a submission that doesn't even complete the game, doesn't attempt to do anything superhuman, but simply does something funny and he writes in his submission text "please publish my machinima video because there are no other places where I can publish it to the internet", would you vote yes if you find it a funny video?
If someone makes a submission of Mortal Kombat 2 where he states that his goal was not to complete the game as fast as possible but to show as many combo moves, finishes and glitches as possible, why would anyone vote "no" if the question is "should this be published"?
Joined: 4/21/2004
Posts: 3517
Location: Stockholm, Sweden
Yes, but he is right Truncated. 99% of all runs from the beginning to now is all about speed/completing a game fast as possible.
Nitrogenesis wrote:
Guys I come from the DidyKnogRacist communite, and you are all wrong, tihs is the run of the mileniun and everyone who says otherwise dosnt know any bater! I found this run vary ease to masturbate too!!!! Don't fuck with me, I know this game so that mean I'm always right!StupedfackincommunityTASVideoz!!!!!!
Arc wrote:
I enjoyed this movie in which hands firmly gripping a shaft lead to balls deep in multiple holes.
natt wrote:
I don't want to get involved in this discussion, but as a point of fact C# is literally the first goddamn thing on that fucking page you linked did you even fucking read it
Cooljay wrote:
Mayor Haggar and Cody are such nice people for the community. Metro City's hospitals reached an all time new record of incoming patients due to their great efforts :P
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
Xkeeper wrote:
That 1% would be this, of course.
And even then, I would be willing to bet that if it wasn't by adelikat and JXQ, it wouldn't have made it.
Jesus tap-dancing Christ, nothing makes you happy, does it? You bitch about how all movies are focused on speed, then ridicule one of the movies that doesn't aim for speed, just because of the authors. If nothing about the site manages to make you happy, then why do you continue to stick around? (that was rhetorical, I really don't want to know)
That 1% would be this, of course.
And even then, I would be willing to bet that if it wasn't by adelikat and JXQ, it wouldn't have made it.
Jesus tap-dancing Christ, nothing makes you happy, does it? You bitch about how all movies are focused on speed, then ridicule one of the movies that doesn't aim for speed, just because of the authors. If nothing about the site manages to make you happy, then why do you continue to stick around? (that was rhetorical, I really don't want to know)
Okay, let me clarifiy it for your puny mind:
If it wasn't for the authors, that movie would not have been published.
That does not say anything about the quality of that movie in my mind. That says that this site would have rejected it because it is not aiming for fastest speed.
P.S. I actually love that movie quite a bit. <3 The baseball part is the best.
