Although I'm okay with the rule for authenticity, there are a couple things I'd like added/changed:
1) Currently published TASses and all obsoletion attempts of those TASses are granted an exemption
I'm sure no one intended these rules to suddenly invalidate any existing publications, but I think it would be good to be explicit about it. But furthermore, in order to be able do a fair comparison, any future runs attempting to obsolete a published TAS must be done at the same CPU settings as the published TAS, regardless of whether or not it adheres to the new rules.
2) I feel as though I must be misunderstanding the following rule:
In-game settings and environment parameters that have arbitrary nature and don't belong to point 1 should be left at default values. Deviations from those are disallowed from Vault.
For example, if the game allows to set arbitrary speed factor (even limited to some range), picking arbitrary values can only belong to side branches, not to fastest completion (and%) or full completion (100%).
There are many games with in-game speed settings, allowing the player to adjust the speed of gameplay. Often times, setting these to the highest speed increases the difficulty of the game, and so acts both as way to make a faster speedrun and at the same time results in a more impressive accomplishment. Examples from my own runs include the Sierra games, Sid Meier's Railroad Tycoon and, of course, the CD Man game. All of these would have suffered in both entertainment and quality if they were forced to keep to the default in-game speed settings.
I don't see why PC games should have a special rule forbidding the use of in-game options that affect gameplay that other platforms don't have to follow.
Finally, because refactoring in JPC-rr is such a pain, it would be good to provide authors with an official list of accepted CPUDivider settings for each year, so TASsers can refer to it and know that they're safe to choose that setting before starting. It would also help judges to know if submissions meet the guidelines. I'd be very happy to submit a list for review and official adoption.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
c-square wrote:
1) Currently published TASses and all obsoletion attempts of those TASses are granted an exemption
I'm sure no one intended these rules to suddenly invalidate any existing publications, but I think it would be good to be explicit about it. But furthermore, in order to be able do a fair comparison, any future runs attempting to obsolete a published TAS must be done at the same CPU settings as the published TAS, regardless of whether or not it adheres to the new rules.
Obsoleting a published movie
Site rules still apply even if the published movie breaks them. We make mistakes sometimes, but that doesn't provide a green light for you to break the rules.
c-square wrote:
2) I feel as though I must be misunderstanding the following rule:
In-game settings and environment parameters that have arbitrary nature and don't belong to point 1 should be left at default values. Deviations from those are disallowed from Vault.
For example, if the game allows to set arbitrary speed factor (even limited to some range), picking arbitrary values can only belong to side branches, not to fastest completion (and%) or full completion (100%).
There are many games with in-game speed settings, allowing the player to adjust the speed of gameplay. Often times, setting these to the highest speed increases the difficulty of the game, and so acts both as way to make a faster speedrun and at the same time results in a more impressive accomplishment. Examples from my own runs include the Sierra games, Sid Meier's Railroad Tycoon and, of course, the CD Man game. All of these would have suffered in both entertainment and quality if they were forced to keep to the default in-game speed settings.
I don't see why PC games should have a special rule forbidding the use of in-game options that affect gameplay that other platforms don't have to follow.
Why are you skipping the part that addresses explicit in-game options directly?
In-game settings and environment parameters that are explicitly supported as modes are allowed.
This means there are limited non-arbitrary options the game was designed to work with, for example a few speed variants. Explicit support can be proven by in-game options, official PC spec recommendations, release notes, source code logic and comments, etc. Burden of proof is on the TAS author here. If this information is completely unavailable for a given game, use the environment specs that were common and popular in this game's era.
c-square wrote:
Finally, because refactoring in JPC-rr is such a pain, it would be good to provide authors with an official list of accepted CPUDivider settings for each year, so TASsers can refer to it and know that they're safe to choose that setting before starting. It would also help judges to know if submissions meet the guidelines. I'd be very happy to submit a list for review and official adoption.
This would help a lot, but being released the year when some CPU was available doesn't automatically mean it was supported. But for cases when no info about explicit support is known at all, this will help.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
1) Currently published TASses and all obsoletion attempts of those TASses are granted an exemption
I'm sure no one intended these rules to suddenly invalidate any existing publications, but I think it would be good to be explicit about it. But furthermore, in order to be able do a fair comparison, any future runs attempting to obsolete a published TAS must be done at the same CPU settings as the published TAS, regardless of whether or not it adheres to the new rules.
Where are you getting this from?
I wasn't clear; the above is what I want to add. Once a speedrun is published, it should set the bar for that game for three reasons:
1) It allows for a clean way of comparing a run against its predecessor to know which one is faster.
2) It means I can still use the same save states and TASScripts if I ever want to try and improve one of my runs. Switching CPUdivider settings would force me to have to start my existing runs from scratch, and I would lose all the work I did. It certainly would make me think twice about making improvements.
3) It means someone else can't come along later and obsolete an existing run simply because they've uncovered an old document saying that the game supported a faster CPU and upped the CPU settings with no other changes.
feos wrote:
Why are you skipping the part that addresses explicit in-game options directly?
I think this is where I'm getting confused. Both the second point and the third point talk about "in-game settings", the difference seeming to be whether the settings are "arbitrary" or "non-arbitrary". Would you explain what is the difference between limited arbitrary settings and limited non-arbitrary settings, preferably with some examples?
feos wrote:
c-square wrote:
Finally, because refactoring in JPC-rr is such a pain, it would be good to provide authors with an official list of accepted CPUDivider settings for each year, so TASsers can refer to it and know that they're safe to choose that setting before starting. It would also help judges to know if submissions meet the guidelines. I'd be very happy to submit a list for review and official adoption.
This would help a lot, but being released the year when some CPU was available doesn't automatically mean it was supported. But for cases when no info about explicit support is known at all, this will help.
I wrote earlier about my concern that the new rules would create an extra process for those making PC runs, and could dissuade people from taking up a PC run for the first time. I still have those concerns about the new rules. In my mind, it's not a far stretch to have a pre-approved list of mid-range CPU settings for each given year that we can reasonably expect the majority of games released then would have supported. An author then has the choice to either pick the pre-approved value based on the game's release date and not worry about having to hunt for documented game specs, or can choose to pick a higher value by doing the research legwork to defend the decision. I think we can strike a good balance that provides reasonable authenticity while still giving runners the option of a quick confirmation of their system setup.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
c-square wrote:
I wasn't clear; the above is what I want to add. Once a speedrun is published, it should set the bar for that game for three reasons:
1) It allows for a clean way of comparing a run against its predecessor to know which one is faster.
3) It means someone else can't come along later and obsolete an existing run simply because they've uncovered an old document saying that the game supported a faster CPU and upped the CPU settings with no other changes.
This is handled properly when we decide on obsoletion. It works the same as with improvements involving emulator accuracy:
If a new trick or optimization works in both emulator versions, but wasn't known when the old run was made, it's considered a valid improvement.
If a new trick or optimization is only possible due to accurate emulation, it's also accepted as an improvement.
If an old trick or optimization is impossible due to accurate emulation, we don't count that against the new run.
If there are no new tricks or optimizations, we don't consider it an improvement.
So you can't obsolete a run by changing the environment, even if you change it to be more accurate. You can only obsolete it with new tricks or optimization, with gameplay improvements.
c-square wrote:
2) It means I can still use the same save states and TASScripts if I ever want to try and improve one of my runs. Switching CPUdivider settings would force me to have to start my existing runs from scratch, and I would lose all the work I did. It certainly would make me think twice about making improvements.
How many of your projects depend on CPU speed that hard? Also since games that don't significantly change along with faster CPU are fine by this rule, your problem should only be with games whose gameplay is affected by CPU speed AND default jpc-rr speed is too high for them.
c-square wrote:
I think this is where I'm getting confused. Both the second point and the third point talk about "in-game settings", the difference seeming to be whether the settings are "arbitrary" or "non-arbitrary". Would you explain what is the difference between limited arbitrary settings and limited non-arbitrary settings, preferably with some examples?
Non-arbitrary explicit options presented as modes:
Choose game speed:
50%
100%
150%
Arbitrary setting:
Choose game speed:
50% - 150% slider
What would be a better way to word this so it's easy to understand?
c-square wrote:
I wrote earlier about my concern that the new rules would create an extra process for those making PC runs, and could dissuade people from taking up a PC run for the first time. I still have those concerns about the new rules. In my mind, it's not a far stretch to have a pre-approved list of mid-range CPU settings for each given year that we can reasonably expect the majority of games released then would have supported. An author then has the choice to either pick the pre-approved value based on the game's release date and not worry about having to hunt for documented game specs, or can choose to pick a higher value by doing the research legwork to defend the decision. I think we can strike a good balance that provides reasonable authenticity while still giving runners the option of a quick confirmation of their system setup.
I see, your suggestion is to swap the priorities: use the CPU speed that was available when the game was released, and if you can find proofs of support for other CPU speeds, you can use them. Though this doesn't address the point about possible problems with contemporary CPU speed, which the rule does address. I'm not sure if we can swap the priorities that way, but in any case, first we'd need a timeline table.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
As a preface: I tried to quickly skim through the other topic on PC environment settings for this, but couldn't easily find an answer for the following. (This post may belong better in that topic also, but I'm not sure, so it's here.)
When it comes to in-game speed settings, I don't understand why using a non-predefined value from within a limited range can't be allowed for vault.
In-game settings and system environment configuration aren't the same thing.
I understand the necessity for limiting/preventing inappropriate configurations in system environment, but I don't feel that in-game speed settings should be limited by the same rule.
It's true that some in-game settings are exclusively used to coordinate with system environment/hardware (v-sync and FPS, for example), but in-game speed options aren't only for that purpose. Some people simply like playing games faster than other people, and developers sometimes allow for this with speed sliders instead of preset speed options. The Sierra adventure games are a good example: earlier AGI based games have limited set speed options, latter SCI games however often used speed sliders.
In-game speed settings and vault eligibility standards.
Why does a game that allows a preset speed option of 125% make it more eligible for vault compared to a game that allows for a speed of 125% on a slider? The 125% on the slider is no more arbitrary simply because it's on a slider; it's still 125% of the default speed. Why is one 125% choice more eligible than another simply because of the method by which the choice is made?
If a system environment (hardware/emulated hardware) is controlled and a speed selector has defined upper and lower endpoints, there is a finite range limit of potential speeds at which the game can be played while still adhering to the appropriate environmental protocols.
The fact that the range is finite should be enough of a limit to arbitrariness that these games should still be eligible for vault using any speed selection available within the game's options. At bare minimum, either of the endpoints of the slider should be allowed in addition to the default setting.
The point of the 'appropriate environment' rule as I understood it was to prevent insane speeds/glitches achieved through use of inappropriate system (hardware) environment. To further limit already finite speed options just because they are on a slider instead of on preset bullet-point speed selections seems itself an unnecessary distinction.
Making this distinction is effectively saying that it's ok to play one game faster than the default but not another simply because the game-provided method of choosing the selected speed differs. It's making a issue over selection method, not the fact that a different speed is being used.
If an in-game speed setting is allowed by the game within an appropriately configured system environment, that in-game speed setting should be allowed for vault regardless of whether it was selected via slider or bullet-point.
To put it simply; if the environmental (hardware) protocols are appropriate, I don't agree that any point on a finite speed selector within the game's options should be ineligible for vault.
EDIT: added headers and slightly shuffled comments.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Please add headers to your post.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Yes please. We'll see how to format it afterwards.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
To answer your megapost, it seems I should make it "and don't belong to points 1 or 2". If a game explicitly tells you to set whatever value you want, any value is supported. If it tells you to set up to 125% and beyond that you're on your own and no guarantees are made, it's not explicitly supported.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
So you can't obsolete a run by changing the environment, even if you change it to be more accurate. You can only obsolete it with new tricks or optimization, with gameplay improvements.
I'm glad to hear that. That certainly alleviates some of my concerns.
feos wrote:
How many of your projects depend on CPU speed that hard?
All of them. As soon as you change the CPU divider, or practically any other setting in JPC-rr, the entire run will desync requiring it to be remade from scratch.
feos wrote:
your problem should only be with games whose gameplay is affected by CPU speed AND default jpc-rr speed is too high for them.
Yes and no. First, all my published DOS runs (and many others' as well) have gameplay that is affected by CPU speed. But furthermore, the rule is:
If changing the CPU frequency introduces gameplay differences ... set the CPU frequency divider to emulate the CPU speed intended for this game
Submitting an improvement of a run using the default slow CPU for a game that was written for a fast CPU is as much in violation of the current rule as vice versa. Thus, unless the default 20 mHz just happened to match the intended CPU of the game, no improvements are possible until the CPU divider is updated which, as mentioned above, would then force me to start over from scratch.
If a runner wants to improve a run using a more accurate CPU setting, that's fine. However, I would at least like to have the option of improving a run using the same CPUDivider settings as the published run. In the end, it increases the quality of the publications on the site, while leaving the authenticity no worse than it originally was.
feos wrote:
Non-arbitrary explicit options presented as modes:
Choose game speed:
50%
100%
150%
So, for example:
This makes total sense to me why this would be allowed.
feos wrote:
Arbitrary setting:
Choose game speed:
50% - 150% slider
An example:
It doesn't make sense to me why we'd be forced to keep this speed setting at the default value.
Then there's this:
It has explicit options presented as modes, but is on a slider, but the slider has 100 distinct non-arbitrary values. I'm not sure how to classify this one, but again, I can't see why we wouldn't allow a runner to use the value that results in the fastest run.
DrD2k9 wrote:
If it helps, I can make a list of valid CPU dividers based on CPU speeds available sorted by year.
That would be wonderful. Could you make it so it has both CPU Dividers and DOSBox settings?
Many thanks, DrD2k9.
To answer your megapost, it seems I should make it "and don't belong to point 1 or 2". If a game explicitly tells you to set whatever value you want, any value is supported. If it tells you to set up to 125% and beyond that you're on your own and no guarantees are made, it's not explicitly supported.
Space Quest is covered by this expansion. I believe CDMan is as well (as long as it doesn't warn you when passing a certain threshold that unexpected effects might happen)
To answer your megapost, it seems I should make it "and don't belong to point 1 or 2". If a game explicitly tells you to set whatever value you want, any value is supported. If it tells you to set up to 125% and beyond that you're on your own and no guarantees are made, it's not explicitly supported.
Space Quest is covered by this expansion. I believe CDMan is as well (as long as it doesn't warn you when passing a certain threshold that unexpected effects might happen)
I may be missing something, but I don't see know adding the line "and don't below to point 1 or 2" solves the issue.
Point 1 is limited to options that can be changed "without affecting the gameplay". Point 2 is limited to "limited non-arbitrary options". The sliders above affect gameplay and are considered arbitrary, so they don't belong to points 1 or 2, and thus fall to point 3 which says they need to be kept at their default values.
The purpose of removing arbitrary selections is to prevent playing at 199% if 200% is a valid, developer-supported option (Example: the aforementioned Jetpack TAS - at the maximum speed that the game supports, the game is casually unplayable on modern systems since it's effectively a no-speed-cap setting)
Let's say that there's an upper bound of 200% speed that the developers tested at. If they give discrete X% step options in a listed menu, then the only arbitrary speeds would be anything less than 200%.
Let's now suppose that instead of discrete step options, it's a slider. If the slider doesn't go above 200%, then that's all well and good - 200% is still the limit. But if the slider goes to 201% with a warning that "unexpected results may occur", then that 201% is an arbitrary selection outside of valid, developer-supported options. 200% would still be the limit.
The purpose of removing arbitrary selections is to prevent playing at 199% if 200% is a valid, developer-supported option
But why should we prevent 199% simply because it's not the default or fastest setting? It's still a developer supported option
While it typically makes characters move faster, increasing in-game speed settings also sometimes makes movement harder to optimize due to the balance between input processing and character movement within the game.
It's therefore possible that the difference between 199% and 200% may actually make it harder to move the character efficiently at 200% and thus allow for the game to be completed in a shorter time using the 199% setting. In such cases, why should 199% be disallowed simply because it's not the maximum or default setting?
What determines whether a given in-game speed setting should be allowed or not should only be limited to what the developers planned as options.
In the cases where the developers provide a warning, as you mentioned above, I agree that those values would be arbitrary and unintended.
EDIT: I should be able to start making that CPU Divider table sometime today/tonight. It might take a while to put together though. I'll see what I can do about including DOSBox cycles as well.
The purpose of removing arbitrary selections is to prevent playing at 199% if 200% is a valid, developer-supported option
But why should we prevent 199% simply because it's not the default or fastest setting? It's still a developer supported option
While it typically makes characters move faster, increasing in-game speed settings also sometimes makes movement harder to optimize due to the balance between input processing and character movement within the game.
It's therefore possible that the difference between 199% and 200% may actually make it harder to move the character efficiently at 200% and thus allow for the game to be completed in a shorter time using the 199% setting. In such cases, why should 199% be disallowed simply because it's not the maximum or default setting?
If you (the reader, not just you DrD2k9) can successfully make the argument for any game Q that arbitrary supported nonmaximum speed X is better for the TAS than maximum supported speed Y or default speed W, then an exception can be made for game Q.
But the argument has to be made first. The rule exists as a baseline.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
I improved the wording. As written in the edit note: "We can't draw the line based on amount of variations of some option. Really wide ranges may be explicitly supported. And non-arbitrary modes may be unintended. So instead, make sure the option is intended for normal play." Now this is in line with the rule about in-game codes BTW.
http://tasvideos.org/MovieRules.html#PcGameEnvironmentMustBeLegitimate
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I improved the wording. As written in the edit note: "We can't draw the line based on amount of variations of some option. Really wide ranges may be explicitly supported. And non-arbitrary modes may be unintended. So instead, make sure the option is intended for normal play." Now this is in line with the rule about in-game codes BTW.
http://tasvideos.org/MovieRules.html#PcGameEnvironmentMustBeLegitimate
I like it; it's now clear that any supported settings are allowed. Thanks for clearing that up.
I'm still concerned about the new DOS TASser who picks up JPC-rr for the first time and does a run with it, just to find out after submitting that the default CPUDivider value is too high and all her work is lost. It is a particular issue because the emulator instructions explicitly state:
Do not touch the other fields unless you know what they do, they are for advanced users.
Replacing this instruction with the list of speeds by year that DrD2k9 is coming up with would fix this. Or we could leave the instruction and always allow the default value as acceptable, with those who want to go higher bearing the onus of proving their case.
Finally, I'm still hopeful for a grandfather clause for improvements on existng runs, however the fact that underclocking a game is now allowed makes this less pressing.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
We will update this warning appropriately when there's the reference table.
What's the grandfather clause again?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Quick note on the CPU clock settings table: I'm still working on it as I'm able.
CPU Divider can be concretely calculated.
Unfortunately, DOSBox cycles can't be well calculated.
First, the formula I provided earlier in this thread is itself only an approximation for converting clock speed to cycles. Second, its accuracy fails toward the upper and lower ends.
This could likely be improved with some better/more accurate curve-fitting than what I did to get that formula in the first place.
That, however, may be a mostly pointless endeavor. As I've been looking more into this situation, I have found that the DOSBox cycle number isn't directly convertible to CPU clock speed. Emulated clock speed is dependent on both DOSBox cycle setting and host system specifications.
In other words, my understanding is that two PC's with different specifications will yield different emulated clock speed even when set at the same DOSBox cycle number.
I'll continue to fill out the table, but it will be limited to CPU Divider settings for JPC-rr.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Thanks for this list. I've looked over it and something just doesn't feel quite right about it. For example, I doubt the 40MHz 386 in 1985 was over 1.5 times faster than the 25 MHz 486 that came out four years later in 1989. I think basing this just on clock speed leaves out all the architectural improvements, as well as the ability to access things like faster RAM and more efficient CPU coding.
If we do want to use clock speed, I recommend we take a simplified approach. As I usually don't have the intended CPU specs for any given game I run, and the values in the list vary wildly even within the same year, I propose we use a trend average for each year by default. I plotted all the clock speeds from the link you referenced and took a trend line (.ods file).
(Processors with multiple values were averaged)
Of course, if someone wants to dig up the original game specs and choose a specifically supported CPU from your list, they'd still be more than welcome to.
feos wrote:
What's the grandfather clause again?
I'd like the option to be able to submit an improvement to a run using the same CPU Divider setting as the currently published run, even if it is higher than the game was intended to run on. Otherwise, none of the work done previously would be reusable, and all work would have to be done from scratch.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
c-square wrote:
I'd like the option to be able to submit an improvement to a run using the same CPU Divider setting as the currently published run, even if it is higher than the game was intended to run on. Otherwise, none of the work done previously would be reusable, and all work would have to be done from scratch.
How incompatible does the old divider have to be to require this clause in the first place? If a compatible machine was available back then, it's a pro. If the gameplay doesn't change, it's a pro.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
This makes total sense to me why this would be allowed.
It doesn't make sense to me why we'd be forced to keep this speed setting at the default value.
Then there's this:
It has explicit options presented as modes, but is on a slider, but the slider has 100 distinct non-arbitrary values. I'm not sure how to classify this one, but again, I can't see why we wouldn't allow a runner to use the value that results in the fastest run.
(Processors with multiple values were averaged)
Of course, if someone wants to dig up the original game specs and choose a specifically supported CPU from your list, they'd still be more than welcome to.
