Moderator, Senior Ambassador, Experienced player
(907)
Joined: 9/14/2008
Posts: 1014
hegyak wrote:
Update TurnByTurn please.
So recently ais523 made a substantive update to turnbyturn.txt. We've made a bitmore progress since then but we're holding off on putting it into turn-by-turn as the state we're in may change due to a misbehaving Archon which we need to force a wand on.
By all means, pester us for updates - we need to start posting updates here more frequently anyway. :)
A.C.
******
Hooray, angry shopkeepers. :)
You should make the entirety of Minetown angry with you (without actually killing anyone, of course), if you have time. Maybe leave Izchak's shop intact.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Moderator, Senior Ambassador, Experienced player
(907)
Joined: 9/14/2008
Posts: 1014
Derakon wrote:
You should make the entirety of Minetown angry with you...
I don't think we'll be headed to minetown as there's not much there we really need. By NetHack standards, we're being very kind to any monsters we don't care about but we're being ruthless to any monster that stands between us and ascention.
A.C.
******
I don't think we'll be headed to minetown as there's not much there we really need.
I meant more "if you happen to have a couple hundred spare turns, this is something you could do with them". I don't really have a feel for how tight the timing is before turn 2000 comes around.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Moderator, Senior Ambassador, Experienced player
(907)
Joined: 9/14/2008
Posts: 1014
Derakon wrote:
dwangoAC wrote:
I don't really have a feel for how tight the timing is before turn 2000 comes around.
That's OK, we don't either. :) If we do have free time we'll probably solicit feedback on what to do with it, but at most that'd be a hundred or so turns. We'll see how things turn out with getting the rocks/meatballs/eggs together.
Thanks for your interest,
A.C.
******
Moderator, Senior Ambassador, Experienced player
(907)
Joined: 9/14/2008
Posts: 1014
errror1 wrote:
I think this run started about two years ago, I'm glad to see there is still progress, can't wait.
Your comment prompted me to go back to reread this thread and look at when I got involved in the run. Based on the forum posts I made and the file dates from the first script I wrote it looks like I got involved around 2010-08-26. Since then, ais523 and I have been working fairly consistently aside from a few breaks to work around our schedules. Someone in IRC recently asked if we would be done before Christmas, to which I asked "Of which year?" as I couldn't come up with a better qualifier. :) We're still chipping away, moving forward on ensuring the ascention run will actually work, but I wanted to look back at where we've come from framework-wise.
Over time the framework has changed considerably. In the beginning ais523 created a straight Expect script which we then decided to move into Python using pexpect - that worked, but we were effectively writing a pexpect script that played the entire game up to a given point and going further meant extending the script and every execution involved replaying the entire game which we knew wouldn't scale. I came up with a crazy idea in IRC one day and moved on to using a VirtualBox guest OS with snapshots. It was sync-stable and worked well, but our restore time was a bit more than 4 seconds per loadstate. Despite the long snapshot restore times we still managed to get a lot of the run done with this early framework. While ais523 was OK with the speed, I wasn't quite as content so I kept looking for better platforms and eventually settled on KVM.
With KVM, we've gradually introduced various speed improvements such as changing from using the migrate command to create snapshots to using savevm and loadvm with "unsafe" cache files. We've also moved to an entirely read-only filesystem (set to mount as a ro filesystem in fstab) and we're doing everything in /dev/shm on the guest. On the framework side I've tweaked pexpect's send delay to the bare minimum it can function at (note that this might cause problems if anyone tries it on different hardware but it's very stable on our shared TAS system). Changing the send delay significantly sped up the speed of execution of the scripts themselves as it now takes far less time to type characters. Finally, I bumped the pts serial speed from 115200 to 203400 which has shown to be helpful when we're sending a lot of characters to NetHack at once. It now takes about .2 seconds to restore state and we can go through 100 executions of a complicated script in 55 seconds.
On the usability front, the older VirtualBox and KVM implementations used a single pexpect-spawned shell and kept disconnecting from the guest to manipulate the state. I've since gone to spawning two separate shell sessions, one for controlling the state of the VM (which with KVM is done by connecting to a control serial interface, although we could also use something like QMP) and one for viewing the guest. This significantly reduced the amount of "flashing" when loading previous states and made it a lot easier to see the realtime results of scripts, although despite months of searching for a solution I still haven't found a way to get whitespace to align immediately after loading a state (there's always a brief delay before the redisplay completes but by that time the script has already moved on to the next result). In order to combat this I've moved to having a semi-HUD where we print the state of scripted results on the screen and update those in realtime. I've also introduced a tmux session inside the guest to use tmux's redisplay function so the exact state we had when the snapshot was taken is displayed.
This means that as of right now, to watch the current activity taking place on the run (think live TASing) the process involves:
- ./nethack (The actual application we're manipulating)
-> stty (for capturing input inside the guest)
-> tmux (for redisplaying the guest's state)
-> GNU screen (for connecting to the serial pts interface)
-> pexpect (for managing scripts and connecting to the guest)
-> tmux (on the host, for multiplayer commandline action)
-> ssh (from Ilari's machine, driven by a script he wrote)
-> termcast capture (for capturing activity for live TASing)
-> termcast.org (for broadcasting through telnet)
-> telnet (for connecting to termcast.org nethack-tas to view the activity)
If we add any more layers to this onion I think things will get incredibly out of hand. :)
On the code structure front, I've primarily used this project to become familiar with programming in general and get my feet under me with Python. I've come a long way, but I'm still a bit of a novice. One of my initial choices which I'm starting to regret was not moving to an abstraction layer earlier in the project, with the result that we now have dozens of scripts with substantial duplicated code. I've moved toward an initial abstraction layer that separates all NetHack references from the virtual machine guest state management which will allow this framework to be reused for other projects where useful state can be saved inside the guest (other rougelikes, in particular). As an asside, adding tmux in the guest was primarily needed because we were previously relying on sending Ctrl+R directly to NetHack to make NetHack refresh itself but that caused problems when we saved states at command prompts among other things. There's a fair bit of work left to migrate the older scripts (which still work) to the new abstraction layer but the result is far more readable scripts.
The next step in addition to the abstraction layer is to move to a class-based structure. What's crazy about this framework is it's actually possible to have concurrent copies of the KVM guest running. The best way to take advantage of this is to move to using Python classes and creating instances (of the pexpect spawn shell) which can each execute independently. Python itself would still be a single thread but the individual instances of the KVM guest could run on separate cores. My goal of parallel execution of scripts on multiple guests is very optimistic and may not come to fruition as it will require a HUD or some other interface to manage displaying the different guests. It's an exciting enough prospect to at least start moving everything in the current expectKvm.py into a class. I don't actually know what I'm doing here completely so it could be a rocky road but I will give it my best.
That's the state of the framework as it stands now. You can see the current code at http://gitorious.org/nethack-tas-tools and the current TAS state can be observed by telnetting to termcast.org and selecting nethack-tas. Thanks again for everyone's interest in this and by all means please feel free to ask questions,
A.C.
******
We definitely hope so!
'Twould be a good day for a grand console verification (although adelikat and Nach tell me that those are meaningless on DOS), and a celebration of all things NetHack. (Especially as it's during one of the major NetHack tournaments.)
Moderator, Senior Ambassador, Experienced player
(907)
Joined: 9/14/2008
Posts: 1014
Enterim wrote:
Wow, now that I've read a 34,000 word text document on not even half of the run so far, I'm pretty excited.
I'm a bit concerned that our submission text will be so long that we'll break the site. I seem to recall some conversation about how if we're too long it'll get rejected, but if we're *really* too long it'll get past the length check and be allowed anyway. I guess we'll see what happens when we submit the run. :)
Thanks for your interest,
A.C.
******
Anyone want an encode of the latest WIP? Here you go! This is a retimed version, with delays added and removed to make the run more watchable (we enter commands into the game at the rate of 1 per second; ttyrec players let you speed this up, slow this down, or frame advance).
The encode is a "ttyrec" format file, that's typically used for recordings of text-based games. (Converting it to a video would make it much larger and also lower quality; ttyrecs are lossless, except for not tracking information about the font used.) It's of the Linux version of the run (running on a version hacked to sync with the DOS version), rather than the DOS version, so it'll look a little different to what you might be used to. All the keen NetHack players out there probably have ttyrec-playing software already, but for the TAS community, I recommend using Jettyplay, which I created specifically for the purpose of allowing the general public to play back ttyrec files. (It's unfinished, but entirely useable for this purpose.)
BTW, we may have to redo the very end of this. The Archon's AI isn't cooperating on the ascension run, and we may need to force it to generate with an attack item; giving it more options on how to act ahead of time allows us to influence its AI ahead of time, because we already know what actions we'll be performing on the way up, and monster actions depend mostly on player actions.
enjoyed watching the ttyrec, I'd keep in mind what you want the screen shot to be at this point. When you get some extra turns, setting up some ascii art might be cool.
We're likely to have a great screen shot possibility during the ascension run, and it won't even require any setup beyond what we need anyway. I think someone said they wanted us to fill a level with monsters, right? What about filling the most open level in the game with monsters in rainbow patterns?
Joined: 7/12/2009
Posts: 181
Location: São Paulo, Brazil
I'm... not going to ask as to not ruin the surprise :)
Been accompanying the wips, and it's awesome how you're abusing every single mechanic in this game.
And I haven't forgotten the run either. I've been really busy at work recently, but dwangoAC and I are still up for continuing, and might even have time for it soon.
We hit a bit of a snag recently; it doesn't seem to be possible to manipulate the entrance to Earth correctly while using conflict. This is a problem with the current plan because we need conflict both earlier in the ascension run (the Quest) and later (Astral).
We've already arranged to be able to lose arbitrary items (other than bulky equipment like armour) on the way up through the dungeon, so it's quite easy to get rid of conflict after the Quest without losing time. However, Astral's proving more of a problem. We're probably going to need to change our strategy for that; our current idea involves cursed figurines, but we need to check to see if the timer on the figurine can be stacked correctly with the egg timers to make it appear in the right location on Water.
We've already arranged to be able to lose arbitrary items (other than bulky equipment like armour) on the way up through the dungeon, so it's quite easy to get rid of conflict after the Quest without losing time. However, Astral's proving more of a problem.
Can't you activate conflict on Water, by putting on a ring right before jumping into the portal, without screwing up the egg hatching?
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
That takes an action. We could just about spare an action there, but it'd mean a lot more manipulation in order to get set up for Astral, but we'd prefer to investigate approaches that don't waste actions first.
If we do spend an action, there are probably more effective ways to use it (in terms of reducing the luck needed).
Popping in to check on this run.
I noticed that there's not been any checkins to gitorious since December. How are the route experiments going? Any word on progress? :)
Build a man a fire, warm him for a day,
Set a man on fire, warm him for the rest of his life.
