If I am remembering correctly, in the SMW case, death should be counted as negatively as possible (fitness = -MAX_INT) as dying saves no time in the game.
I'm not sure why you would consider coins in your fitness function (they certainly shouldn't detract from fitness though).
Time remaining versus distance to goal works, except that some of the castles will consider regular level progression as poor fitness, since the moving blocks will travel left, and you need to consider that. It can't prune the actual solution, which is again why death needs to count as maximum negative fitness.
Letting it learn which enemies are death and which ones are safe is a good solution, but that assumes that MarI/O grows nodes as it learns. I haven't looked at the code at all, so I don't know its behaviour to new information.
Training levels would be interesting, you might also consider seeing what it does to Kaizo.
I am still the wizard that did it.
"On my business card, I am a corporate president. In my mind, I am a game developer. But in my heart, I am a gamer." -- Satoru Iwata
<scrimpy> at least I now know where every map, energy and save room in this game is
If I am remembering correctly, in the SMW case, death should be counted as negatively as possible (fitness = -MAX_INT) as dying saves no time in the game.
I am still the wizard that did it.
"On my business card, I am a corporate president. In my mind, I am a game developer. But in my heart, I am a gamer." -- Satoru Iwata
<scrimpy> at least I now know where every map, energy and save room in this game is
See, it's interesting that the problem with mario ai isn't the network itself, it's quantifying what, exactly, a victory condition is. Do we want the ai to completely fear death? (it may pause the game in order to not die) Do we want the to worry about what it does when it is low on lives? this all has to go into the fitness function.
Building a fitness function is by far and away the hardest part to quantify, what, exactly, does it mean to do well in a level?
And to top it all off, we also have to account for some of the funky level designs that require the player to weave through the level.
What about Yoshi's? This script needs a lot of work before it can be considered enough to beat the whole game on its own. I'd recommend we create a script that helps it navigate the top level map, for starters, though I have no idea where to start. on that. If someone has a script that can tell us what the id's of each level are when we are on the top map, i'd be happy to write a bit of a navigation system into the script.
I'd like to put it into a game where defeat and victory are more easily defined, like Advance Wars 1/2 - 3 victory conditions - win by destroying all enemy units, or taking their hq. At the same time, we also need to make the ai not fear defeat entirely. Perhaps we need to design the ai from the top level so it can navigate and select levels on its own. Or perhaps we use an open world game instead that allows for more things to run. But if the games get more open world, how would we quantify progression in something like metroid? After so long of playing, the game would the NN be able to progress, or would we need to guide it by hand?
Putting death as fitness = -MAX_INT is a terrible idea, because then it irrationally fears losing. On the other hand, if the NN was a real creature, then these games ARE its reality. It can literally die (cullStaleSpecies()) if it's unable to perform, but it receives its reward (offspring) by being able to satisfy the fitness function.
I know this sounds crazy, but we are literally creating a primitive lifeform whose reality is mario, and their sole function is to play mario. editx5: And that just blows my mind out of the water. This network is emulating the exact way that your and my brains fire off. Though in the current format it won't get many more neurons than a goldfish or jellyfish i imagine, (ability to reason and navigate divided by the complexity of network would be an effective way of expressing the level of intelligence any given network can optimally express)
Edit: if you want to add me on steam and talk about this stuff in live form, this username with a lowercase d is my steam name.
I already showed how to add more inputs in my version. Just add to that Inputs array/list/thing.
As for "rules", that's for the script to figure out on its own.
Why does everyone hate lua so much? :(
Anyway, in terms of fitness, implementing the distance from the end of the level seems like a decent idea imo. As in using a pathfinding algorithm, not a straight line
Do we know how to get that kind of stuff? What about deaths on-death event?
Edit: i've been playing with the script a bit, so here's a pastebin:
http://pastebin.com/Xvr5LQEP
This is still really crummy, but have a look at it. Please read the top notes before using it.
As it stands, you need a DP1, DP2, and DP3.state to run this or it will error out on you. Along with that, don't click the "show map" box, there's a bug and idk how to track that one down.
Edit: if someone could find something to show level victory, that would be pretty sweet.
Anyone knows how to make the green lines in the displayGenome() display other colors, like yellow for instance?
I am having a hard time seeing the green lines in Super Mario World, due to the green backgrounds in World 1-1.
Anyone knows how to make the green lines in the displayGenome() display other colors, like yellow for instance?
I am having a hard time seeing the green lines in Super Mario World, due to the green backgrounds in World 1-1.
So I tweaked the parameters a bit. In 4 hours, I was able to get this:
I bet you all have different variations of the input map, but my map is specifically evolved into checking multiple nodes first, before checking for enemies. Cool though.
Yeah, it's an issue with the way it's been programmed. SethBling uses the game screenspace (for showing it off, presumably) whereas we should be using the emulator's screen space, imo.
At any rate, have I mentioned that i hate lua enough? I want types and structs back. This programming language sucks. I'll take a language that requires an interpreter sure, but hot damn how they handle everything is sooo unintuitive.
I'm almost apt to write my own scripting interpreter for the emulator, though it wouldn't be so fully featured. Who uses 1-based indexing by default?
And going over the lua manuals:
With those two functions, it is straightforward to implement
Could they be any more pretentious?
For example, i'm trying to migrate fitness from a right-based movement approach to a discovery-based focus, thus using the rooms and current level to indicate fitness, so that the ai is more apt to explore.
So, to you guys out there that know more about lua than me. Am I doing this right so far?
local levelsPlayed = { local LevelIdx = nil, local RoomsVisited = { x = nil, y = nil} }
local currentLevelIndex = 1
function isOldLevelAddtoLevel(lvl, lvlsPlayed)
for i, #lvlsPlayed do
if lvlsPlayed[i].LevelIndex == lvl.LevelIndex then return true end
end
lvlsPlayed[#lvlsPlayed + 1] = lvl
return false
end
-- rooms are an x and y value sooo
function isNewRoom(testRoom)
local currentLevelPlayed = lvlsPlayed[currentLevelIndex]
for i, currentLevelPlayed.#RoomsVisited
if testRoom.x == currentLevelPlayed[i].x and testRoom.y == currentLevelPlayed[i].y then
return false
else
currentLevelPlayed[i] = testRoom
return true
end
end
end
I'm presently engaged in a long-running process to evaluate squirrel as a possible replacement. Doesn't help you any directly, but reading about a language designed by people who hated the same things about lua as you do might help you feel better while venting.
local levelsPlayed = { local LevelIdx = nil, local RoomsVisited = { x = nil, y = nil} }
You cannot use locals inside the table.
for i, currentLevelPlayed.#RoomsVisited
The a.b construct is interpreted as a["b"], except when b is a reserved word or has forbidden chars, like #.
EDIT: this might be useful for people who hate Lua
http://tylerneylon.com/a/learn-lua/
I'm presently engaged in a long-running process to evaluate squirrel as a possible replacement. Doesn't help you any directly, but reading about a language designed by people who hated the same things about lua as you do might help you feel better while venting.
Thanks Zeromus, if you ever started with it, I could help out with the implementations of it, as I also understand a bit of C# and having worked on an emulator would be good for my resume. I'll look into squirrel later as well, but for now I gotta add stuff to the script!
As a side note I am also working in my off time on a compiler for a lisp-styled scripting language. It's not nearly as featureful as lua but it's a fun project. :)
Back on topic. That code above has become this:
local level = {} --used for object creation, don't modify
level.__index = level --used for object creation, don't modify
function level.new(init) --used for object creation, don't modify
local self = setmetatable({}, level)
self.lvlID = init[1]
self.Room_index = init[2]
self.x = init[3]
self.y = init[4]
return self
end
function level.set_value(self, newval) -- used for object manipulation
self.lvlID = newval.lvlID
self.x = newval.x
self.y = newval.y
end
function getVScreenCount() -- zero-indexed
return u8(WRAM.vscreen_number)
end
function getHScreenCount()
return u8(WRAM.hscreen_number) -1
end
function getCurrentVScreen()
s8(WRAM.y + 1)
end
function getCurrentHScreen()
s8(WRAM.x + 1)
end
function getCurrentRoom()
return bit.lshift(u8(WRAM.room_index), 16) + bit.lshift(u8(WRAM.room_index + 1), 8) + u8(WRAM.room_index + 2)
end
local current_level = level.new({getCurrentRoom(), getCurrentVScreen(), getCurrentHScreen()})
i'm shamelessly stealing (thanks henke!) from the smw project from earlier. One thing i'm having trouble with though, is the size of the script file, so i'm going to try and separate it between the neural network and the main file with the main, executed bits. I hope I don't slow down LUA too much if I have a getMemoryValue() function like above.
Do note that the fittness criteria needs to be rather fine grained. Counting rooms won't do.
Yeah I've been keeping that in mind. I just realized I've been doing my inputs all wrong. A neuron's value should be firing or not firing, and therefore on my inputs, they need to be either -1 or 1, and the network should decide the weights on its own, therefore I need to find a way to set this whole thing up so that all the numbers in the game are -1 or 1, which clarifies things as I try to update the network.
The more I look at this code, the more I realize there's limitations, if you will, to when a certain action should be taken. Secondly, this also means that mario is uncapable of differentiating between what is an enemy, friend, or powerup. All it knows is that there is something nearby.
Therefore, this mario ai cannot, by its nature, beat a level without memorizing it. There are a couple of things here which makes things difficult for us then. In order to get an accurate sense of the game to the ai, we need better values. We need to have sprite types passed to mario. We also need to change it from mario's x and y to a vector distance between the closest point on mario's collision box to the enemy or to the other sprites' collision boxes, and have x number of neurons firing when an enemy is close, varying by distance from mario to the other sprite.
The getInputs() function is where the magic happens, and I am ashamed of myself for not seeing this sooner than I have. I had even watched a video describing this - neurons are either firing or they aren't. There's nothing else to it.
Then the inputs from there are passed in and the controller outputs are spat right back out. It's way simple, and I guess I got lost by lua's loop form.
On that note, this is a week from Hell for me. We'll see if I can get anything added to it, but I'm not sure. If someone could help me out, I would love to have a reliable way of getting the exact locations each object on the map is going to be. not enemies, but pow, end level, keyholes, keys, et cetera.
If there was a way to get that location on the map, I would be severely impressed.
OR if you want these neural nets in other games (legend of zelda, super smash, advance wars, et cetera), i'd love to see if we can get interesting things from this script.
I'm presently engaged in a long-running process to evaluate squirrel as a possible replacement. Doesn't help you any directly, but reading about a language designed by people who hated the same things about lua as you do might help you feel better while venting.
Sorry about derailing, but completely replace or as another option for scripts?
I bet you all have different variations of the input map, but my map is specifically evolved into checking multiple nodes first, before checking for enemies. Cool though.
