This awesome paper is all about vectorizing 8- and 16-bit videogame art so it can be upscaled without loss of quality. There's some frankly amazing examples there; I'd love to play Super Mario World with this as a realtime filter. It doesn't always work out well (the Doom Guy's head at the end of the article, for example), but even then it's hard to say that the vectorization made things any worse.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
ho-ly-crap that is awesome. I think I'll try and code this algorithm and see how it turns out.
It would be great if we could try this for high-resolution encodes. Even if it's processor-intensive, encodes don't need to be made in realtime.
Before anyone spends too much time working on an independent implementation, though, it'd be worth contacting the authors and asking if they have source code available.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria
Derakon wrote:
It would be great if we could try this for high-resolution encodes. Even if it's processor-intensive, encodes don't need to be made in realtime.
Have I told you how much I love you? That is one of the best encoding related suggestions I've seen for a while.
I hope this can go past congress. *pokes Grunt*
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Derakon wrote:
Before anyone spends too much time working on an independent implementation, though, it'd be worth contacting the authors and asking if they have source code available.
I'm already in contact with them. Regarding mistakes in their paper, and source code.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Okay, I had a nice correspondence with the authors of the paper. They're happy to correct issues with it as soon as their server stops being hammered.
Regarding code, whatever they wrote is owned by Microsoft. Dani did not write any of the code. Johannes on the other hand is very interested in seeing us reimplement it and using it.
Johannes Kopf wrote:
I am more than happy to answer questions and give advice if people from the emulation community want to reimplement the algorithm. I certainly hope that this will happen, since the algorithm is not too complicated after all.
So, anyone want to get to work on this?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Random thought: it'd probably be a good idea to split things out by sprite (and, for the SNES, by layer).
I might be interested in poking at this a bit, but I shouldn't be the core effort; my time's too unpredictably allocated.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
I am concerned about the inability of their algorithm to cope with stark, contrasting dithering patterns. Another problem with vectorization algorithms is that the shape of character may dramatically change when moving across suitable surfaces; for example, if Mario is outlined in black, and he moves across a background that contains details outlined in black, the shape of Mario's outline may fluctuate rather distractingly.
In other news, pessimist for hire.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
To both of you, my thoughts exactly.
However, on their homepage, they have a video of Super Mario World in action with their filter.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
However, on their homepage, they have a video of Super Mario World in action with their filter.
On which homepage?
In that note, regarding http://www.cs.huji.ac.il/~danix/ -- WOW, that guy has done a lot. I was just the other day studying the tone-mapping algorithm devised by Raanan Fattal, but that guy was a co-author to that too.
So has the other guy. Half of them, along with Dani.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Well, this is surely a nice step forward since EPX and HQ*x. I particularly like how it handles wiggly contours.
However, as for adopting it for high-resolution encodes, I suggest not doing so for two reasons.
First, there is still a loss and/or deformation of detail, which is pretty much inevitable due to how these algorithms work. Even the test cases in the abstract have numerous artifacts (except for the simplest forms), and complex sprites make it more obvious. Nearest neighbor upscaling we're currently using, while looks blocky, preserves all of the detail because it doesn't transform pixel boundaries.
Second, low-res pixel graphics is custom-made for certain conditions, it's never a downscale of a higher-res artwork. Our perception makes allowance for that and fills in the details automatically. When you upscale it like this, shades and certain contours "remain" unnatural, but there is no more noise "hiding" the details, thus no gap filling done by brain. The resulting image looks like bad watercoloring or stencil drawing (see the Doom guy's head for a good example). Who's to say a Space Invader should have curves so smooth? Maybe it did have flat surfaces — at least Taito has never drawn it otherwise.
At best you're trading authenticity for compressibility. The games don't look better like this; they look differently, and by my taste it's considerably worse.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
I certainly wouldn't suggest using this in place of the existing "simple" upscaling for the high-res encodes. I do think it would be fun to try.
It would be good to get some estimate of runtime, though. Even though encodes don't need to be realtime, if this filter takes e.g. hours per frame then it's not practical. The YouTube video is of a very limited portion of the screen, which is worrying.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Joined: 11/27/2004
Posts: 688
Location: WA State, USA
These kinds of smoothing filters only really work on games with clean, cartoony graphics, like Mega Man X, Gimmick!, or Kirby. Games with more detailed graphics, like Batman, Lionheart, or Golden Axe, tend to look much worse. This algorithm seems like no exception. Honestly, when I want my games to look better when emulating, I just use a 25% scanline filter, or sometimes an NTSC/RF filter (depends on the game). The improvement usually tends to be surprisingly substantial.
I could see this algorithm being useful as the first step in upscaling graphics for an HD remake or whatever, though even then you'll still need artists to clean everything up. For example, in all of those Mario World examples the outlines seem way too thick and would need to be made quite a bit smaller (though this could be partially solved by factoring in the relative brightness of pixels into the equation...).
Bisqwit wrote:
Another problem with vectorization algorithms is that the shape of character may dramatically change when moving across suitable surfaces; for example, if Mario is outlined in black, and he moves across a background that contains details outlined in black, the shape of Mario's outline may fluctuate rather distractingly.
This problem could be solved by processing layers and metasprites individually. Of course, the problem here now becomes how to determine which hardware sprites compose a metasprite without resorting to game-specific solutions. And, of course, you couldn't easily separate parallaxing layers if you just had a raw video stream.
Nach wrote:
I also used to wake up every morning, open my curtains, and see the twin towers. And then one day, wasn't able to anymore, I'll never forget that.
I personally prefer simple nearest-neighbor upscaling when I run games in an emulator; sure it looks a little fuzzier, but it's not as blocky as straight magnification or as wobbly and processor-intensive as the more sophisticated algorithms tend to be, and it also works well with anti-aliased pixel art.
I personally prefer simple nearest-neighbor upscaling when I run games in an emulator; sure it looks a little fuzzier, but it's not as blocky as straight magnification
I think you are confusing terminology. "Nearest neighbor" means taking simply the color of the nearest pixel as-is. In other words, the original pixels get simply scaled up to larger squares. (If you are scaling up to a resolution that is not an exact multiple of the original resolution, then some of the squares will be a pixel larger in one or both directions than the others, which may look ugly.)
You are probably talking about upscaling using bilinear filtering (iow. linear interpolation of in-between pixels from the original pixels, on both axes).
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria
RT-55J wrote:
This problem could be solved by processing layers and metasprites individually. Of course, the problem here now becomes how to determine which hardware sprites compose a metasprite without resorting to game-specific solutions. And, of course, you couldn't easily separate parallaxing layers if you just had a raw video stream.
If we can get emulators that are able to dump into a video codec that actually properly supports 32-bit alpha transparency or can set the base colour to a specific one never used by the game, we could easily achieve Blue/Green-screening methods of upscaling, assuming this could be implemented as an AviSynth filter that would be able to remove the colour that needs to be transparent before processing.
This does increase the encoding time massively, mind you.
I created a few scaling algorithm tests. I used Imagemagick for the filters that are based on a mathematical filter function, and DOSBox's source code for the context-dependent filters.
http://bisqwit.iki.fi/kala/filt3/
I'll add a Depixelizing Pixel Art version once there's a version I could use.
