Posted: 10/11/2010 8:18 AM
Post subject: Alpha-blended, superimposed logos; graphical subtitles, etc.
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
[img_right]http://w-create.com/%7Ebisqwit/glc_hud1.png[/img_right]For a while now I have contemplated on trying out a new format for the subtitles. It bugs me to see such simple and raw subtitles being used on TASVideos publications. I am talking about the "<game> in <time> played by <player>; this is a tool-assisted blahblah see blahblah" standard text you see in every movie. I have been thinking of making it look better. My inspiration was my favourite TV channel, God's Learning Channel, as seen in this picture on the right. I wanted to make something similar. Also, it has been bothering me that the logo blending technique I've been using for years has still not been used by any other TASVideos encoder. By logo blending, I mean this effect where the video is already playing when the logo starts, and the video and logo are superimposed, as seen here, below (the bulk of the logo is in the process of fading out, but Milly is still fully opaque until she runs out from the screen). Other logos using the same technique are also shown (each image has a link to a sample publication where the respective logo was used). The motivation for doing partially superimposed logos is that I can have a logo that is more than 1 seconds long, without making the viewer wait for the actual content to begin, and also without masking the entire game video content for the duration of the logo. I strived to only mask things like static copyright / title screens, and even them only minimally; at least a glimpse of them can be caught while the logo is fading out, usually much more. In the standard logo (shown for Castlevania above), the TASVideos text begins to fade out at 0.8 seconds and has completely faded out by 2.8 seconds; but Milly continues running until 5.4 seconds when she exits the screen. I had also intentionally customized her running path so that she appears to run on the same surface where mr. Belmont walks. Since its inception, my logo blending technique has been implemented through terms of an emulator patch. My standard "nesvideos-piece" module reads PNG files from a specific directory, and superimposes them over the emulator's video while the video plays. However, this approach has been used by virtually nobody else than me, and emulator patching is not very convenient either. To solve the two problems simultaneously, I created a standalone program, which: -- Renders subtitles into a graphical, visually impressive format -- Reads a raw encoding (RGB24 video and raw PCM soundtrack) -- Superimposes it with a number of graphical sequences (the logo sequence and any number of graphical subtitles) -- Sends it to a dedicated encoder (by the standard "nesvideos-piece" mechanics, i.e. the same as --videolog parameter in fceux). An example of the end result of this program can be seen in my Rockman 2 encoding, available here (SD/mkv) or here (HD/youtube). It is still in alpha stage, and vital configuration (such as subtitles, video resolution, fps, and logo image locations) are still done in source code only. The source code can be acquired from the Git repository at git://bisqwit.iki.fi/hudmaker.git . I am providing it here in case someone wants to develop it further. Here is how exactly I encoded Rockman 2 (mandatory details like hudmaker configuration and filesystem paths, and optional details like remote shell and mencoder parameters, are omitted for brevity): 
Language: shell
# Set up an encoding script that simply puts audio into a raw file and video into another raw file.
VIDEO="cat s.log > 1.raw & cat > 2.raw"
# Run fceu
fceux --soundrecord /dev/null --videolog "$VIDEO" rockman2.nes --playmov rockman2.fm2
# It produced 1.raw (audio) and 2.raw (video).

# Run hudmaker
neshudmaker
# It produced audio.mp3 (audio) and tmp.avi (video).

# Produce final mkv (or, you can first use x264 to re-encode tmp.avi with optimal settings)
mkvmerge tmp.avi audio.mp3 -o encode.mkv
# It produced encode.mkv (final product).
The end result is a video encoding that has a logo and a subtitle like below: So there. I do not intend to make many more encodes for TASVideos (probably only the next Rockman movie), but I made this as a proof-of-concept one. The graphical appearance of the subtitle is not a masterpiece of art (I made it in less than an hour), but still very different from what has been used before, and more impressive.
Posted: 10/13/2010 11:52 AM
Post subject: Examples
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
And here is how I create transparent logo animations to begin with. As I have written before, logo animations are created as a set of PNG files. Each PNG file has the same resolution, that is the resolution of the movie being encoded (such as 256x224), and each file corresponds to one display frame, and (optionally) the filenames are numbered sequentially so that the name indicates which frame it is. For example, a 190-frame animation might consist of files frame000.png, frame001.png, and so on until frame189.png. I generally include the intended resolution in the animation, such as: logo_256_240_f103.png is the 104th frame of a logo sequence that is rendered at 256x240 resolution. To create the animation sequence (individual image files), I use a script of some sorts. In some cases, the script is wholly a single computer program (such as POV-Ray). In some cases, it is a bash-script that runs many tools (such as Imagemagick's composite) to superimpose actors on top of a static background image. Here are some frameworks for creating a transparent animation. 
Language: php
<?php

/* Define screen resolution */
define('W', 256);
define('H', 240);

/* This function, called once per frame, formats a filename for that frame's resulting image */
function ffn() { global $frame;
                 return sprintf('logo_%d_%d_f%03d.png', W,H,$frame); }

/* Begin rendering image frame */
function imgbegin()
{
  global $im;
  $im = ImageCreateTrueColor(W, H);
  ImageSaveAlpha($im, true);
  ImageAlphaBlending($im,false); 
  ImageFilledRectangle($im, 0,0,W,H, ImageColorAllocateAlpha($im, 0,0,0,127));
  ImageAlphaBlending($im,true);
}
/* Finish rendering image frame */
function imgdone()
{
  global $im,$frame;
  ImagePng($im, ffn()); ++$frame;
  ImageDestroy($im);
}

/* Render image frames, put your program here */
imgbegin(); render($im); imgdone();
imgbegin(); render($im); imgdone();
imgbegin(); render($im); imgdone();
imgbegin(); render($im); imgdone();
Language: shell
# Create a name for frame's image file
geto()
{
  printf 'frame%03d.png' $frame
}
run() 
{
  frame=0
  for x in `seq 256 -2 196`;do
    # Superimpose Alf into the static background image, put in output file
    composite -geometry +"$x"+93 alf.png background.png `geto` &
    frame=$[frame+1]
  done
}
Language: shell
# Run POV-Ray to create an animation, render at specified size
antialias="+a0.5"
maxframe=360
maxclock=3.0
# clock=seconds, frame=frames.
loop()
{
  x="$1"
  y="$2"
  size="+w$x +h$y"
  clock="+kfi0 +kff$maxframe +ki0 +kf$maxclock $3"

  povray $clock -D $size +ua $antialias millylogo.pov +ologo_"$x"_"$y"_f.png
}
The avatar used in this post is also created with a similar scheme.
Posted: 10/14/2010 5:54 AM
Quote
OmnipotentEntity
He/Him
Player (36)
Joined: 9/11/2004
Posts: 2630
Looks pretty awesome, I'll see if I can't manage to get this working in my setup.
Build a man a fire, warm him for a day, Set a man on fire, warm him for the rest of his life.
Posted: 10/16/2010 5:59 PM
Quote
GabCM
He/Him
Joined: 5/5/2009
Posts: 901
Location: QC, Canada
Bisqwit, that stuff you made is awesome! That's why I worked to make a single AviSynth which results of about the same thing as you did. I'm not talking about that sprite moving around the movie. I'm talking about the transition and the graphical subtitles. You can see the script I've made here: http://pastebin.com/c74Gsjjh Let me talk a bit about the script. There are comments in it, so those questions you want to ask are probably already answered here. About the fact I used HQx, you can learn more about that in this thread. I've used the SubtitleEx plugin (Documentation) and the TransAll plugin (Documentation) to accomplish this. You can see the result of this specific script right below. Link to video I know, the banner isn't as pretty as Bisqwit's, but I'm not an expert in image editing. And I can choose another transition, since this plugin has a lot of them. I used [1350] Genesis Alien Soldier by Dammit & Truncated in 18:32.27 for this experiment.
Posted: 10/16/2010 6:05 PM
Quote
Nach
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Mister Epic: I think the banners in your case would look better at the top of the screen.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Posted: 10/16/2010 6:07 PM
Quote
GabCM
He/Him
Joined: 5/5/2009
Posts: 901
Location: QC, Canada
Nach wrote:
Mister Epic: I think the banners in your case would look better at the top of the screen.
Good thing I've set a variable integer about its vertical position. However, since it's just an example, I won't reencode this for such a reason.
Posted: 10/16/2010 8:06 PM
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
I think MisterEpic's example at http://www.youtube.com/watch?v=w7qChIVCNU8 has the subtitle rectangles completely masking way too much interesting content...
Posted: 10/16/2010 8:08 PM
Quote
GabCM
He/Him
Joined: 5/5/2009
Posts: 901
Location: QC, Canada
Bisqwit wrote:
I think MisterEpic's example at http://www.youtube.com/watch?v=w7qChIVCNU8 has the subtitle rectangles completely masking way too much interesting content...
Maybe if I make the subtitles smaller and/or anywhere else, it would solve the problem.
Posted: 10/16/2010 8:22 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
While it's nice to look at and all, I'm of the opinion of keeping the screen clean as much as possible. I'm pretty sure I'm not alone in this. To me, it should be about the run, and I actually prefer that there was no subtitles at all, but I understand why it's there. I don't see how does plain subtitles "bug" you since it does it's intended purpose perfectly but besides that, it's interesting information and I'm glad you have shared how you have made it.
Posted: 1/11/2011 9:47 PM
Quote
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3570)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
TASer of 2010 NES TASer of 2010 DS TASer of 2010 Arcade TAS of 2010 NES TASer of 2008 NES TAS of 2008 Funny TAS of 2008 TASer of 2007 NES TASer of 2007 Funny TAS of 2007
Nach wrote:
Mister Epic: I think the banners in your case would look better at the top of the screen.
Agreed. If moved to the top it is actually a very nice concept. It doesn't get in the way and adds a touch of professionalism to the encode. Kudos
It's hard to look this good. My TAS projects
Posted: 1/19/2011 9:20 PM
Quote
henke37
Editor
Joined: 3/10/2010
Posts: 899
Location: Sweden
I was suggesting something similar on IRC the other day, that we could add a permanent semitransparent logo in one of the corners like most tv stations does.
Posted: 1/19/2011 9:44 PM
Quote
Derakon
Joined: 7/2/2007
Posts: 3960
Ick. I always found those annoying.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Posted: 1/22/2011 3:11 AM
Quote
Sticky
Joined: 2/26/2007
Posts: 1365
Location: Minnesota
I agree; they would be annoying. I understand why it would be nice to include them in publications, but I already saw the tasvideos spiel before the run started. I want to see the video game in its original state, not watch a glorified TASTV Channel with icons everywhere.
adelikat wrote:
I very much agree with this post.
Bobmario511 wrote:
Forget party hats, Christmas tree hats all the way man.
Posted: 5/3/2011 8:50 AM
Quote
Bisqwit
Editor, Active player (297)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
I made the first release of this software. You can compile on Linux systems. I was not able to cross-compile it for mingw. http://bisqwit.iki.fi/source/hudmaker.html New features: It can now be configured without having to recompile it. The configuration is supplied in an XML file. Example configuration: 
Language: xml
<?xml version="1.0" encoding="utf-8"?>
<hudmaker>
<input type="video">
 <file>/tmp/video.raw</file>
 <bpp>16</bpp> <width>256</width> <height>224</height>
 <fps> <nom>60</nom> <den>1</den> </fps>
</input>

<input type="audio">
 <file>/tmp/audio.raw</file>
 <rate>48000</rate> <channels>2</channels> <bits>16</bits>
</input>

<output>
 <bpp>24</bpp> <width>2048</width> <height>1792</height>
 <fps> <nom>30</nom> <den>1</den> </fps>
 <!-- These frames will be sampled: -->
 <firstframe>0</firstframe> <lastframe>2147483647</lastframe>
 <!-- Add this number of empty frames in the beginning
      before beginning of stream (to overlap subtitles/logo) -->
 <delay>0</delay>
 <recipe>
        mencoder NESVSETTINGS
        - -o /tmp/result.avi
        -mc 0
        -aspect 4/3
        -oac mp3lame -lameopts aq=0:preset=extreme
        -sws 7
        -ovc x264 -x264encopts crf=1:fullrange=on:threads=4:me=dia:subme=0:partitions=none:no_8x8dct:b_adapt=0:no_mixed_refs:trellis=0:weightp=0:no_weightb:scenecut=0:no_deblock:no_cabac:bframes=0
        -ovc lavc -lavcopts vcodec=mpeg4:vqscale=3:mbd=2:trell:cbp:v4mv:cmp=6:precmp=3:dia=-3:predia=-4:vb_strategy=1:aspect=4/3
 </recipe>
</output>

<overlays>
 <width>2800</width> <height>2400</height> <font>Monapo</font>
 
  <overlay>
   <file>/home/bisqwit/povray/nesvlogov5/logo_%u_%u_f%03d.png</file>
   <startframe>0</startframe>
   <numframes>248</numframes>
   <fadelength>0</fadelength>
   <fadewidth>0</fadewidth>
  </overlay>

 <hud startframe="250" endframe="780" tasvideos="1">
  <text size="13.0">囲碁★九路盤対局</text>
  <text size="9.0">IGO - Kyuu Roban Taikyoku in 4:50.33</text>
  <text size="7.0">Played by Joel Yliluoma</text>
 </hud>
 <hud startframe="860" endframe="1100" tasvideos="1">
  <text size="9.4">This is a tool-assisted speedrun.</text>
  <text size="14.0">http://TASvideos.org/</text>
 </hud>
</overlays>
</hudmaker>
Example result: http://www.youtube.com/watch?v=CALw_xr6GpQ (Not produced with this exact configuration file, but close.)
Posted: 5/3/2011 2:04 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
Cool! Thanks for releasing the software!