I experience the exact same thing with another game, Ys II Complete. Even if I attempt to create the movie file with a different starting system timer value (usually, I just go with the default 6000), it desyncs very quickly. As Bernka noted, it's not always at the same spot; quite the opposite in fact, and so it's pretty much impossible to play back the movie correctly.
Incidentally, this is not an issue in the prior release of the game, Ys II Eternal, even though the two versions appear to be almost 100% identical to one another gameplay-wise, excepting a couple bug fixes and other small updates. Even so, both versions appear to have another bug in Hourglass, in that neither of them will allow the game to be sped up whatsoever (none of these issues are present in Ys I, but then again, it is my understanding that Ys I is very different code-wise from Ys II). So, I've got logs for both the Eternal and Complete versions, if you need to compare them. Hourglass settings are at the default values for r81, and the OS is Windows XP, SP3:
Complete -
https://www.mediafire.com/?ocwdndvcodq0bk8
Eternal -
https://www.mediafire.com/?9vfdxqd57ghuya9