Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
Since DTM Reader is woefully out of date, I figured, why not make a new working one?
So, I did. It's ready for public testing.
This is a Release Candidate. It reads and writes correctly. But it hasn't been fully tested against every possible situation.
Download:https://files.tasvideos.org/emulatordownloads/tastools/DTM_Editor_1.6.7zRelease notes:
Current Release: 1.6
What's new?
Version 1.6
Gamecube controller data can now be saved. It should be correct. My Hex Editor seems to reflect this.
Corrected reading of Gamecube controller data for more then one controller.
Version 1.5:
Full Gamecube controller data is being read correctly. Wiimote and Wiimote with Gamecube Controller data is NOT ready yet.
Yes, it will support any number of Gamecube Controllers in any port configuration.
Saving is NOT ready yet. Don't get too excited yet.
Version 1.4:
Corrected a bug with CPU core value being written incorrectly for Interpreter Vs. JIT.
Added the start of Input Editing for Gamecube Controllers. Only input data can be read and only some values are being read correctly.
Added Form Controls for MD5 hash of the disc. Multi Disc games are untested.
Added Form Controls for DSP Files. Dolphin and Official (Nintendo) hashes are supported.
Version 1.3:
Everything.
Current Features:
All Header data except for Recording Start Time, SHA-1 of GIT revision are unchangeable in the interface.
Reads Gamecube Controller input based on the selected Frame. Yes, Frame 0 is valid according to me.
Wiimote Input data and Wiimote with Gamecube Controller Data is not read yet.
Future Features:
Read and write Wiimote data packets (input). Read and Write Gamecube Controller with Wiimote frame data.
Special notes:
Older DTM files may have issues due to organization, layout structure and/or blank data. Modify them at your own risk. Newer Dolphin (anything post 4.0-2000) DTM files should work properly.
Make sure you back up your DTM files before using this program. There is a chance the program will break your files. If you encounter an error in the program, please post the DTM and what went wrong.
Known issues:
Extremely large values will not be written correctly due to the inefficiencies with Double.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
All multi-byte integers are little-endian.
That answers that about Tick. Go figure. It goes crazy high fast.
0x0E5 N/A 12 Reserved
0x0ED Integer 8 Tick Count
I think something is wrong here. Why write 12 bytes of reserved data over the Tick Count's value? Or should that have been written as 0xF5 N/A 12 Reserved?
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
I think something is wrong here. Why write 12 bytes of reserved data over the Tick Count's value? Or should that have been written as 0xF5 N/A 12 Reserved?
Fog had said that the implementation of the tick count was buggy, but I don't know if he released a Dolphin revision that fixed it. You could contact him to ask what he changed.
Though, that might not be helpful to know, since hundreds of Dolphin revisions have already been released with the tick count bug and DTM Editor will have to support those movies as well.
Number of controllers is not actually the number of controllers, it is a bitfield, generated as such:
for (int i = 0; i < 4; i++)
{
if (SConfig::GetInstance().m_SIDevice[i] == SIDEVICE_GC_CONTROLLER || SConfig::GetInstance().m_SIDevice[i] == SIDEVICE_GC_TARUKONGA)
controllers |= (1 << i);
if (g_wiimote_sources[i] != WIIMOTE_SRC_NONE)
controllers |= (1 << (i + 4));
}
Same with bongos.
Also, can you not simply include the struct from https://github.com/dolphin-emu/dolphin/blob/master/Source/Core/Core/Movie.h#L60 and then just read the dtm file into that struct? Not sure if c# has a convenient way to do that, since i know nothing about it, but it will pretty much just make everything work automatically, including keeping unknown values (so it doesn't totally break if the format is updated). In c++ reading the file is as simple as
Also, can you not simply include the struct from https://github.com/dolphin-emu/dolphin/blob/master/Source/Core/Core/Movie.h#L60 and then just read the dtm file into that struct? Not sure if c# has a convenient way to do that, since i know nothing about it, but it will pretty much just make everything work automatically, including keeping unknown values (so it doesn't totally break if the format is updated).
You make a good point. As long as you make sure to alter u8, u32 and u64 to the correct types (unsigned byte, unsigned int and unsigned long, respectively), that struct should be directly usable. The reading will be more complicated, but if you read (and write) the file as a byte[], it's conversion to the struct is doable with some magic from the Marshal class.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
I am getting close to releasing a Beta. I got the Reading and Writing events done. In theory the program shouldn't have a problem. The values are being read correctly for the important stuff like Frame count. I tried some old DTM files and they appeared to be read correctly (and missing some data).
Can anyone send me some DTM files any of the following things:
Memory Card in Slot A nothing in Slot B
Memory Card in Slot B nothing in Slot A
WiiMotes and GC Controllers in the same DTM (Brawl is a good option)
Any Wii DTM using Dolphin Version 4.0-2250 or greater
One or more sets of Bongos connected
More then one GC Controller connected.
A mixed number of Bongos and GC Controllers connected. Maximum of four total.
A run where a Disc needed to be swapped. Speed is not a factor. Just need to verify the Second Disc Name in the DTM.
Any sample DTM does not have to do anything except have around 100 frames of VI Frames or however long it takes for a WiiMote to be recognized by the system. I would like to know what version of Dolphin recorded the DTM so I can note how it behaves and make sure it's supposed to be that way.
I will use these files to verify my reading and writing methods are correct.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Here are 7 of the 8 tests you requested. (All except 3; I don't currently have access to my Brawl disc.)
Test 1: Dolphin 4.0-3022, Free DSP ROMs, on Paper Mario: TTYD
Test 2: Dolphin 4.0-3022, Free DSP ROMs, on Paper Mario: TTYD
Test 4: Dolphin 4.0-3022, Free DSP ROMs, on New Super Mario Bros. Wii
Test 5: Dolphin 4.0-3022, Free DSP ROMs, on Paper Mario: TTYD
Test 6: Dolphin 4.0-3022, Free DSP ROMs, on Paper Mario: TTYD
Test 7: Dolphin 4.0-3022, Free DSP ROMs, on Paper Mario: TTYD
Test 8 (included): SoulCal's Resident Evil 4 TAS, Dolphin 3.0-389, Official DSP ROMs, Disc names are Biohazard 4 [disc1].gcm & Biohazard 4 [disc2].gcm (not included)
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
I did some checking on my own.
If there's a Memory Card in Slot A, only, the value written is "3"
If there's a Memory Card in Slot B, only, the value written is "2"
If there's a Memory Card in Slots A and B, the value is "3"
If there's no Memory Card in Slots A and B, the value is "3"
Am I parsing the data wrong? My Hex Editor doesn't seem to think so. It reflects the same thing my program is reading.
My program does know a Wii DTM is a Wii DTM. So that's good.
I am having an issue with the Controller data.
Number of Bongos, is ALWAYS 0F. Even if I have NO Gamecube controllers connected.
GameCube Controller data:
If I have a single controller in Port 1 the value is 1.
If I have a single controller in Port 2 the value is 2.
If I have a single controller in Port 3 the value is 4.
If I have a single controller in Port 4 the value is 8.
This makes sense. This behaves. Four Controllers = 15 (0F). This still makes sense.
Do the Non-Bongo Controllers have anything special? I don't see anything about that.
I don't have the Multi Disc DTM yet. If you want the DTM files I am testing against to see what I am doing wrong, let me know.
Edit:
I checked CoolKirby's files to see if I am going insane.
I was. 4.0-2936 was writing bad data for Memory Card and Bongos from what I can tell. Or my custom edited Dolphin (restores Sonic Unleashed Video Hack) was barfing bad data.
Edit More:
The Memory Card data is having problems. I keep getting the result of 3. RachelB, what is going on here?
Memory Card DTM files:
http://1drv.ms/1quHoGv
Controller Data will be corrected before I release the beta. Bongo data will be corrected as well.
Edit For the Last Time:
For some reason, GC controllers are treated as Bongos.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
It's a bitfield as well.
For memory card A the first bit is set, ie 0000 0001
For memory card B the second bit is set, ie 000 0010
For both, both bits are set, ie 0000 0011
It's same as controllers.
Really i recommend just reading it directly into the DTMHeader struct, and then you don't have to worry about any of this. Parsing it bit by bit is silly when you can do the entire file at once.
Do the Non-Bongo Controllers have anything special? I don't see anything about that.
Only gamecube controllers, bongos, and wiimotes are supported.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
RachelB wrote:
It's a bitfield as well.
For memory card A the first bit is set, ie 0000 0001
For memory card B the second bit is set, ie 000 0010
For both, both bits are set, ie 0000 0011
It's same as controllers.
Memory Card A as Hex should be 01
Memory Card B as Hex should be 02
Memory Card A and B as Hex should be 03
But, I don't get that. Despite anything I set, it always seem to come out as 03 (both). Sometimes it writes as 01(Slot A Only) but I have not seen a 02 (Slot B Only) value. I am using a Hex Editor to make sure I see the values before my program even touches the file. I have several DTM files that appear to be writing the wrong value.
RachelB wrote:
Really i recommend just reading it directly into the DTMHeader struct, and then you don't have to worry about any of this. Parsing it bit by bit is silly when you can do the entire file at once.
Do the Non-Bongo Controllers have anything special? I don't see anything about that.
Only gamecube controllers, bongos, and wiimotes are supported.
If I set the Controller in any port to anything (Bongo or Controller), The Bongos is also the same value. Controller (GC) in Port 1 means the value is 01 (correct behavior). But, Bongos's Value is also 01 (Incorrect Behavior). The DTM does seem to think Controller = Bongo always. I can confirm this across multiple files.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
After some more work, I have properly read in, the Controller status for Gamecube controllers. Enjoy putting your controllers in any combination of Port(s) and having this program recognize where they are. I am unsure what will happen if you change the controller's status/position. Do so at your own risk.
Memory Cards are read properly but Dolphin appears to have an issue with wrting the correct value. I keep getting the value "3" which means Slot A and B have a memory card. CoolKirby's DTMs prove this is not an issue I created. The file "2-MemcardB-NothingA.dtm" memory card value should be "02" or 0010 but the value Dolphin wrote is "03" or 0011.
Bongos are read like controllers but they ALWAYS appear as controllers even if I don't have a bongo connected. Dolphin appears to be mirroring the Controller value for Bongos.
I still need to look into WiiMote controller info and determine how it's being written and parse it correctly. Probably 16+ values based on CoolKirby's DTM for NSMB Wii. Yes, I will be checking for Extension data and WiiMote position (vertical/horizontal). This will most likely take a while to get properly determined.
I also need to determine how a mixture of WiiMote and Gamecube controllers is written and handle that as well.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Found the bug with memory cards. It happens if you start recording, then stop, then disconnect a memory card, then start recording again. I'll get that fixed.
Seems to be same thing with bongos.
For wiimotes, check http://wiibrew.org/wiki/Wiimote
I also need to determine how a mixture of WiiMote and Gamecube controllers is written and handle that as well.
It's written to the dtm in the order they are polled. Both are polled at regular intervals (1/60 sec for gc, and 1/200 for wiimote), so it shouldn't be much trouble.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
The Wiimote Data also appears to be "stuck" in whatever mode it was initially recorded in. :(
Good news is I determined the values for Horizontal Wiimote and Vertical Wiimote.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Found the bug with memory cards. It happens if you start recording, then stop, then disconnect a memory card, then start recording again. I'll get that fixed.
This+bongos is fixed now, thanks for reporting it.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
RachelB wrote:
RachelB wrote:
Found the bug with memory cards. It happens if you start recording, then stop, then disconnect a memory card, then start recording again. I'll get that fixed.
This+bongos is fixed now, thanks for reporting it.
Can you verify the Wiimote value is being written correctly? I can't seem to get the values correct.
Also, which Version should I be using? Dolphin 4.0-3073 is the current version as of this post.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
RachelB wrote:
What wiimote value?
I am currently trying to parse the number of Wiimotes and their status. Input data is later.
I made a DTM with the Wiimote vertical and one with the Wiimote Horizontal.
The written value was the same. Using CoolKirby's DTM shows a different Wiimote status value. While my DTM files have the same value for Vertical and Horizontal.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
That answers that about Tick. Go figure. It goes crazy high fast.
0x0E5 N/A 12 Reserved
0x0ED Integer 8 Tick Count
I think something is wrong here. Why write 12 bytes of reserved data over the Tick Count's value? Or should that have been written as 0xF5 N/A 12 Reserved?
Typo, forgot to change that when working on the file format stuff.
The error was unrelated to that.
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
RachelB wrote:
I'm not sure what you mean. Nothing about wiimotes is saved to the dtm, except which ones are connected.
What's strange is the NSMB Wii DTM from CoolKirby has the Wiimote data of 0x16. The game expects you to have Wiimote Horizontal (sideways) to play.
My Vertical Wiimote DTM has the data of 0x25.
The problem is, when I make a new recording with the Wiimote set to to Horizontal, my value comes out as 0x25 again. Will this be a problem if I attempt to playback the DTM?
The other issue is the Extension. Is that noted/recorded in the DTM as well? Some games require an extension like the Nuncuk or the Classic Controller. Some games do not WANT any extension attached.
Sample games that require an extension:
Mad World
Pop 'N Music
Sample game that do not want any extension:
Wii Sports (any mini game except boxing)
Sample game that can use, but does not demand a Nunchuk:
Bust-A-Move Bash!
Also, I would like to know what version of Dolphin fixed the Bongo and Memory card issues. Or are the not merged in yet?
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
I'm not sure what you mean. Nothing about wiimotes is saved to the dtm, except which ones are connected.
What's strange is the NSMB Wii DTM from CoolKirby has the Wiimote data of 0x16. The game expects you to have Wiimote Horizontal (sideways) to play.
My Vertical Wiimote DTM has the data of 0x25.
The problem is, when I make a new recording with the Wiimote set to to Horizontal, my value comes out as 0x25 again. Will this be a problem if I attempt to playback the DTM?
The other issue is the Extension. Is that noted/recorded in the DTM as well? Some games require an extension like the Nuncuk or the Classic Controller. Some games do not WANT any extension attached.
Sample games that require an extension:
Mad World
Pop 'N Music
Sample game that do not want any extension:
Wii Sports (any mini game except boxing)
Sample game that can use, but does not demand a Nunchuk:
Bust-A-Move Bash!
Also, I would like to know what version of Dolphin fixed the Bongo and Memory card issues. Or are the not merged in yet?
horizontal/vertical setting is NOT recorded.
16 = 1 wiimote, no gc pads. 25 = 1 wiimote, and a gc controllers in port 1, and 4.
Again, NOTHING about wiimotes are saved to the dtm except which ones are connected.
Also, I would like to know what version of Dolphin fixed the Bongo and Memory card issues. Or are the not merged in yet?
Joined: 3/31/2010
Posts: 1466
Location: Not playing Puyo Tetris
RachelB,
Thanks. I confirmed the merge was done right. The memory cards, the shared Wii/GC Controller status and the bongos behave. Now I need to calculate the values out.
My guess is that Vertical/Horizontal won't be a major factor for editing input as the player knows what's going on. The Nunchuck stuff, will be more interesting to parse out. I wish there was a way to say, "There be a nunchuk here." but 8X Controllers (4GC, 4Wii) = 255. It's maxed out already since it's using a single byte and changing that would break old DTM files horribly.
Coming Soon:
Version 1.3 release with full support for all kinds of stuff. Including GC and Wii Controllers in a single DTM.
Coming after 1.3 release:
Input data and editing.
When TAS does Quake 1, SDA will declare war.
The Prince doth arrive he doth please.
Current:
Rayman 3 maybe? idk xD
Paused:
N64 Rayman 2 (with Funnyhair)
GBA SMA 4 : E Reader (With TehSeven)
TASVideos is like a quicksand, you get in, but you cannot quit the sand
RachelB,
Thanks. I confirmed the merge was done right. The memory cards, the shared Wii/GC Controller status and the bongos behave. Now I need to calculate the values out.
My guess is that Vertical/Horizontal won't be a major factor for editing input as the player knows what's going on. The Nunchuck stuff, will be more interesting to parse out. I wish there was a way to say, "There be a nunchuk here." but 8X Controllers (4GC, 4Wii) = 255. It's maxed out already since it's using a single byte and changing that would break old DTM files horribly.
Coming Soon:
Version 1.3 release with full support for all kinds of stuff. Including GC and Wii Controllers in a single DTM.
Coming after 1.3 release:
Input data and editing.
The wiimote stuff should be done, and it's been on my TODO for an incredibly long time, but i haven't gotten around to it, and it's not exactly high priority, since wiimotes still don't even always sync i believe. Although they sync pretty well now, so maybe it should be higher priority. You can simply parse the input data to figure it out though, for your dtm editor.
