News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: The Console Tool (by Low Lines)  (Read 254620 times)

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #380 on: January 05, 2012, 12:45:56 am »
Ah, I am actually Windows (then XP, now Win 7). I just had a nifty GUI wrap going on that made the system look... not like default Windows :P

I got it running up now. But I couldn't get it to open ANY tlp files which was weird (opened fine with other programs though). I was wanting to go spelunking in the GameCube Fire Emblem game, looking for some unused graphics. Worked fine with some DS games though... hum. Thanks for all the work you've put into this program, it really is spectacularly awesome :)

That's probably because I haven't implemented TPL (assuming that's a spelling error on your part) at this time. If they were supported in the past then I must have missed it :/

Celice

  • Hero Member
  • *****
  • Posts: 646
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #381 on: January 05, 2012, 12:50:24 am »
Ah, alright. So much seemed changed between the two years ago and now I wasn't sure if some of the more generic formats were completely covered yet :)

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #382 on: January 05, 2012, 02:27:00 am »
Ah, alright. So much seemed changed between the two years ago and now I wasn't sure if some of the more generic formats were completely covered yet :)

It's always a challenge to live up to expectations :p This will be included as part of the next release :)

Celice

  • Hero Member
  • *****
  • Posts: 646
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #383 on: January 05, 2012, 02:46:41 pm »
Awesome :p Actually, Would you be willing to try and get this game supported for 3D models? We don't have much data documented on the format (mostly cause we don't dabble in 3D much :p) but we do know how to decompress the package files and what the headers of the files say, like this:

Quote
There's a small trick to converting CMP files. I think CMP is short for compressed package, which should give you a clue. After decompressing a CMP file using the LZ77 compression, it'll become a standard pack (.pak) file, which I think you may have encountered in FE9/10.

I'm not sure if there's any documentation about these pack files, but from memory the format is, using system.cmp as the example (I think American version?) :

first 4 bytes = "pack" in ASCII = 70 61 63 6B
first 2 bytes = number of packed files = 00 10 (i.e. 16)
padding until
4 bytes starting from 0x0C = pointer to label indicating the file's name = 00 00 01 08 (i.e. 108 -> 0x108 -> FE8Data)
4 bytes starting from 0x10 = pointer to the start of the file = 00 00 02 20 (i.e. 220 -> 0x220)
4 bytes starting from 0x14 = size of the file in bytes = 00 02 3D 2C (i.e 146732 bytes, check the unused file FE8Data.bin in the root and size matches)
repeat the last 3 until there are no more files
'Cause we know there's unused models just sitting on there, 'cause we can read the main file with its table naming every internal resource. We just haven't found a way to get at the files yet... :p

I've also seen Square reuse an fbac package format for their Chocobo Fables and Chocobo Dungeon game on the DS... I don't think any of the data inside is especially different, it just needs to be unpacked.


RetroHelix

  • Full Member
  • ***
  • Posts: 148
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #384 on: January 05, 2012, 04:47:16 pm »
It's always a challenge to live up to expectations :p This will be included as part of the next release :)

Are all TPL-formats (and Palette-Formats) supported?

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #385 on: January 06, 2012, 01:35:11 am »
Are all TPL-formats (and Palette-Formats) supported?
Good question! It only supports the formats that are currently supported in BTI files, which are formats I have test files to check against. If people can send me files (basically anything with a generic texture in it, bmd/bti/tpl etc) of any texture formats I don't have, that would be awesome :)

I'll have to check this but for one I know I definitely don't have format type 10 (CI14X2).

[edit]

Awesome :p Actually, Would you be willing to try and get this game supported for 3D models? We don't have much data documented on the format (mostly cause we don't dabble in 3D much :p) but we do know how to decompress the package files and what the headers of the files say, like this:
'Cause we know there's unused models just sitting on there, 'cause we can read the main file with its table naming every internal resource. We just haven't found a way to get at the files yet... :p

I've also seen Square reuse an fbac package format for their Chocobo Fables and Chocobo Dungeon game on the DS... I don't think any of the data inside is especially different, it just needs to be unpacked.

CMP files are simply PAK files LZ77 compressed, notably in Little Endian format. Just out of curiosity to anyone that would know, can compression formats be either endian format, or are they generally not interchangeable?
Also there was a few errors in your format spec, which I've fixed up as below...

Code: [Select]
Square-Enix Package
Byte Order: Big Endian

-- header --
u32 magic "pack"
u16 # files

-- fat * # files --
u32 file type? (0=file)
u32 fnt offset (relative to start of package)
u32 file offset (same as above)
u32 file size (in bytes)

-- fnt --
list of null terminated (00) strings, max length 0x12?
« Last Edit: January 06, 2012, 05:05:17 am by Low Lines »

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #386 on: January 18, 2012, 08:50:29 pm »
since your a digimon fan I wanted to ask if you would ad the digital monsters ver.s format (from the sega Saturn) I know a few Saturn games use it

http://www.gamefront.com/files/21195411/AGUM.rar

Romsstar

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #387 on: January 19, 2012, 05:39:59 pm »
It would be great if you could take a look at the Digimon World 1 (PSX Game) MMD Format of the 3D Models
http://www.fileserve.com/file/DVqMzAQ/AGUM.MMD

It uses a Structure which seems to be similar to TMD but with some additions, which I guess define the positioning of the objects.

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #388 on: January 20, 2012, 12:58:18 am »
Oh, a couple other things I noticed. In Windows, if Console tool is maximized and closed, trying to unmaximize it when it's opened again won't work since it forgets the unmaximized size and does a fake maximized thing.

Also, it's nice how it always seems to know which files to link to in building images out of NANR, NCLR, NCER, etc, but why won't it let me pick any other file? In this particular game's files, the Nitro formats for tilemap, animations, and tiles are used, but the palette one is a raw palette and console tool has no way of associating it with the sprite.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #389 on: January 20, 2012, 02:51:04 am »
Oh, a couple other things I noticed. In Windows, if Console tool is maximized and closed, trying to unmaximize it when it's opened again won't work since it forgets the unmaximized size and does a fake maximized thing.

Also, it's nice how it always seems to know which files to link to in building images out of NANR, NCLR, NCER, etc, but why won't it let me pick any other file? In this particular game's files, the Nitro formats for tilemap, animations, and tiles are used, but the palette one is a raw palette and console tool has no way of associating it with the sprite.

I did not know that with the maximizing thing. I know I remember having a few issues with saving the windows size state, but since I generally don't keep it open long enough nor really utilize max screen (Mac also has sucky maximize functionality), its an easy thing to miss. If you do start to have any issues with display when it opens, deleting the settings.ini will reset everything to default. You can also edit it with a text editor if you are game (though it isn't all that complicated to be honest).

At the moment I think when it loads the Assets panel, it pulls all assets from all open files as well as assets from the same folder, the latter possibly needing said folder open in ctool. Force opening files currently behaves separate from this, in that when you force open a file, it makes a clone of the file handler and applies the new functionality to it, rather than overwriting the original. The main reason for this is in the case a force open fails or you overwrite its original state, you don't have to reopen the browser folder to reset it again etc. Just thinking about it now, adding a refresh but to the file browser would probably be a good idea :) By rights though, if you do force open a file and it opens. It SHOULD become available as an asset in the assets panel for any open files that can use it.

@Romsstar
I have in fact toyed with MMD models myself, however the problem lies in the lack of a specification for the format. I have gotten the model meshes displaying but they are missing the bone data or whatever is used to animate them as this is not part of the TMD format, which I suspect is somewhere else in the MMD file.

@viperzerof-2
Are these sprites or models? At a glance the files appear to have some form of a header, but they also look like raw image data too, but without some idea what I'm looking at they could very well be anything :(

@Update
Lately I've been mostly working on the gui side of things and getting more of that data onscreen. I've started using HashMaps to hold all object variables (and I don't know what I wasn't already), which long story short, makes it much easier to put generic and custom variables specific to certain formats into the gui in an editable manner. I've also moved asset bank switching (ie where you pick which bank you want to display for palettes, images etc) into their own separate panels where you will also be able to access the object data directly. Probably doesn't make a lot of sense, but I think it's a good step in the right direction.

Also in one of my other projects I've been experimenting with LAN Video Streaming through the use of ffmpeg and executing native code. The good news is that means I know how to add support for ffmpeg encoding/decoding, at least as far as the ffmpeg's command line goes. However as I AM running native code to do this, I will have to have a slightly different implementation for each platform although mac and linux are similar so that's a good start :) It's not a pure java solution for opening multiple file formats, which is kind of what ctool was intended to be however it provides a workaround for some of the problems I've had to face with various compressions and what not. Just looking at the internal documentation though, ffmpeg already has support for various game based file formats such as Gamecube THP which is really neat :) Something to look forward to in the next release perhaps if all goes well :)

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #390 on: January 20, 2012, 08:52:19 am »
But does it do the videos from ds games?

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #391 on: January 20, 2012, 08:53:43 am »
But does it do the videos from ds games?
No. And if you want to know why, ask Google.

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #392 on: January 20, 2012, 12:21:13 pm »
Oh, another small issue. Don't know if it's because new features are being implemented but if files are opened in the tool, then closed via the tab's X to the point where no file is open, opening new files will open up the files just closed. Even if a file is closed and the program is open, the opened file can't be moved or deleted because apparently it is still being used.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #393 on: January 20, 2012, 03:52:37 pm »
Oh, another small issue. Don't know if it's because new features are being implemented but if files are opened in the tool, then closed via the tab's X to the point where no file is open, opening new files will open up the files just closed. Even if a file is closed and the program is open, the opened file can't be moved or deleted because apparently it is still being used.
Is this done through using the File > Open or by dragging and dropping files into ctool? I don't think I've tested this specifically but I have made some changes to the opening/closing process, in that for one, until I implement a proper File > Open dialog that action will open up the home folder for the current user in ctool OR the parent folder of the currently selected file if there is a file open. Also there was a bug with the Close All option where it did not actually close all files open. That's been fixed, but maybe that also has fixed the issue you described, I'll have to check that :p

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #394 on: January 20, 2012, 04:15:00 pm »


@viperzerof-2
Are these sprites or models? At a glance the files appear to have some form of a header, but they also look like raw image data too, but without some idea what I'm looking at they could very well be anything :(


ops they are sprites, in a very similar style to digimon world one

Romsstar

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #395 on: January 20, 2012, 07:46:39 pm »
ops they are sprites, in a very similar style to digimon world one

Now this statement can't be right. Sega Saturn never used 3D Models and while technically the GPU of the PSX isn't capable of displaying real 3D it's quite different.

Digimon World used 3D Meshes with applied textures.

Sprites are images with no texture and no meshes.
They are flat.

The sprites you sent are 4bpp linear with a palette that is most likely embed into the SPR file. Although there is a chance that it might be in the .ATR as well.
Uncompressed and easy to figure out, you just need to locate the palette. this could be easily done by checking the savestate of a Genesis Emulator with Tile Molester since the right palette with a very high chance will be in there.

Then you just check whether you can find it in the SPR file or not.

@Low-Lines:
I don't know if the TOD data is contained in the MMD files but the information of the position of the meshes must be in there. In some different kind of format.
What kind of is the difficult task I guess.
The TMD objects bone data or "linking" if you will must be somewhere in the MMD File. I've searched the disc and can't think of any other place it could be.
I mean there is a slight chance that it is in the texture data, but as far as I've checked it contains only multi-clut tim files. There is already a good viewer for that so no need to bother.

But I really would like to view those 3D models (and actually to mess with but hey let's start small)
If I can help you I'd gladly do my part.

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #396 on: January 20, 2012, 08:01:37 pm »
what did I say wrong? they are sprites that look like digimon world one's models the game even uses the same sounds :)

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #397 on: January 21, 2012, 12:26:49 am »
Is this done through using the File > Open or by dragging and dropping files into ctool? I don't think I've tested this specifically but I have made some changes to the opening/closing process, in that for one, until I implement a proper File > Open dialog that action will open up the home folder for the current user in ctool OR the parent folder of the currently selected file if there is a file open. Also there was a bug with the Close All option where it did not actually close all files open. That's been fixed, but maybe that also has fixed the issue you described, I'll have to check that :p
I dragged them in. Maybe you've already fixed it. I'll have to wait and see.

Romsstar

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #398 on: January 21, 2012, 07:13:43 am »
what did I say wrong? they are sprites that look like digimon world one's models the game even uses the same sounds :)

Ah it just feels like looking down on the digimon world models which were brilliant I think.
I'm just a bit sensitive in this point, they never got the Credit they deserved.

It's true the sprites are good work but I think the models in digimon world are much better and can't be compared to Sega Sprites.
But well we have just a different opinion in this point. In any case I meant to help.

As I said try following with Tile Molester: 4 bit linear mode, try a canvas size of 32x16 and 2d-mode, and now all left to do is look for the palette.
Try import the palette from the SPR file and try different offsets. Look through the file and see if you can spot something.

Eventually you will succeed. Once you figured out the settings of the format the rest is just implementing it into Console Tool and I think you can save Low lines some work if you do the Research part :)

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #399 on: January 21, 2012, 07:23:14 pm »
I don't recall insulting the models, I just said they use the same art style or based on the same art work I never intended to elevate one over the other since i've played both for years