News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: The Console Tool (by Low Lines)  (Read 254672 times)

Next Gen Cowboy

  • Hero Member
  • *****
  • Posts: 1766
  • "People are like dice"
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #260 on: December 02, 2010, 12:38:25 am »
Floops ftw! Sorry, it's not as though it's my speciality, but I will always remember that...that...well whatever the Hell that thing was.
"Remember when we were in Japan? You said you were my gun, if you're the gun then that means I'm the bullet."

"All my life I've been waiting for the gunpowder to go off, you know what you need to ignite gunpowder? You need a gun."

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #261 on: December 02, 2010, 05:25:06 am »
There are a fair few XML-based formats to choose from, X3D is the successor to VRML according to Wikipedia. 3DMLW is another (more recent) one. Also I was only a kid in the 90s so I had to google just to figure out what you guys were talking about XD

Azkadellia

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 868
  • 提交向你的情妇!
    • View Profile
    • Princess Translations
Re: The Console Tool (by Low Lines)
« Reply #262 on: December 02, 2010, 05:41:47 am »
If you're going to support the .3ds format, may I suggest this: http://code.google.com/p/lib3ds/ ? It's a library for dealing with the 3ds format.

Awesome tool BTW.
Current Projects: On hold indefinitely.
I do the Twitter thing now: https://twitter.com/MistressSaeko (expect lots of game streaming announcements)
Mistress of the RHDN Discord server.

Lexou

  • Jr. Member
  • **
  • Posts: 9
  • a mantaray
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #263 on: December 02, 2010, 02:20:54 pm »
Wells what would be ideal is if there's one that can be opened by proper 3D editors, or else exporting them to another format would be pointless. The other alternative I can think of is writing a custom I/O plugin for those apps, with possibly a custom bones format or something, though I only know my basic way around 3D editors and have not touched the scripts either, so that would be beyond me. IIRC someone wrote an importer script for gamecube models?

[edit]

The Maya ASCII format doesn't look too hard to write an exporter for, however I think that it is limited to Maya.
FBX seems to be openable by both Maya and 3DS Max, and it does have an ASCII version also. Probably the best option out of what I can find so far. But if people know of better options, I'm all ears...
I suppose what the real question is, which apps do people use? Does anyone use Blender?

OMG I should have seen this earlier. As a part-time modeler, I would recommend The COLLADA .DAE format. A lot of 3d editing programs support it, and/or have other importers for it. It's opensource (I think), and supports rigging, bone hierarchy, jointing, etc... You can learn more about it on their wiki.

Also, all the formats the other members here have posted about (apart from .3ds and .obj, duh) are really uncommon, even though importable in today's main 3d editing apps.

Lastly, are you going to add (no rush at all) import functions ? If so, again, .dae is recommended.

Zerox

  • Jr. Member
  • **
  • Posts: 16
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #264 on: December 02, 2010, 05:55:29 pm »
I'm pretty sure there is lot's of information on Valves SMD model format out there and it's even got bone support. I know of lots of 3D modeling programs capable of using it as well. =D

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #265 on: December 13, 2010, 06:09:02 am »
Turns out this job is a lot more work than I was expecting. Imagine this, IT geek assigned to a computer which only has IE6, AND a ball point mouse. Not a single person at my job knows the first thing about web design, and apparently they have to get some guys from America to do any fancy stuff like Flash because they don't have anyone with enough skill in Australia. So I've been going retro for the past week, reliving my childhood with good old Notepad and Paint XD
And if anyone has heard of SharePoint...It's going to be my buddy for the next 3 months!

Anyways come the weekend, I almost didn't even want to touch my laptop let alone do anything other than watch videos, so I guess this project is on hold for bit until I get a decent amount of this job completed :(

At least I'm getting paid though :)

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #266 on: December 17, 2010, 01:55:58 pm »
Nice work yet again. :cookie:

Here are some suggestions:
  • Set the main window title to something that includes the file name.
  • Implement context menus.
  • Make it a lot clearer how the links between files work.
  • The blue file names need to be explained in the manual.
  • Inactivate/hide currently irrelevant panels.

Fileformats I would like to see:
  • SDAT
  • .arc as in the professor layton games.
  • .arj as in the professor layton games.
  • .vx
  • .pac as seen in Apollo justice.

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #267 on: December 20, 2010, 11:02:31 am »
Yeah, the file links was very confusing. I didn't want to say anything because I thought it was explained somewhere. It seems unintuitive and not like Tahaxan at all.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #268 on: December 21, 2010, 03:37:26 am »
Really?

Wells first off blue files are compressed files (or at least have been interpreted as compressed). It was something present in the older versions of the application, which I had previously explained so it had slipped my mind when I was working on the manual.

As for the file linkage, I honestly thought that was pretty straight forward.

There's an array of file objects which keeps track of all files that have been opened. Whenever you refresh the UI (typically done by selecting the appropriate file tab), it first checks to see if the reference of the file of each asset type (ie Palette, Image, ANimation), still is opened, and if it isn't, it then tries to open the last opened file of that type, or nothing if there aren't any files of that type left. The Assets property, which is located under the Info panel shows you which assets are currently loaded into the selected file. If you click on them, it'll display a drop down menu which you can then switch to another file of that type if there is another one loaded.

In terms of loading multiple files at once, at the moment it won't attach an asset that is loaded after it, (ie NCGR then NCLR will not have assigned a reference of the palette to the NCGR), however as soon as you switch tabs, it will run the above routine again and you should be fine unless you already have a palette opened beforehand, in which case it would already have a reference to that. But again its just a simple matter of swapping the selected asset.

Was this mechanic really that confusing?? If so I'll have another go at explaining it later.

Bear in mind though that the Open File Dialog is only able to load one file at a time as it is just a quick implementation therefore lacks any further functionality. Same goes for the archive viewer, it's only set up to load one file at a time. Context menus are planned, though you should already be able to right click on any image and copy it to your clipboard.

I mostly use the Drag & Drop myself, though I suppose most people are more accustomed to dialogs >_<

Anyways, due to the fact that most people at my workplace are going to be on leave next week, thus limiting the amount of stuff I can get done since I'm still in the process of getting approval/feedback from heaps of people. I've got all of next week off, so hopefully I'll get to sit down and work on this a little :)

@henke37
SDAT is on my list, but if you want the others you've got to provide at least some of the information yourself, or link me to a place that has information about them. I know this sounds kind of lazy, and I'm sure I will eventually get round to looking into them myself since I find it fun, but if your looking for a quick way to be able to accessing custom formats, it'll happen much sooner if I've got the know-how already.

I've had one person (who I think is still wanting to keep what they are doing secret) PM me with their notes on a custom format, and it only took like 10-20 mins to add support for it. That being said, archives are by far the quickest and easiest formats to add support for since there's not a lot of variation in the data structure, but other things will take longer since their data structures are still a work in progress.

Ryusui

  • Hero Member
  • *****
  • Posts: 4989
  • It's the greatest day.
    • View Profile
    • Tumblr
Re: The Console Tool (by Low Lines)
« Reply #269 on: December 21, 2010, 04:13:42 am »
Don't worry, Low Lines, I came clean in the IRC.

It's Solatorobo, and Low Lines helped me immensely by implementing support for its CCB archives (I'd worked out the format, but Low Lines figured out a few extra things I got wrong).

Project is stalled for now; I've got script extraction/insertion going, but of course I'm trying to figure out if there's some way to make the game display its English-language resources. Yes, there are English opening credits tucked away in the ROM, and in multiple flavors to boot.

Back to you. :3
In the event of a firestorm, the salad bar will remain open.

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #270 on: December 29, 2010, 12:54:56 pm »
I have asked the guy who wrote the sprite extractor. He isn't very active and is a pain to contact, especially when the relevant forum currently being down.

The pac files look rather easy in a hex editor. My wild guess is that it is a few bytes of header followed by a pointer/pointer+size table of the contained files.
« Last Edit: December 29, 2010, 08:27:03 pm by henke37 »

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #271 on: December 30, 2010, 01:44:20 am »
@henke37
You are correct. PACs have a simple structure with file offsets rounded up to values divisible by 4. First DWORD is the file count, followed by a FAT table with the file offset and size (DWORD for each). Support has been added for them (tentatively titled Capcom Packages), which will be in the next build.

VX (Magic: VXDS) files are some kind of structured video format (possibly from the Nitro SDK?), however I have tried to look into these in the past and haven't had much success since I'm not that familiar with video/audio formats.

The Layton game I looked at (Diabolical Box), had both ARC & ARJ as well as ARB, but none of these seem to have a very obvious structure to them at a glance.

@Status
I have not had as much focus on this project for my week off work as I had hoped, which I think is mostly due to the festive season and not wanting to get too stuck into anything and then have to drop it again come next Tuesday. That being said, I've been studying Garyong's Huffman code to try and finally understand how the compression works so I can add support for it as well as other compressions that use a combination of Huffman & LZ, which is the case with a lot of common web formats.

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #272 on: December 30, 2010, 05:30:56 pm »
Talking about strange package formats, the hotel dusk package files seem to come in pairs. One is obviously the file table (it got file names, yay!) and the other seems to be the file data (why is the file names repeated in that file as well? beats me.).

But hey, thanks for the effort.

Anyway, is there an option in the GUI to force a file to be parsed according to a certain format? Just in case you end up wanting to support some unguessable "*.bin" format.
« Last Edit: April 05, 2011, 10:21:34 am by henke37 »

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #273 on: December 30, 2010, 07:24:12 pm »
Having a complimenting file isn't that strange at all, I was looking at this iTunes App the other day and every image had a dat file which stored the name and coords of subimages within the full image.

As for a force option, at the moment no, but the idea is that if a file matches several format conditions (such as with BIN being a common extension for multiple files) and there aren't any overriding formats (ie with a matching MAGIC stamp), it will ask you which format to try and load it as. However as it doesn't currently support many files, there hasn't been a need to implement it yet. Forced format loading could also be added as a manual option however I think you'd probably find it to be a lot less useful than you'd think. If a file isn't automatically detected as one of the generic formats, it is most likely a custom one, and since there can be a real variation in how that format is structured, forcing it to open a certain way would be like trying to put an American plug into an Australian power socket. It wouldn't work out too well :p

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #274 on: December 31, 2010, 08:02:15 am »
I just remembered that the Wiibrew wiki has some file format specifications. Implement them as you feel like.

I also recalled that the custom file formats used by Westwood (the C&C guys) have been reverse engineered. I found two links on google. Consider adding it if you don't think it is too out of scope.

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #275 on: January 25, 2011, 11:43:09 am »
you mentioned you wanted to add ps2 support at some point. this guy could help when your ready

http://ps23dformat.wikispaces.com/


also did you ever get tmd working (i have to reinstall java on my pc before I can try)

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #276 on: January 25, 2011, 01:47:02 pm »
@henke37
Before I got into this project, I was totally into the whole Wii Homebrew scene, and I still regularly go there and know of all the file specs detailed on the wiki. I only wish they had more information on the signing process. I've tried to port Segher's code several times, but I'm not big on math anymore and elliptic curve cryptography is bit out of my comfort zone, which has prevented me from my own hacking of savefiles etc.

As for C&C...perhaps a tad out of the current target of this project? I think the main focus at the moment is console games (and probably even more specific, NDS ones) since the resources available for them vary and are often only partially implemented, whereas PC games might even have whole communities devoted to developing new content for individual games. But hey, maybe somewhere down the line :p

@viperzerof-2
Nice, lots of custom formats!! As for TMDs, I'm pretty sure I've got a working version somewhere, however I have not yet even considered adding it yet to the current version. From what I recall though, they don't include bones/joints (or whatever is used for positioning body parts etc), so some models (ie a digimon from digimon world 1), will have have all objects relative to zero, which makes them look like they have been crammed into a suitcase. Will have to refresh my memory to be certain what's the go though when I am able to look at it again.

@Update
Once again, really have not had much of a chance to work on this project since my job wipes out the whole week and I end up spending the weekends recovering (is not made for waking up at 5:45am to catch a 6:40am bus and then another 40 min bus ride out to the site, and then catching that same bus back at 4pm in the afternoon, thus essentially getting home near on tea time and barely having enough time to do anything else but prepare for the next day -_-). I'm 2/3 of the way through it now and after that its back to uni again come late February, early March, so with any luck I'll be able to get a bit of work done before the studies kick after that.

Sorry for that lack of activity guys, but I hadn't even realized it had been a whole month already, that's how full-on this job has been!!

viperzerof-2

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #277 on: January 25, 2011, 03:51:58 pm »
oh you were on wtw right? anyway you project is awesome, though i'm surprised I only just found out about it

AzuShin

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #278 on: February 01, 2011, 02:46:42 pm »
Speaking of models, it would be awesome if Someone could crack Star Ocean 3's file system. Putting it in the disc drive only shows like 3 files whic don't equal up to the game's actual size. Anyway, good luck with the project, I'm looking forward to it.

azoreseuropa

  • Sr. Member
  • ****
  • Posts: 392
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #279 on: February 03, 2011, 11:31:09 am »
Is this tool available for SNES roms too?