The Console Tool (by Low Lines)

A new species of the Slime!

EDIT:  Whoa, this thing can get to be a real power hog.  Both of my processors are being used to the max :O

Yeah, that's probably because it's updating constantly...For whatever reason a simple "render once and not unless it changes" hasn't been my friend (it displays it once then is blank). I'll sort out that problem eventually, for now...it's a beast and that's just how it is :p My laptop is fairly decent though, so i don't really notice it, however I've noticed it also creates a lag with windows explorer.

Are you using it on Linux or something?

[edit]

Looky looky

Now this is just a quick implementation I did partly because I wanted to and partly because loading multiple models (and a map), on the scene is something I'll have to implement eventually anyways. As you can see I am yet to figure out what determines transparency with textures (aka paletted images will always have color 0 as transparent) which is why some of the ground is see-through, but it's a WIP.

I was also looking at using NSBMD courses as a secondary model for implementing 3d maps, has anyone used the NSMB Editor before? It just crashes for me when I try to load the NSMB rom

[edit 2]

Okay I think I've mostly figured out NSBTP files...

 - Each NSBTP can store multiple texture/palette swapping animations
  - Each animation contains all materials that use this type of animation
   - Each material contains a number of frames which makes up the animation for that material
    - Each frame contains the first frame number, texture id and palette id

There's still some data I'm not too sure about, which mostly likely controls how the animation plays back, as this type of animation can also be used for stills like a Thwomp (in NSMB) that changes his facial expression depending on how close mario is to it.
Also an annoying fact about NDS models is that all textures/palettes are referenced by their actual name! Though I suppose it makes some sense in a way

Are you using it on Linux or something?

I'm on XP (the thing that never dies), I just have a front-end GUI running in place of the standard look.

NSBTP files have only 2 bytes per animation that I haven't figured out now. I can successfully open them up with loaded texture/material info and can manually swap frames no problem, but it took a really ugly quick gui do up and it's just a mess -_- So I'm adjusting the GUI a bit so I can add all the different animations into it without clogging up the workspace :p

I've added support for BRRES archives which contain a collection of 3D data files that make up a character or scene. Should be most well known by those who hack Super Smash Bros Brawl, but the format is also found in New Super Mario Bros Wii (though apparently different somewhat). The reason why I'm looking at some of the wii stuff too is because I've noticed that some of the magic stamps for formats are very similar to some of the NDS/GC ones I've been looking at, and there's a better chance I'll figure those ones out if there's documentation I can reference to since Nintendo doesn't seem to change their formats very much between consoles.

Also for those who are interested in Super Smash Bros Brawl models check out this project I came across. It is seriously awesome and it's given me some ideas to work into my own model viewer.

[edit]

Still in the process of adding more support for gamecube models. The entire mesh is drawn now, though I've had to modify the implementation a little to make it work, so I'll have to make those changes to my NDS code too...

Also now because I can load full maps such as ones from Super Mario Sunshine, there was a clear lag and CPU overkill, so I looked around and set it up so it won't update the screen unless something changes. What was most sad is that I discovered a thread which was updating the values of the translation slider guis was using a whopping 50% of my CPU...so I killed that too, and now everything is much faster and isn't constantly putting a strain on the processor. At worse large models take up to about 60% of the CPU when refreshing going by my laptop (before it was actually using like 80% plus *constantly* so its quite significant). I'm still a newbie to threads and such so I'm bound to make more dumb mistakes in the near future :p

Kryal, the author of that SSBB tool is letting me use his documentation on the Brawl models formats, so I'll add them later once I get past the current hurtles I'm facing right now.

[edit]

For those of you who are geared toward PSX hacking. Can you send me some valid TMD models and/or other model formats with documentation preferably. Of all the games I have, only Digimon World provided me with some valid TMDs, but I would like a few more from other games to make sure I have good compatibility. Thanks

Over the last couple of weeks I've been working at basically overhauling my whole project. I did this because it was getting really messy (with code) and I was finding that the GUI format just wasn't working all that well with every new feature added (such as the multiple Model Animations). Another major reason was my lack of understanding on how classes work, as I now think I am better utilizing them than I was before. The code behind the GUI also now (or wells I hope so) follows the standard as mentioned by Nightcrawler. I've also found that some of the built in UIs visually break (such as the Vista one I use), on things such as InternalFrames, so I'm making my own custom skinnable ones using the original classes as a basis.

Major changes, is that the tab system will be restricted to the browser and archive files and everything else will use a Workspace Layout like photoshop (where each file is a separate internalframe, and there are toolbars etc). I've also tried to simulate some of the workspace aspects of photoshop since I think they work really well for this kind of thing. The basic functions are done, I just need to start adding the appropriate toolbars/panels for each type of file and the functionality that goes with it

Atm I'm updating my file loader code to use classes (instead of nasty objects), and add preview/thumbnails to the browser. I start my Uni studies back up in about a week and I'll probably spend less time on this from then on...But hopefully this project will look better and better once I have everything up and running

Can't wait to see what the final version will be like 

Finally after x amount of months I have a few days free where I'm awake enough and not being hassled with all the assessments I have to do at Uni.

Of what I've managed to do over this weekend, I think my new interface is coming along nicely. Most of the work has been improving my file management classes as well as customized GUIs, but as I've had very little time to work on it, I only have a Palette viewer working, which is still very basic in terms of stuff you can do with it.

I'm using the Photoshop interface as a base which I'll customize later to look unique to my app. I'm trying to make sure I keep a clear separation between the interface and the data management, because in one of my classes we were taught so with Web Development, and its honestly a good thing to be doing because it makes it easier to go back and change things without having to locate stuff all over again which I think has been one of my ongoing problems. All data manipulation will be handled by the individual Plugin classes themselves which implement abstract classes for each unique type of file (ie palette, image, sound, etc), that way the interface only need to call something like getColors() and it will parse the file into say an array of 32bit colors etc. The tricky part has mostly been getting the GUIs to display properly which is what burns most of my time. However pretty much once I've made the custom object classes, the latter GUIs will take much less time to make.

Unfortunately starting tomorrow I'll be really busy again with studies so I might not make much more progress on this for a little while yet, but I've actually been using a much older and more stable version to rip stuff from games, which has resulted in how I chose to spend this weekend. I might try and make it more of a habit

[edit]

Oh and I forgot to mention that I've also figured out how to add drag and drop functionality to my application so you can drag any file from your desktop (or where ever) and it will try to load it

Will it ever be possible to edit the model and collision files from SM64DS or any of the other DS games? Because it would be good to be able to create custom levels, edit characters etc.

Project looking really good so far.

Wells at the moment I've been trying to get the GUIs to work right. Having some trouble putting a 3d render panel inside an internal frame, which just seems to vanish when I refresh it. But once I can get the GUIs working I can start adding functionality.

If you've ever used 3D editors before, you should know they are quite complex to use, and most likely even more complex to code. I think basic mesh editing is a nice goal, however more complex stuff would have to be done in one of the proper programs which are built for it. So you would have to export models and edit them and then import them back in.

My studies only finished last week, so I've now got a month off. Don't expect any like overnight updates or anything though, still have lots to do...But with luck I can get some of the annoying GUI problems sorted and get stuck into working on formats again...

[edit]

Figured out what the problem was. Was trying to put a heavyweight component into an internal frame which is a lightweight. It kept disappearing because heavyweights ignore parent components.

[edit2]

Right...now here's a screenshot of where the current interface is at. I totally rewrote my InternalFrame code, making it much cleaner and it also does everything I want it to now! At the moment I am working on building the Toolbar interface which will allow multiple docking on all sides of the workspace as well as floating toolbars. As you can see that's mostly implemented already at least for the layout part as I might leave drag and drop functionality to later since I'm much more keen to get back to adding file format support. I still have to setup a method that will allow actions to be performed when buttons are clicked however I've already got it figured out in my mind so that shouldn't take too long. Once I can start adding actions, it should start to get interesting.

At this point I have several format type frames working including the palette viewer, graphic viewer, model viewer and an initial start on my hex viewer. The model viewer uses a lightweight component as opposed to the heavyweight one that was used in the old implementation, so its works a lot faster and I don't have to do much in terms screen updating since it does it all by itself (mostly), making it less cpu hungry. I haven't updated my NDS and GAMECUBE model codes to my new file handling structure, but I did up basic PLAYSTATION tmd models and they display fine I've also started making my own GUI icons (see the bottom of the screenshot), since I can't find any they show what I want these buttons to do.

Will post more once I get actions up and working.

This is just exceptional work.  Fantastic job on this, absolutely loving it.

GUI Actions seem to be working alright now, they can be called from pretty much anywhere and don't require much work to change etc. Non-generic toolbars such as the one in the screenshot below affect each separate opened file, so if you say want on Model file to display wireframe, it will only set wireframe for the currently selected window. Also toolbars that aren't used by the window type are hidden every time you switch windows, so it should help keep stuff neat and tidy
I've fixed a bunch of bugs in the file browser, which actually was loading a folder several times every time it was refreshed -_-
The archive window is now fully working, and loads inner archives really really fast since all the file scanning is only done once each time the source file if opened.

My main focus at the moment is coming up with a decent model structure class that I can use across as many different formats as possible like I've been doing with image formats. The trouble is there is quite a bit of variation in how people like to store their 3d models and stuff, which means I have to study them more to get a better understanding
So I've been looking at Autodesk's 3DS format, which I have a Zelda TP pack for to experiment on. With that, basic support for TGA images has been added too

The last thing I've been doing was making a reboot on my documentation. Since I did a Web class this semester at uni, I've gotten a bit more wise with HTML & CSS, so I've used that to build an interface for my docs. It works like a Directory tree with inner nodes able to be expanded and hidden which is much easier to navigate than with crappy tables a mile long. The docs themselves are kept in XML format and loaded in via Javascript though only because I don't need to upload the files to view them, will make a PHP version once I decide to do so. It makes use of a bunch of neat CSS3 properties and displays alright on Safari and Firefox (sorry IE is a loser). Much easier to maintain and I think looks cool

[edit]

I've come up with a fairly generic Model Data Structure which is based on how gamecube models are packed. Basically in terms for NDS models, the polygon data is broken into four levels...
 - polygon
  - shape (ie quad, quadstrip, triangles, etc)
   - vertices (just a group name for the attributes below)
    - attributes (ie position, normals, scale, restoreId, color, etc)
Not all vertices will have every type of attribute, which can be easily checked upon rendering using struct styled classes (this is java after all).

Each time I reimplement my NDS Model code, it always seems to be a little different from the last time >_<
The above was one particular model that really hated me in all my older implementations, and now it just works...A lot of other funky models now also display properly too however some of the ones I had gotten perfect now have some spastic vertices.

[edit 2]

Found the problem. VTX_DIFF wasn't being added to the vertex list.

[edit 3]

I am now able to get nearly every single NSMB character model displaying properly now! Models with Joint Scaling and Rotation, or funky stack IDs are the only ones that don't display properly, which covers pretty much all Zelda game models -_-

Gamecube models have also been updated to the new format structure, however they still require a lot of work.

The Model GUI has gone through a bunch of improvements, including basic lighting (or at least what is supposed to be lighting) and Perspective Transformations using the mouse buttons (ROTATE, MOVE, SCALE). Toggles for most of the interface have been added too.

A preliminary Key Listener has been added as well to allow for shortcuts etc in the future

NDS Roms are back and files can be exported from archives.
Fixed a bug that caused multiple reads per file while checking for compression, folders load quicker now.

I still need to build some properties windows to display stuff and create a resource manager so files can be linked with external files, such as external textures. Currently it will load the last texture selected by default if no texture is found within the model file.

[edit 4]

While trying to figure out joint rotation in NDS models, I decided to have a go at drawing the joints on screen to help with debugging. I was kind of surprised at how easy it was to do and so now the model viewer is one step closer to total awesomeness

The joint rotation data is made up of 8 signed decimal values, which I figure sit in top right hand corner of the 4x4 matrix stack. Looking at "bomb_menbo", I've managed to get his left legs looking kinda right, but I'll have to do more trial and error to figure out how exactly these values are stored...

Although I haven't looked at them yet, the joint scaling is made up of 6 signed decimal values, however I have seen some impossible data (ie really really big values) pop up in these slots so I'm not completely sure on them yet...

[edit 5]

I figured it out!!! Each joint had an used word which turns out to be a type of scaling factor. I only figured this out because I came across a model in NSBMD where the bird's wings had been obviously flipped in the model editor and hadn't had their transformations reset (for those not familiar with modelling...sorry). Each wing had an inverse scale factor so when I inputted that into the Rotation matrix...It displays properly

So in simple terms the 8 rotation values + the scaling factor are put into a 4x4 matrix like so...Note that for whatever reason when people represent a 4x4 matrix it is shown as top-bottom, left-right.

Code: [Select]

s = scale factor (signed)
r = rotation[8]

[    s, r[2], r[5], 0]
[ r[0], r[3], r[6], 0]
[ r[1], r[4], r[7], 0]
[    0,   0,    0,  1]
I'll assume that this scaling factor affects all joint data, hence why alot of models aren't sitting in their bounding boxes. Going to test that assumption out now...

[edit 6]

Since figuring out how joint rotation data is stored, a very large number of models now display properly. The above is a Ben 10 model, which I've also pasted what comes up in KiwiDS's NSBMD model viewer so you can see the difference in accuracy

I haven't figured out how the joint scaling data is used. It's made up of 6 values, which is twice as many as any typical scaling (aka one for each axis)...however it seems to have little visual impact on how accurate the models look so I'm not that worried.

Pretty much all issues remaining lie in the bones data...There's still a few commands that aren't properly implemented if at all, plus there's still the issue with objects overwriting used stack locations due to wacky stack IDs...The funny thing is that KiwiDS's model viewer doesn't seem to face this problem and yet I've gone over his source code and cannot see how he implemented it so I'm guessing I have an older version of the code or something :/

[edit 7]

Lol this is turning into a long post...Anyways figured out how to get models with wacky stack ids to display right. You keep a copy of each joint matrix in a separate stack and when each polygon is loaded you "restore" that joint matrix to the main stack, thus allowing multiple joints to use the same area in the stack...It was kind of obvious now that I think about it, however the fact was that all restore calls from vertices reference to the main stack and not the joints themselves. Zelda models now display well too now, though ironically Link is among the few that DON'T display properly yet -_-

7 edits is enough for one post I think :p

Most of the wacky models have a very unordered bones list, often jumping between joints and polygons and all what not. So its kind of obvious that the ordering needs to be retained for rendering. With that pretty much every model displays with correct geometry now, except a few with unknown bones commands such as 0x9. The "lamb.nsbmd" model from Harvest Moon: Sunshine Islands is one example that has like a dozen 0x9 commands in its bone list.

Most models from the Zelda games work too now

I still have to figure out what's contained within material data, along with figuring out how to get alpha blending working right. See with the dog's cheeks, if you turn on transparency, you'll see they have a gradual softened edge, however it will totally disregard polygons behind it and only display the background and the grid in the transparent area. Its more than likely that each material has their own alpha blending options.

This tool is really fascinating. I have had trouble using it, though. Like, how is the model rotated? The actions are slow for me.

But I would really like it if the nsbmd files contained in the ZIP download could show up properly. It's coming really close but a huge texture gets in the way most of the time.

Here are the files.
http://www.mediafire.com/?uvk1e2oy2wq

This is what it's looking like for me.
http://i55.photobucket.com/albums/g144/mega_rock_exe/lostbn5netmapct.png

First of all that demo link in my signature is over 6 months old. The build I'm working on at the moment is like completely re-written and way better than that shitty demo I put up. You should be able to tell the interface in my newer screenshots looks totally different to the one you are using.

Interestingly though those models you linked me (which you should remove due to copyright) have some scaling issues when I view them, with some objects and platforms being drawn outside the map (assuming its not supposed to do that of course).

My uni studies start back up in 3 weeks so I'm considering posting another demo so people can test out the new interface and give me feedback on the model viewer, however there are still a few things I want done before then.

As can be seen in the above screeny, I've been slowly working on interfaces that will allow for more manipulation options and hopefully a start on actual EDITING of model stuff. The upper window is basically your Selector and Show/Hide Toggler...Selecting stuff makes wireframe and vertex points and joints change color, and hiding stuff makes them very transparent. In future this interface would also be done using mouse input on screen however I'm not too worried about doing that as of yet.

The second window is my start on an Attribute Editor. It's based on the Netbeans GUI property interface which I thought was a really nice clean way to display information, and is very easy to manually edit values via keyboard input, since it uses the JTable interface (something I didn't know existed until late last night -_-)

Aside from a handful of bug fixes, the last thing I want to have in place is an interface for animations, which has been the main thing holding me back from making a start on adding support for animations. Since figuring out the rotation data for joints I've got a much better understanding of the NSBCA format, I just need a way to load them into the model viewer so I can start debugging.

I'll probably have to start writing a User Manual with in all the controls, since there's starting to be quite a lot of them :/

[edit]

Here's a screenshot of the character model from those files (466.bmd) showing selection/show/hide in action

Just wanted to stop in and say, this is still quite fucking awesome