News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: The Console Tool (by Low Lines)  (Read 252688 times)

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #80 on: July 01, 2010, 04:25:39 pm »
Very interesting.
Thanks Low Lines! ;)

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #81 on: July 02, 2010, 08:07:30 pm »
Wells here's the first bit of actual model editing! Albeit VERY basic and limited to manual input, at least its something. The only problem is that I don't know how to pull stuff from a 4x4 matrix (like a rotate X value), and several bits of data in NDS model joints are stored as 4x4 matrices :(

« Last Edit: November 17, 2010, 08:26:19 pm by Low Lines »

Friedslick6

  • Jr. Member
  • **
  • Posts: 22
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #82 on: July 04, 2010, 05:31:29 pm »
Oh my :cookie:, this looks so good. Like, Tile Molester only molests tiles. But this... molests everything. That doesn't sound good, but it certainly is.
I've already managed to rip DS sprites from Mario Kart DS using the heavy version (mind you, using print-screen. Will there be an export/convert feature?)
Anyways, you're a genius. Keep on geniusing, please.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #83 on: July 04, 2010, 08:15:06 pm »
I've already managed to rip DS sprites from Mario Kart DS using the heavy version (mind you, using print-screen. Will there be an export/convert feature?)

XD you poor soul.

I'm still trying to decide how I want to display/edit certain file types.

Take Model Animations...first off there are at least three kinds, you have geometry animations that affect how a model is drawn on screen, material animations that I think manipulate how textures are skinned onto models, and texture animations which swap between multiple textures such as blinking eyes etc. Now each of these have fairly different data structures, and they all can be animating at once, and by themselves there is little to no other way of displaying that data other than in a table/list which isn't very fun to edit!! What I think I could do instead, is have a resource manager in the workspace that holds all assisting file types such as animations, palettes, mapping etc, and when you load such files into the workspace, it adds them to the resource list and can be accessed from all windows currently opened. So if you want to edit a palette, you'd be editing it in an attribute panel as opposed to its own window. It's going to take a little reworking to do this I think, but it makes more sense to do it this way rather than have X number of windows open to achieve one outcome such as a map or a sprite sheet.

I've been studying the NSBCA format heavily over the last few days trying to figure it out. I've determined that the first offset in a joint animation contains Pivoting data, and the second section contains Rotation data. Rotation keyframes are called when the second byte equals 0 and Pivot keyframes are called when it equals 128, however the data itself is definitely stored a little differently to how it is stored in NSBMD files. Scaling keyframes hold two scaling values for whatever reason, with Translation keyframes being just straight values that can be a signed word/dword.
Each animation has a frame length, however each object in an animation seems to have a start and end position that typically don't match up with the total number of frames. Now I haven't had much experience with model animations, but I'm guessing this might have like a decay effect on the animation if anyone has ever played around with animations in Maya? The trickiest part is calculating the right number of keyframes stored in each object as there isn't an actual value written down anywhere. From what I can tell it's calculated based on a bunch of things including the difference in object frame length over animation frame length, the rate in which the object (and possibly the animation) plays back as well as rounding to the upper whole number. This looks a bit funky and overcomplicated but so far all the animations I've been working with calculate the correct number of keyframes, so I'm assuming I'm on the right track. Am yet to get to the point of loading animations into a model, but hopefully I won't get some spastic result :S

If anyone's interested, here's my notes which I've been keeping while studying the format..So far I've only worked on small files as it takes a while to highlight all the data of complex animations in a hex editor. These are all animations found in NSMB. Sorry if my notes aren't very easy to understand :(

Code: [Select]
Object Flag: --zyx-Sr-RZYX-T-
> found in the header of  each object of an animation
===========================
T - has Translation keyframes (0 Yes| 1 No)
XYZ - flags for Translation attributes
R - has Rotation/Pivot keyframes (0 Yes| 1 No)
r - flag for Rotation/Pivot attribute
S - has Scale keyframes (0 Yes| 1 No)
xyz - flags for Scale attributes
===========================
if T-XYZ = 1
> Fixed Translation value (signed dword)
if R-r = 1
> Fixed Rotation/Pivot value (dword/2*word?)
if S-xyz = 1
> Fixed Scale value (2*dword)

Note: The below is only done when the bit flag equals 0 for that attribute (TX, TY, TZ, R/P, SX, SY, SZ)

a|b = datasize = playback speed
> a & b are flags stored in the object header for each attribute

Translate
-----------------------------------------------------------
2|0 = word = 1/1
2|1 = word = 1/2
2|2 = word = 1/3
0|1 = dword = 1/2

Rotate
-----------------------------------------------------------
0|0 = 2*byte = 1/1
0|1 = 2*byte = 1/2
0|2 = 2*byte = 1/3
> byte0 = index
> byte1 = 0 Rotation | 128 Pivot
> not completely sure on rotation yet

Scale
-----------------------------------------------------------
2|1 = 2*word = 1/1
2|1 = 2*word = 1/2
-----------------------------------------------------------

Attribute    [animation flag|start|end|a|b] bytes/keyframes - animation length
> bytes - the actual size of the data stored
> keyframes - bytes/datasize(see above)

Basabasa - 0 Pivot - 33 Rotation
===========================================================
Translate [3|0|34|2|1] 38/19 - 36 Frames
Rotate [3|0|34|0|1] 38/19 - 36 Frames
Scale [3|0|34|2|1] 76/19 - 36 Frames
===========================================================

Basabasa2 - 1 Pivot - 20 Rotation
===========================================================
Rotate [3|0|78|0|1] 82/41 - 80 Frames
===========================================================

Bilikyu - 31 Pivot
===========================================================
Translate [3|0|58|2|1] 62/31 - 60 Frames
Rotate [3|0|58|0|1] 62/31 - 60 Frames
===========================================================

Donketu - 12 Pivot / 20 Rotation
===========================================================
Translate [3|0|10|0|1] 24/06 - 11 Frames (dword)
Translate [3|0|10|2|1] 12/06 - 11 Frames
Translate [1|0|02|2|0] 04/02 - 02 Frames
Rotate [3|0|10|0|1] 14/07 - 11 Frames
===========================================================

Gesso - 39 Pivot
===========================================================
Translate [0|0|16|2|2] 14/07 - 19 Frames
Translate [0|0|16|2|2] 12/06 - 18 Frames
Translate [0|0|12|2|2] 10/05 - 14 Frames
Scale [0|0|16|2|2] 28/07 - 19 Frames
Scale [0|0|16|2|2] 24/06 - 18 Frames
Rotate [0|0|16|0|2] 16/08 - 19 Frames
Rotate [0|0|16|0|2] 16/08 - 18 Frames
Rotate [0|0|12|0|2] 14/07 - 14 Frames
===========================================================

TTL Bird
===========================================================
Translate [1|0|05|2|0] 10/05 - 05 Frames
Rotate [1|0|05|0|0] 10/05 - 05 Frames
===========================================================

Gamaguchi - 11 Pivot - 52 Rotation
===========================================================
Translate [3|0|08|2|0] 16/08 - 08 Frames
Translate [3|0|07|2|0] 14/07 - 07 Frames
Translate [3|0|16|2|0] 32/16 - 16 Frames
Rotate [3|0|08|0|0] 16/08 - 08 Frames
Rotate [1|0|07|0|0] 14/07 - 07 Frames
Rotate [3|0|16|0|0] 32/16 - 16 Frames
Scale [3|0|08|2|0] 32/08 - 08 Frames
Scale [1|0|07|2|0] 28/07 - 07 Frames
===========================================================

drowssap14

  • Guest
Re: The Console Tool (by Low Lines)
« Reply #84 on: July 06, 2010, 05:43:40 am »
Model editor/viwer looks good.
Will it support .kcl collision data - I think mariokart ds and sm64ds use them?

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #85 on: July 06, 2010, 02:52:03 pm »
Model editor/viwer looks good.
Will it support .kcl collision data - I think mariokart ds and sm64ds use them?

Considering I've only gotten models displaying and haven't even looked at collision stuff, not to mention the fact that Super Mario 64 DS models use a much older version of NSBMD that's only partially supported...That'd be quite a feat anytime soon :(
Interestingly though you got me looking at Mario Kart DS which in turn led me to a few discoveries. For starters, there was a bug in my NARC reader that didn't take into account that subfolders CAN be empty (it sounds like a stupid error, but due to the way NARCs are read, its an easy mistake to make).
Another thing I discovered in another game I was checking (Goldeneye - Rogue Agent), it turns out NSBMDs can sometimes not contain joints and material data...whether that means that the bones don't rely on them or that what I was actually looking at was a partial file that loads on top of a common NSBMD with the necessary data, I'm not totally certain, but it definitely puts a spanner in the works so to speak -_- If it is the latter, then there's all the more reason to follow through with my Resource Manager idea so that you can actually view these "special" cases...

[edit]

It seems NSBMD files can ALSO not contain polygons, or rather polygon data seems to be under something else :/ There is also definitely no obvious common file for Goldeneye models, so I am guessing that these files must use some unknown data blocks that haven't been documented yet by anyone.

[edit 2]

The model header is 12 bytes shorter in Goldeneye Rogue Agent NSBMD files, therefore it clearly used an earlier version of the NSBMD format. All the necessary sections are in the file, its just that the Joint section was indexed incorrectly because it starts directly after the model header.


[edit 3]

Heh...while mucking around with the model header I realised I had totally overlooked a few bytes. It turns out the proper model scaling was stored in them, so now those Megaman/Rockman maps display properly now as well as a bunch of other models (particularly with maps). Still haven't got the bounding box right though...

« Last Edit: November 17, 2010, 08:28:06 pm by Low Lines »

Koetsu

  • Jr. Member
  • **
  • Posts: 45
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #86 on: July 07, 2010, 12:21:45 am »
Wow! That's awesome stuff there. Great work! The map is resembling the 2D version...
There should be a boat but I can't see it. Also, it seems like something is wrong with the building behind the pink building. There should be a texture with words above the door.

One more thing, if you're looking into the NSBCA motions format, I believe some of the character files I sent you may have animations associated with them. It could be an older format though.
« Last Edit: July 07, 2010, 01:34:10 am by Koetsu »

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #87 on: July 08, 2010, 03:09:47 pm »
Typically moving objects such as boats are loaded ontop the map in the same manner in which 2D sprites are loaded into 2D maps. Unless this "sign" on the pink building is in the textures, it's probably an object loaded onto the map as well. I haven't added the ability to access internal textures in models yet so I can't check that.

Have figured out NSBVA (VIS0) files now. They store a bit array of joint visibility for each frame of an animation, it allows you to toggle parts on and off though it is clearly under used in most games. There are only 3 models in NSMB that use them, being Koopa Jr, the Spiny Boss and the normal walking Koopa. Koopa Jr for example often has instances where his apron and bandit mask are on or off, whereas normal Koopas toggle their inner body and shell so both a shelled koopa and a shell-less koopa use the same model. There's still a few bugs with properly toggling said joints however as other joints often use the same stack causing them to disappear as well.

I've chosen to separate the File Browser from the main window making it a popup window as I found it quite annoying having to change tabs back and forth between it and the workspace constantly. I've also decided to put floating toolbars/menus into separate top-level windows so they aren't limited to the dimensions of the workspace, and by default the first opened file will be loaded maximized rather than little tiny windows. Am mostly doing an overall clean up of the GUI code before I start building more menus such as the resource manager....

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #88 on: August 23, 2010, 11:49:52 am »
This is very cool. I am looking forward to this. But I have to report a bug from the version I downloaded yesterday. When trying to open the sdat from Apollo Justice, I get an error tree view instead. The error is:
Code: [Select]
Error reading contents:
java.lang.ArrayIndexOutOfBoundsException: 6

And I have to say that the pac files used by the same game are looking rather interesting. A quick glance at them tells me it likely begins with offsets to sub files. And the stored data looks compressed to me.

At least I know that the .vx files are video. Maybe something to put on the low low priority wish list?

Anyway, I will be keeping an eye on this, it's interesting for sure.

DaMarsMan

  • Hero Member
  • *****
  • Posts: 1288
  • Bring DQV
    • View Profile
    • DQ Translations!
Re: The Console Tool (by Low Lines)
« Reply #89 on: August 23, 2010, 11:51:09 am »
Just stopping in to say I'm very impressed with the ground breaking work you are committing to.   :cookie:

Zerox

  • Jr. Member
  • **
  • Posts: 16
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #90 on: August 25, 2010, 10:49:51 am »
I love this tool. It's the first thing I've found that displays models from Kingdom Hearts 358/2 with accuracy. I was wondering though on your project page of your site it mentions an export feature yet I can't seem to find it. not even for textures which it seems would be easiest. I would really appreciate it if you could tell me where to find it and if it exists.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #91 on: August 25, 2010, 06:24:17 pm »
It falls under plans and future implements. My site hasn't been updated in yonks...in fact when that site gets updated (as in total overhaul and redesign to new web standards) that's kind of when I can start working on this project again. I've also been doing a Java unit this semester and there's already a bunch of new things I didn't know about that you can do with it too in including shortcuts and better use of inheritance. I have a feeling what I will have learned by the end of my unit is going to greatly improve my programming skills, so I'm like quite stuck into it right now. It's nearly mid-way through my semester with one more week next week then I get a week off, so hopefully I can spend a bit of time on hobbies again *yay*

Also I really appreciate the support I keep getting off and on as time goes on, even after x-number of weeks/months with no updates someone suddenly digs up this thread and says "hey we're still here!!" lol It's probably half the reason why I have stuck to this project for so long and intend to actually go somewhere with it eventually.

Also I'm pretty damn sure I promised I'd have a new demo up for people to play around with before I started my studies...It's a little late, but here you do :p

Console Tool Demo #2

This is the last working build I have on my computer so it shouldn't crash much :p It will look like it can open a bunch of files, but it's mostly only able to handle a handful of archive types and model files. The real point of this demo is to demonstrate the new interface. I've tested it on both a Vista and an XP and it runs fine on them, won't work on a Mac though and I doubt for Linux either as I haven't been exporting different OS builds yet.

Have fun everyone!

[edit]

I haven't actually touched the code in over a month due to uni commitments, so my memory is a little fuzzy on where its at, so keep that in mind :p
« Last Edit: August 26, 2010, 01:52:17 am by Low Lines »

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #92 on: August 26, 2010, 07:31:58 am »
"a handful of files" -> no bitmaps, pallets, sound or such things

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #93 on: August 26, 2010, 07:36:21 am »
...it's mostly only able to handle a handful of archive types and model files. The real point of this demo is to demonstrate the new interface.
I think I was quite clear...

It can read other stuff just there's no interface to manage it at the moment so there's really no point and saying they work.

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #94 on: August 26, 2010, 09:39:06 am »
I understood you fully, I just clarified it.

BTW, I took the liberty of submitting your fileformat doc. Talk to the staff to get your forum account linked up with the submission.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #95 on: August 26, 2010, 09:42:53 am »
I understood you fully, I just clarified it.

BTW, I took the liberty of submitting your fileformat doc. Talk to the staff to get your forum account linked up with the submission.
Tis cool, I just get a few replies questions off and on so I misread.

Zerox

  • Jr. Member
  • **
  • Posts: 16
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #96 on: August 26, 2010, 02:35:26 pm »
Awesome a new demo. This version has a save feature easily visible. But it doesn't appear I can save a model still. Is there any chance in your next update that this will be possible? I'd love it even if all I could export to was .obj or something else that was simple.

Friedslick6

  • Jr. Member
  • **
  • Posts: 22
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #97 on: August 30, 2010, 04:04:21 am »
Vast improvements to the original make me happy :) . But would you by any chance remember how to zoom in on models in the newest release?

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #98 on: August 30, 2010, 04:12:12 am »
Vast improvements to the original make me happy :) . But would you by any chance remember how to zoom in on models in the newest release?
It works on a 3 button mouse system. Left button controls rotation, right button controls translation (up and down), and the mouse wheel is scale. Its down by holding the button down and dragging your mouse left or right, similar to Maya. It was set up to use the mouse wheel scrolling however sometimes caused an overflow of some sort so I disabled it for now.
« Last Edit: September 08, 2010, 06:36:49 pm by Low Lines »

Zerox

  • Jr. Member
  • **
  • Posts: 16
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #99 on: September 01, 2010, 01:35:22 am »
Hey Low Lines I'm curious will DirectX support ever be something added to your program rather than openGL?