News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: The Console Tool (by Low Lines)  (Read 252690 times)

Aurelazza

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #300 on: April 08, 2011, 08:01:31 pm »
Wow, what an amazing-looking program!

I came close to posting this question on the Newbie board, but in the end it's directly related to Console Tool. Suppose that I wish to export a Nitro Graphic Resource image (in this case, a Pokemon White trainer sprite) from Console Tool to Photoshop or a similar program. I know of the Copy Image function, but once I've copied and edited the sprite in question, I'm ignorant as to how to make it viewable in Console Tool again. Would simply compressing it in an LZ77 format do the trick? If not, how might I re-import an edited image or directly export one instead of using the Copy function?

Many thanks!

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #301 on: April 08, 2011, 08:15:10 pm »
It's still not quite there yet.

Palettes are the only thing that can really be converted between formats at the moment. The copy function is just so people have a way of ripping images until I get a proper implementation working.

That being said the type of task you want to do probably requires being able to edit multiple files that make up the complete image and that's even more functionality than just converting a flat image from one format to another. I am working on it but it's unlikely there will be a working solution any time soon.

Sorry to disappoint :p

Aurelazza

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #302 on: April 08, 2011, 08:29:34 pm »
Bummer =(

Forgive me for asking in the wrong place, but might you know of any tools capable of doing such a thing?
« Last Edit: April 09, 2011, 09:44:19 pm by Aurelazza »

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #303 on: April 10, 2011, 05:32:44 am »
@Aurelazza
Here's one alternative that's been mentioned in this thread...
Even though it can't export/import graphics yet, you can use it to narrow down the search for the graphics you need to edit.

My usual method is to extract the whole ROM to a directory structure using ndstool (and the same to reassemble the ROM for testing) and edit individual files using Tile Molester.
In general you will have to get your hands dirty to pull off any significant level of hacking (ie hex editing, writing your own code to do stuff, etc).

Aurelazza

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #304 on: April 11, 2011, 09:52:06 pm »
Gotcha, thanks Low Lines!

Edit: I just realized that Console Tool might be able to help me with another issue that I've been pondering over for quite some time. Has the map-editing aspect of CT progressed to the point of being able to insert small things like chairs, trashcans, or shrubberies into an area in games such as Pokemon White?

If it has, then might you kindly point me in the right direction as to get started?If it hasn't, then what sort of shortfalls are currently in the way?

Thanks again.
« Last Edit: April 13, 2011, 03:37:18 am by Aurelazza »

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #305 on: April 19, 2011, 12:27:31 am »
Hi there Low Lines,
I admit that I am lazy to read all 21 sites so post first and check later and sorry if I noticed something well known or ask something answered many times.

3 thing in my mind:
- it seems that your hex viewer misses the first 5 bytes of a file. Here u see CT on top and Hew workshop below with the same file open:


- is there a "in-deep" manual for CT??
-what is the best way to view the NDS graphic formats?? What is the right sequence if everything in 1 file??

Thanks alot

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #306 on: April 19, 2011, 12:59:51 am »
First off, the most likely case with missing bytes is that the file is being decompressed automatically (ie LZ77 first 4 bytes is the compression header and the fifth being the start of the compressed data). Normally this would be indicated with filenames in the browser being colored blue however the current browser won't indicate that. Secondly the hex viewer is just a viewer, you can't use it to edit files yet just in case you weren't aware.

The only manual at the moment is whatever documentation that is included in the zip folder. If you have suggestion on where and/or what should be more detailed, let me know and I'll try to improve it for the next version. When you design an application you tend to not think everything needs explaining since you know how it all works, so it always helps to have another person look at it and point out what they don't understand.

Best order of opening associative files like graphics is opening them in reverse based on their relationships.

NANR
 needs NCER
 needs NCGR
 needs NCLR

NCER
 needs NCGR
 needs NCLR

NCGR
 needs NCLR

therefore the best order in which to open them is
NCLR, NCGR, NCER and then NANR

though version 3.0 should automatically reorganize its assets upon selecting an opened file, which can also be changed via the "Info" tab under "Assets".

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #307 on: April 20, 2011, 01:14:36 am »
Umm, can an overlay on the NDS be compressed?? There is not the typical x11 on that file either.

I saw that u can write in the hex viewer but i believe nothing happens to the file...right??
Nothing would happen is I start corrupting an NCGR in your viewer while opened in CT..right??

My suggestion (for now) is to add this little pieces of advices u gave me to the doc in the zip.
I never used before I opened this files.

CT will reorganize its assets upon selecting an opened file but only if it countains only one complete set of format and not if the file countain multiple NCGR (for example)..right??

Is there an import/export function of those formats planned in one of the next versions of CT??

Thanks



Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #308 on: April 20, 2011, 03:04:03 am »
There a multiple flags in a file header that might trigger file decompression of multiple types (0x11 is just one example), and currently there is no way to manually turn this off. There is no way for ctool to recognize what rules apply to a file unless it provides some kind of header in the file it recognizes. As far as its concerned, an exported overlay file is just some unknown binary file.

The hex viewer is is just loading the binary data into the fields. After that it's just text on the screen, you mangle this in any way you like and it will have no effect on the file.

You can have as many files of the same data type loaded at the same time. Every time a file is reloaded in the viewer it will first check to see if it has an asset for each association (ie an NSCR has two assets, NCGR and NCLR), then it will check to see if that asset is still opened, if it isn't it will try to open the last opened file of that data type. If there aren't any files of that data type opened, it will try displaying the file with dummy information, for instance NCER should just display numbered squares representing oams etc

Import/Export is planned with palettes being the only CURRENT data type that can be converted between supported formats in the current version. As to when it depends on when I have added support for other formats that they can be converted to. The main thing that holding up support for formats such as PNG/APNG/GIF (which would be the most ideal formats for exporting NDS graphics/animations), is the compression/decompression routines, which aren't my strong point. Java does have its own APIs for handling images however it has a limited level of control over the exported file.

You can also copy whatever graphic is loaded in the viewer to the clipboard by right-clicking and selecting "Copy" from the popup menu. This is just a basic implementation to allow graphics to be exported until a proper one is done.
« Last Edit: April 20, 2011, 05:18:45 am by Low Lines »

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #309 on: April 20, 2011, 01:52:09 pm »
Understood.

I know of only one game where there is need to mess with the overlays and that is "Ace Attorney Investigation: Miles Edgeworth" where the text is stored in the overlays.

I will play with those NDS graphics. I think the best is to split my files into pieces and then open them up one by one.

Saddly i can't help u with programming...not my strong point either :(

Thanks

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #310 on: April 22, 2011, 04:45:07 pm »
There seems to be some sort of rendering bug with NSCR files, they show pure green in spots where it is not supposed to do that. It only happens to some files.

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #311 on: April 22, 2011, 05:43:37 pm »
That means it ran out of palette colors (as in indexoutofbounds) during rendering. It should only happen when you try to match up assets that weren't meant to be used together. You can always pm me the files if you want.

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #312 on: April 22, 2011, 10:09:41 pm »
Ï am not sure it's only that ... look:



but here ok:



the 2 button look like this:
« Last Edit: April 22, 2011, 10:17:38 pm by Auryn »

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #313 on: April 23, 2011, 12:54:17 am »
Yeah I've been looking at that exact bin archive myself...The cause is most likely the palette offsetting in both NSCRs and NCERs, as when just viewing the NCGRs they render okay. Will look into it once I get up to reviewing my implementations of those formats :p

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #314 on: April 23, 2011, 06:38:23 am »
Auryn, What exactly is the problem? The button loads fine. Did you temporarily forget the drop down menu in the second toolbar?

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #315 on: April 24, 2011, 03:12:05 am »
Somebody from the french translations said it's really saved like that and not an error of CT.

My question to be sure it's like that, is simple:
On the NCER screen u have those 4 rectangles, the NCGR data doesn't have to match those rectangles...right??
It's not that in the NCGR there has to be a piece exactly big as rectangle 03 (for example) but the can be 2 pieces (example an upper and lower half) that compose that rectangle 03..right??

The more we dig on this game, the more it get's better :p (is this a double screen image??)


or even better (double screen and bigger in width to allow a side scrolling??)


No NCER inside that directory, just some NSCR but none seem to work with this 2 images.
 
For now we are lucky that those images don't need to be translated.


KC

  • Full Member
  • ***
  • Posts: 209
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #316 on: April 24, 2011, 03:26:25 am »
Umm, can an overlay on the NDS be compressed?? There is not the typical x11 on that file either.

Overlays are completely defined in the Overlays Source area of the NDS rom. That's usually extracted as some small .bin file. Every overlay has 32 bytes there.

Code: [Select]
typedef struct sOverlay {
   int OverlayId;
   int RamAddress;
   int RamSize;
   int BssSize;
   int StaticInitStart;
   int StaticInitEnd;
   int FileId;
   int Flags;
} tOverlay;

Flags decides whether the overlay is compressed or not.

Code: [Select]
int GetOverlaySize(tOverlay& Overlay)
{
   if (Overlay.Flags & 0x1000000)
   {
      return Overlay.Flags & 0xFFFFFF;
   } else {
      return Overlay.RamSize;
   }
}

If 0x1000000 of .Flags is set, then the overlay is compressed with the typical backwards LZSS stuff (pretty much like the BIOS one with a different header, just in a different direction). The lower 3 bytes of .Flags then point to the end of the file where it can start decompressing.
If you clear 0x1000000 and provide an uncompressed overlay, it will work just fine. The compression is optional and completely defined with this structure. As far as I'm aware, all NDS games support it.

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #317 on: April 24, 2011, 03:07:53 pm »
Oh thanks KC, even if it's too late because we solved.

About those pics, Low Lines, it seem they are just with the wrong size ( i don't know if from header in the files or error in CT).

Low Lines

  • Full Member
  • ***
  • Posts: 173
    • View Profile
    • lowlines
Re: The Console Tool (by Low Lines)
« Reply #318 on: April 24, 2011, 09:23:45 pm »
They're images which are meant to 512 wide but instead are being rendered as 256. Since NSCRs work with tiles, that's how you get that stripy look. 256x256 is like your normal drawing area, when the image is bigger than that, it has to get drawn a little differently. I've NSCRs that store each 256 block after each other, but that one is clearly doing the entire row sequentially :p

[edit]

Ok with regards to the coloring error (where it displayed green), that happened because I hadn't included a conditional check to see if the Map (or rather the NCGR) was 4bit or 8bit. By default it opens as 4bit, which means it only uses 16 colors instead of the full 256.


Gyakuten Kenji 2 (BOXJ) - contents/com/logo.bin/2.nscr

Am looking into the other issue now...

[edit]

Looking at the specs on GBATek, it looks as though the internal screen size can be set to different sizes...

The unknown value in the NSCR map appears to hold different values for the mucked up images, though I've also noticed that NCGR with no obvious NSCR assigned to them are also affected by this internal sizing thing. What I've done for the time being is add a few if statements that will use a bigger internal size if the image is bigger than normal...


Gyakuten Kenji 2 (BOXJ) - contents/com/upcut.bin/406.ncgr

[edit2]

It looks like some of the NSCRs use a weird mapping layout...


com/upcut.bin/94.nscr

The upper image is what the NCGR looks like (which is drawn row by row), and the lower image is what the NSCR looks like. It only seems to affect graphics wider than 256px so it's likely an error in my rendering code.

[edit3]

Appears as though the problem has been fixed...


Gyakuten Kenji 2 (BOXJ) - contents/com/upcut.bin/94.nscr

Even though it's repeating the image twice, that can't be an error in ctool because the index is counted in the loop (not by using math to figure out the index). I went and tested a few other big NSCRs to make sure it was working...


Digimon World (ADNE) - contents/dat/BTMAP/10as.nscr

Note: The above is a before & after.

I've also uploaded a modified build with these fixes applied to the project page. Keep in mind that this is using the old gui since I'm still working on the next version which has the more stable interface.

Let me know if you find any NSCRs that still display wrong...Also it would be helpful to include the location of the file like I've started doing in my posts...

ie Game Title (Game Id) - filePath

« Last Edit: April 25, 2011, 06:27:46 am by Low Lines »

Garyong

  • Jr. Member
  • **
  • Posts: 15
    • View Profile
Re: The Console Tool (by Low Lines)
« Reply #319 on: April 25, 2011, 10:04:24 am »
[snip]

I've also uploaded a modified build with these fixes applied to the project page. Keep in mind that this is using the old gui since I'm still working on the next version which has the more stable interface.

Let me know if you find any NSCRs that still display wrong...Also it would be helpful to include the location of the file like I've started doing in my posts...

ie Game Title (Game Id) - filePath
I hope I'm not being picky, but the latest version (3.1a) cannot load any file (trying to do so results in a null-pointer (and subsequent errors));
Code: [Select]
Exception in thread "AWT-EventQueue-0" java.lang.ExceptionInInitializerError
        at util.gui.CDragDrop.drop(CDragDrop.java:279)
        at java.awt.dnd.DropTarget.drop(Unknown Source)
        at sun.awt.dnd.SunDropTargetContextPeer.processDropMessage(Unknown Source)
        at sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.dispatchDropEvent(Unknown Source)
        at sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.dispatchEvent(Unknown Source)
        at sun.awt.dnd.SunDropTargetEvent.dispatch(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processDropTargetEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$000(Unknown Source)
        at java.awt.EventQueue$1.run(Unknown Source)
        at java.awt.EventQueue$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
        at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue$2.run(Unknown Source)
        at java.awt.EventQueue$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
        at util.FileAccess.read(FileAccess.java:53)
        at util.struct.DPath.read(DPath.java:190)
        at util.struct.DPath.read(DPath.java:149)
        at util.Plugin.scan(Plugin.java:80)
        at util.struct.DPath.check(DPath.java:62)
        at util.struct.DPath.<init>(DPath.java:31)
        at util.struct.DPath.<init>(DPath.java:22)
        at util.struct.DPalette.<init>(DPalette.java:9)
        at ctool.Loader.<clinit>(Loader.java:14)
        ... 33 more

Also, if you truly want large NSCRs to test with, try the ones found in Izuna - Legend of the Unemployed Ninja (AHSP) -> contents/data/gra/village/
(eg; Dungeon01BG1.NSCR, which is 1280x1280)

Apparently the internal screen size functions as something to make large, composite tiles. When loading a large NSCR, a sequence of those internal screens are filled with those tiles. To display the full NSCR, those screens should be laid out in a standard tiling fashion (row by row). At least, that is how I got the example image from Izuna given above.
Note that only the first 2 bytes of the 4 labeled as 'Unknown' in your documents indicate the internal screen size; the other two are supposed to be the 'BG Type', which I think only indicates if the game should rotate/scale it at some point or not.


Also, regarding the compression of 'overlay' files; The format is indeed like the built-in LZ with 'type' 0x10 read backwards, but has a different header as well as a slight modification to the algorithm.
The 'header' format is:
Code: [Select]
{
    u8* padding; // 0xFF-s
   
    // length of compressed data. May be less than the file size (minus header size).
    // Same format as the decompressed size in the 'normal' compressed file formats.
    'u24' compressedLength;
   
    u8 headerSize;
   
    // The size of the decompressed file is the original file size plus this value.
    u32 extraSize;
}
The data is located in front of this 'header', so 'extraSize' is the last 4 bytes of the file. When 'extraSize' is 0, the rest of the header is absent.
Only the 'compressedLength' bytes directly before the header are compressed data, anything before that is uncompressed.

The algorithm is slightly modified in the sense that the 'Disp' given in the compressed blocks is 2 less than the 'Disp' that would have been given for the regular algorithm (so just add 2). It also appears to occur regularly that the 'Disp' is far too large for the amount of data that has been decompressed up to that point; in that case I've found that setting it to 1 gives results that appear to be correct.
« Last Edit: May 02, 2011, 08:02:51 pm by MathOnNapkins »