News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Compressing Data with RNC ProPackED  (Read 2301 times)

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Compressing Data with RNC ProPackED
« on: March 20, 2021, 04:48:22 pm »
I'm looking for some guidance with respect to compressing data using RNC ProPackED (http://www.romhacking.net/utilities/1310/), specifically when reinserting files back into a PSX image.

I'm currently trying to see what's under the hood of NBA Jam T.E.  I've used CDMage to view and export data files, and I've been able to decompress certain .BIN, .RNC, .GFX, and .COL files within that image using commands in a .bat file that executes RNC ProPackED.

I use the following command in a .bat to reinsert decompressed images:
Code: [Select]
rnc_lib p decompressed.bin compressed.bin -m=2

The headers in the orginial files indicate Mode 2 RNC compression is used.  Upon executing the command, I then take the contents of the .bin file and -- using HxD -- I overwrite the header and image data in the original compressed file at the proper address.  I then import the file into the PSX image via CDMage and run the game in ePSXe.

I'm currently just looking at splash screens in the HICOL.GFX file.  When the game boots up to the Acclaim screen, which is the image I'm messing around with, I get 1 of 2 results:

  • 1) A very garbled version of my image (rare) or
  • 2) A clip of the Iguana screen (instead of my image) with garbled pixels at the top (most common result).

I can see my image perfectly in the decompressed .bin that I've modified; the image appears as a 40x25 tile graphic using a 15bpp RGB codec and 2-dimensional mode in Tile Molestor.  If I attempt to decompress the file that I had used RNC to compress, it gives me an error.  I can also say that the inserted compressed image appears to be a few KB less than the original compressed image that I am trying to replace.

I did a Google search for instructions and found those for the RNC 2.14/2.08 version (like on the Aminet site).  That application, however, seems to have more robust parameters that can be used, and those parameters don't seem to be working with ProPackED.  RNC 2.14/2.08 is also too old to run on my comp at current.

Does anyone use RNC ProPackED?  Any advice on how I should be structured the pack/compress command, or what I otherwise might have missed?



Quick update: I decided to skip a step and used HxD to insert the recompressed image directly into Track1.bin.  The result is the exact same.  I also tried to decompress, edit and recompress another image, this time the title screen.  Again the result is a different screen being pulled, plus garbled pixels at the top.

In each case, despite overwriting the correct section in the game that has the screen I'm trying to change, the game loads the screen that is compressed just prior to it.

Example 1: The location for the Iguana screen is at $1AC9A in HICOL.GFX (the RNC header begins here), just before the Acclaim screen (its header is at $2867F).  When I try to replace the Acclaim screen, a messed up version of the Iguana screen displays in its place.  When it is time to load, the Iguana screen displays correctly, so I know I've replaced the correct section of code and have impacted nothing else.

Example 2: The location for the loading screen is at $0 in HICOL.GFX, just before the title screen's header at $E9C2.  When I try to replace the title screen, a messed up version of the loading screen displays in its place.  When it is time to load, the loading screen displays correctly, so I know I've replaced the correct section of code and have impacted nothing else.



Any ideas?
« Last Edit: April 14, 2021, 01:28:43 pm by eskayelle »
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0

DrMefistO

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #1 on: April 10, 2021, 04:08:50 pm »
Try to use the last build from here: https://github.com/lab313ru/rnc_propack_source/releases
Also, make sure you're not overwriting another data/code with the repacked one.

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #2 on: April 10, 2021, 06:42:23 pm »
Thanks!  With v1.8, I was able to successfully compress, import, and display a replacement for the Acclaim logo, but I came across the same issues as previously reported when following the same approach for the title screen and for one of the player portraits in the game.

Code: [Select]
::rnc_propack_x64 p "decompressed\acclaimscreen(ZZ4).bin" "recompressed\acclaimscreen2xZx2867F.bin" -m=2
::PASS
::rnc_propack_x64 p "decompressed\titlescreenlogo(zz).bin" "recompressed\titlescreenlogoZZxE9C2.bin" m=2
::FAIL
::rnc_propack_x64 p "decompressed\PRESQUAT118colemantest.bin" "recompressed\colemanpresquatx4A785.bin" m=2
::FAIL

I checked all 3 results and can confirm in each case the compressed data I am inserting is smaller in size than the data block being replaced, so I'm not currently overwriting any other RNC-compressed image block.

Anything else I might be missing?  I'm not sure why I was successful with the one compression and insertion but not the other two.
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0

DrMefistO

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #3 on: April 10, 2021, 08:56:45 pm »
Also, your game may use a different dictionary size. Check the readme of the repository for how to specify the dictionary size.
But to identify the size which has been used by developers you must find it in the code, or just bruteforce - at some value it must produce the same binaries as in the game.

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #4 on: April 10, 2021, 11:58:18 pm »
Interesting.  To remove variables, I did a "search and extract" to decompress the image; then I recompressed the extracted file, rather than manipulate it in any way. I've tried about 15-20 different dictionary values (ranging from 0x200 to 0x10000), but in each case the compressed file size was less than when the dictionary parameter was set to 0x0.  When the dictionary is set to 0x0, the compressed byte count is 0xE4 or 228 bytes less than the original compressed data.  I don't appear to be getting much closer to the original compressed size.

Code: [Select]
rnc_propack_x64 p "extracted\data_00e9c2.bin" "recompressed\data_00e9c2.bin" -d=0xF000 m=2
Is there a standard or typical method for finding dictionary size in a PSX rom's code?
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0

DrMefistO

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #5 on: April 11, 2021, 05:51:19 pm »
I can try to find an exact value in the game rom, but it requires a time.

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #6 on: April 11, 2021, 07:11:50 pm »
I definitely appreciate it if you end up finding it!
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0

DrMefistO

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #7 on: April 12, 2021, 07:23:38 am »
By the way, this game is not the first one I saw with a different RNC output algo. Another one for GBA, as I remember.

April 12, 2021, 10:49:31 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Just checked the algo: my program compresses as it must be. I've just checked the repacked file (reinserted it) and it works as usual.
It means, that you reinserting your binaries in a wrong way. Try to fill up the rest of the new packed binary with zeroes to match the original size.
« Last Edit: April 13, 2021, 02:38:05 pm by DrMefistO »

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #8 on: April 13, 2021, 03:47:25 pm »
Hey, DrMefistO.  Thanks for all your help so far.

The result appears to be the same, meaning I'm still doing something incorrectly.  Here's my process.

1) I use RNC ProPackED v1.8 with a .bat file to decompress a specific portrait, Derrick Coleman (New Jersey Nets). 
I first search to verify the address.  My command is
Code: [Select]
rnc_propack_x64 s original\PRESQUAT.GFX
The result I want is at 0x04a785, so I next use command:
Code: [Select]
rnc_propack_x64 u original\PRESQUAT.GFX decompressed\coleman.bin -i=0x4A785
The decompressed file is in the folder as expected.

2) I use RNC ProPackED v1.8 to recompress the same file.  (I make no changes to the image.)  My command is
Code: [Select]
rnc_propack_x64 p "decompressed\coleman.bin" "recompressed\colemanpresquatat4A785.bin" m=2
RNC ProPackED displays text stating that the file successfully packed.

3) In HxD, I do a search of the original PRESQUAT.GFX file and see that the address range from the RNC header to the end of the image is $4A785 to $4B08B.

I copy the full contents of the colemanpresquatat4A785.bin file and overwrite the bytes starting at $4A785 in PRESQUAT.GFX.  From $4B04E through $4B08B, I pad the file with 00s.  I save and exit.

No bytes are added to make the file any larger than it originally was; this is a pure overwrite (a CTRL+B in HxD).

4) In CDMage, I open the .cue file for the ISO and import my modified PRESQUAT.GFX file.  (I select Track 1 and right-click PRESQUAT.GFX.)  Size noted in CDMage of the new file appears to have the same value as the original one.

5) I close CDMage and run the game in ePSXe202-1.  The image is garbled, and upon selecting the team, the game crashes.

If I skip the header and only copy the data (making the header same as in the original PRESQUAT.GFX file), I get the same result.



Is there a different/better process you're using to get the proper result?  Is one of my steps significantly incorrect? 

Is length of file name an issue?  One of my commands or parameters?

Is there a way to directly insert the recompressed image into PRESQUAT.GFX using RNC ProPackedED and not needing HxD?



EDIT: Double checking a typo in the mode parameter (m=2 versus -m=2) that, per Mattrizzle, may not be registering an error in the program but just defaulting to mode 1 compression.  Will report in later.
« Last Edit: April 14, 2021, 07:27:15 am by eskayelle »
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0

DrMefistO

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #9 on: April 14, 2021, 01:34:23 pm »
Ah, yeah, you missed `-` in `-m=2`. Is that helped you?

eskayelle

  • Full Member
  • ***
  • Posts: 171
  • NBA Jam 2KTE / TMNT of Rage / Pong de las TMNT Dev
    • View Profile
Re: Compressing Data with RNC ProPackED
« Reply #10 on: April 14, 2021, 05:14:38 pm »
Precisely.  I'm now getting consistent successful results.  Thanks for your help!
I once wrote a blog.  Maybe you'll find something in it useful?  https://www.romhacking.net/forum/index.php?topic=30593.0