News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Ganbare Goemon Gaiden 1 (NES) text compression  (Read 6488 times)

MrFwibbles

  • Jr. Member
  • **
  • Posts: 34
    • View Profile
Ganbare Goemon Gaiden 1 (NES) text compression
« on: March 04, 2012, 06:50:38 pm »
Does anyone know how the text in this game is handled? I tried using the FCEUXD SP emulator along with RedComet's Text Routines guide but didn't really get anywhere with it. I told debug mode to break when PPU address 0x2282 was written to, with the condition A == #ee, and it broke when expected and pointed to this line:

$D6AB:B9 00 02  LDA $0200,Y @ $0206 = #$EE

So I created a CPU Write Breakpoint for 0x0206 (condition: A == #ee), and it broke in the expected place again, but this time at this line:

$CDA5:9D 00 02  STA $0200,X @ $0206 = #$17

I don't understand why it broke when the accumulator value was 17, and looked around for LDA commands or the #$EE value and came up with nothing. Has anyone had any luck in manipulating the text in this game?

Pennywise

  • Hero Member
  • *****
  • Posts: 2250
  • I'm curious
    • View Profile
    • Yojimbo's Translations
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #1 on: March 04, 2012, 07:02:14 pm »
This is one of those games I might or might not get around to hacking eventually, but quite honestly the Gaiden games are a real pain in the ass in typical Konami fashion. There's even emulation issues with the end credits for this game in particular plus both Gaiden games require custom tools to handle the uncommon pointer table format which I've only ever encountered in a Goemon game.

Anyway, both Gaiden games use a huge dictionary to compress the text. I think the technical term for them is pascal strings in that the beginning byte specifies the length and then the dictionary values follow. You need a custom dumper/tools for that stuff unless you want to spend countless hours going through it manually. Unfortunately, I only messed around with Gaiden 2.

« Last Edit: March 04, 2012, 07:21:00 pm by Pennywise »

FinS

  • Full Member
  • ***
  • Posts: 192
    • View Profile
    • nothing much
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #2 on: March 04, 2012, 07:43:28 pm »
Quote
$CDA5:9D 00 02  STA $0200,X @ $0206 = #$17
This line only states that $206 is #17, not the accumulator.

KingMike

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 6905
  • *sigh* A changed avatar. Big deal.
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #3 on: March 04, 2012, 10:37:08 pm »
Right, #$17 is the previous value of $0206 at the time the CPU attempts to write it.
"My watch says 30 chickens" Google, 2018

Pennywise

  • Hero Member
  • *****
  • Posts: 2250
  • I'm curious
    • View Profile
    • Yojimbo's Translations
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #4 on: March 05, 2012, 09:29:55 pm »
Now that I've thought about it some more, both were probably programmed by the main programmer Shinamon(I know he was behind Goemon 2 and did a lot of the programming for Konami's NES titles) so the code and the compression is probably exactly the same or pretty similar. Thus the tools I have will probably work for Gaiden 1 as well. Also just another little tidbit, you're probably dealing with 2 byte dictionary values in the script. Gaiden 2 has the dictionary check go from 68-6B which is like 1000 entries and is absolutely huge. Gaiden 1's gotta be pretty similar I imagine.

So I looked at Gaiden 1 and I believe it uses the exact same compression as 2. I was actually a bit off, but what it actually does is store the lengths in a separate table then it loads the text from another table. Looks like the length table starts at $B91D or 0x392D and the dictionary starts at $BA2B or 0x3A3B.
« Last Edit: March 05, 2012, 10:04:36 pm by Pennywise »

reyvgm

  • Hero Member
  • *****
  • Posts: 1177
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #5 on: March 06, 2012, 08:46:52 am »
Looking at the credits for both Gaiden games and it seems Shinamon was only involved in the first one.

A Japanese dude made a patch that fixed the Gaiden 1 credits for me.

Fei Wong

  • Jr. Member
  • **
  • Posts: 25
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #6 on: March 06, 2012, 02:59:23 pm »
May i have a question? its hard to translate the Snes ones? thats because they never saw a translation? because i dont see translations for those but the ones on NES have more attention.

reyvgm

  • Hero Member
  • *****
  • Posts: 1177
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #7 on: March 06, 2012, 03:49:42 pm »
Apparently they are hard to hack, yes. The SFC Goemon 2 has been in development hell for years now.

Pennywise

  • Hero Member
  • *****
  • Posts: 2250
  • I'm curious
    • View Profile
    • Yojimbo's Translations
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #8 on: March 06, 2012, 07:15:19 pm »
Well I happen to know of someone who is sitting on compression tools for at least one of the SNES games.

KingMike

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 6905
  • *sigh* A changed avatar. Big deal.
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #9 on: March 07, 2012, 01:32:52 am »
Me? I wrote a graphics decompressor for 3, but not recompressor yet.
"My watch says 30 chickens" Google, 2018

Drag0nscythe

  • Jr. Member
  • **
  • Posts: 11
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #10 on: May 27, 2016, 04:52:48 am »
Any chance I can get a hold of the programs your are mentioning. I want to try and dig into the game and try and translate it

cccmar

  • Full Member
  • ***
  • Posts: 243
    • View Profile
    • Nebulous Translations site
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #11 on: May 27, 2016, 06:13:55 am »
Any chance I can get a hold of the programs your are mentioning. I want to try and dig into the game and try and translate it

While you're at it, you may want to ask jink640 about the script. I believe he had it partially translated.
« Last Edit: May 27, 2016, 12:36:51 pm by cccmar »

Drag0nscythe

  • Jr. Member
  • **
  • Posts: 11
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #12 on: May 28, 2016, 09:07:02 am »
Is he still working on the translation?

I haven't heard of anyone working on the translation until you mentioned it.

cccmar

  • Full Member
  • ***
  • Posts: 243
    • View Profile
    • Nebulous Translations site
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #13 on: May 28, 2016, 10:01:21 am »
Is he still working on the translation?

I haven't heard of anyone working on the translation until you mentioned it.

He mentioned it in one of the threads (Script Help and Language Discussion / Re: Script Depository?). You may want to send him a PM and ask about it - not sure if he just dabbles in translating scripts only or hacking as well. Perhaps he could help you out or at least give some useful pieces of advice.
« Last Edit: May 28, 2016, 11:55:58 am by cccmar »

VicVergil

  • Hero Member
  • *****
  • Posts: 715
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #14 on: May 28, 2016, 02:50:51 pm »
Apparently they are hard to hack, yes. The SFC Goemon 2 has been in development hell for years now.

Not really.
Yuki-hime's Disappearance has uncompressed text in all versions, meaning a re-translation is possible.
Shogun McGuiness/Shishi Jirokubei/Kirakira Dochuu have compressed graphics and scattered text pointers, but nothing too wild.
Hoshizora Dynamite (GBC) has compressed text (probably LZ if it's anything like the Hunter x Hunter/Survival Kids GBC games) and some graphics.
Ganbare Goemon 5: Uchu Kaizoku Akokingu (PS1) has uncompressed text and fonts and somewhat weird but not too hard to figure out text pointers, the big problem is the 12x2 font, the 2-byte encoding, and the graphical format.

The main hurdle blocking any translation effort for the SFC games would be the graphical compression (to get the font in).
Said graphical compression was figured out for all four games by various people (with even proof of concept videos and screenshots produced) but they abandoned their projects and didn't release their findings and there was quite a bit of wheel reinventing (for 2 and 3 in particular).

They're not entry level games, but not exactly hard/impossible to mod games to the extent of Sakura Taisen/Tokimeki Memorial fan-translator-nightmare-tier stuff either.
Besides that, there's a lack of interest.
And some have that strange idea that the story is impossible to translate.

jink640

  • Sr. Member
  • ****
  • Posts: 377
  • From nuggets to dust.
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #15 on: May 28, 2016, 09:28:27 pm »
While you're at it, you may want to ask jink640 about the script. I believe he had it partially translated.
I translated the opening scene, but not much else. I actually don't know where I put that script. I had to fill in the japanese text by hand mostly, the script dump was weird and jumbled I think.

cccmar

  • Full Member
  • ***
  • Posts: 243
    • View Profile
    • Nebulous Translations site
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #16 on: May 29, 2016, 04:05:13 am »
I translated the opening scene, but not much else. I actually don't know where I put that script. I had to fill in the japanese text by hand mostly, the script dump was weird and jumbled I think.

Oh, I see - still better than nothing. :D It's actually interesting that nobody ever made a serious attempt to translate this game. I guess it all boils down to the fact that it wasn't deemed sufficiently interesting by the community, or something like that.

Drag0nscythe

  • Jr. Member
  • **
  • Posts: 11
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #17 on: May 29, 2016, 10:28:44 am »
Quite surprised. I keep seeing goemon being referenced as one of the franchises that everyone wants to return.

:D I guess people were thinking someone else will eventually translate it, I can wait.

Can you please point me towards your translation. It should help me in the long run.


Rai

  • Hero Member
  • *****
  • Posts: 631
    • View Profile
    • The official webpage for my Super Robot Taisen Alpha translation project.
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #18 on: May 31, 2016, 03:24:18 pm »
MagniusKun is actually my youtube account.

I posted this video a while ago showing my progress:
https://www.youtube.com/watch?v=FBLOTV8u4O8

The main problem with the SFC Goemon games is the compression.

I actually got pretty far on making tools to decompress and recompress the graphics. However, I was never able to perfect my tools and they still have bugs.

I tested my tools on both Goemon 2 and Goemon 3; so I can confirm that the SFC Goemon games use almost identical compression schemes.

It's also possible to expand the rom and just send uncompressed graphics to VRAM. But I would rather have a true decompressor and recompressor.

The compression in the SFC Goemon games is a combination of LZ77 and RLE. It uses RLE to compress strings of zeroes into one byte. Similar to LZ77, it uses a (length,distance) pair to reuse data within in a 0x400 byte sliding window. If a byte is between 0x80 and 0x9F, it will be followed by uncompressed data. The game decompresses graphics to ram in 0x400 byte chunks.

Other than the graphics; I think the text is compressed too. Not only that; but the pointers in Goemon 3 are scattered.


« Last Edit: May 31, 2016, 03:30:19 pm by Rai »

Drag0nscythe

  • Jr. Member
  • **
  • Posts: 11
    • View Profile
Re: Ganbare Goemon Gaiden 1 (NES) text compression
« Reply #19 on: June 08, 2016, 06:13:17 am »
Sounds quite complicated. Would IDA be useful in locating the pointer? I know that IDA has the ability to decompress psx, GB, and other games, keeping many of the pointers intact. I was originally going to try and use it for the nes but I decided to try my hands at using fcuex first to get an idea of hex code.