News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Translating Saturn Grandia  (Read 2950 times)

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Translating Saturn Grandia
« on: September 15, 2018, 03:12:59 am »
    So a while back I made a topic here about translating Lunar Silver Star Story for the Saturn. I got stuck early on and wasn't able to make much progress. However recently members over at SegaXtreme such as Ms. Tea have been able to make headway due to a somewhat recent iOS port that is based on the Saturn port. So since it's now at a point where there's not much for me to really help with at the moment with that project, I've been snooping around in Grandia's files for the Saturn and PS1. And it's actually proving to be quite a bit easier to figure out so far due to the following reasons:

    • The PS1 version and the Saturn version's files are actually quite similar.
    • The English PS1 version's script is completely uncompressed and in ASCII formatting.
    • The Saturn version's script is also completely uncompressed, but uses a custom encoding format.
    • The Saturn version was somewhat recently translated into Korean which has made it easy to isolate where the script is.
    • The Saturn version has 2 fonts stored in FDATA.BIN.
    • There's an uncompressed 8x8 font that has a full English ASCII font for menus.
    • The rest of the game uses a compressed 16x16 fixed width font. Luckily however members at SegaXtreme made tools to decompress and recompress it years ago. These can be used to replace the font and put it in an order to allow for ASCII formatted text.
    • The English PS1 version uses a fixed width 8x16 font instead.

    The games file structure also seems to be pretty straight forward. The FIELD directory has the following files of interest:
    • Numerous .MDT files - These contain the games script. In the PS1 version they're now .MDP files.
    • ITGET.BIN - I'm guessing this is where the menu text lives. This is probably the equivalent of the ITEM.BIN file in the PS1 version.
    • CMDMN.BIN Again this looks like either Menu text, guessing command related based on what's in it and the name.
    • CNFIG.BIN This again looks to be like menu text, guessing Config menu based on the name.
    • SHOP.BIN This is most likely Shop Menu text by the looks of things.
    • SVLD.BIN Again looks to be Menu text, I'm guessing this is for the Save/Load screen based on whats in it and the name.
    • AMAP#.ADD - I'm not sure what these are, but they were modified in the Korean translation. I'm guessing Map screens?
    • WDATA.DAT - I'm guessing this is Weapons Data.
    Most of these guesses were made by comparing the Korean, Japanese, and English versions.

    So with that out of the way, here's some of more fun stuff I've been able to figure out. First of all, I was able to figure out how at least some of the text encoding works in the Saturn version. Basically the game uses the position in the font table + 31 as the code for the font characters in the .MDT files. And going further, the Japanese PS1 uses this same method. So let's say you want to display the letter A. This is the 11th character in the font table. So to display it, you use the hex value 2A. This is because 11 + 31 = 42, which is 2A in hex. With this knowledge I was able to modify the file BA38.MDT and change some text in the intro cutscene:



Next, using tools made by CyberWarriorX on SegaXtreme, I was able to extract the font for both the Japanese and Korean Saturn versions:





And I was also able to get the English PS1 font:


Using the tool TileMolester, I was able to stretch the English PS1 font to 16x16, and place it in the Saturn font file in ASCII order. This allowed me to use ASCII encoding to change the text to English:



Currently, I've been working on two things. First I've been making a Kanji table based on the games Font File. Basically this should at the very least make viewing the script easier in hex editors as the Japanese characters will be displayed correctly. I'd imagine it will also make dumping the script easier. Next I've been figuring out the control codes for the text boxes. So far I've been able to determine the following, which I hope is correct:

x09 = Pause in writing the text. Seems to correspond to pauses in voiced sequences.
0FXX = These seem to correspond to various different things. XX I've seen range from 01 up to 0A.
0F03 = This seems to control the Left Character Portrait.
0F21 = This seems to control the Right Character Portrait.
0A0C 00XX  = This seems to control which portrait is being displayed, and appears to be called when we change portraits as well.

So for example if you see this:

0F03 0A0C 00XX

It means change the Left Character Portrait to portrait XX. I was able to confirm this by changing the values of XX to arbitrary values which gave the following results:



So the area I think I'm going to get stuck on is the issue of dealing with the font size. I'd imagine some kind of ASM Hack will be needed to change the game to use an 8x16 font so the English PS1 font can be used. That's an area that I really have no experience in and it's simply a skill set I just don't have. There's an old thread over at SegaXtreme that goes into some details on how the Font Compression works, which might also shed some light on how text is rendered:

https://segaxtreme.net/threads/saturn-hacking-notes.4934/

After that however, I think it's just a matter of figuring out the rest of the control codes for the games scripts as well as figuring out formatting of the menu text files. Once that's figured out I think making tools to dump the script and inject new values shouldn't be too difficult to do.

Obviously I don't think this will be a trivial matter, but I figured I'd share my findings none the less.[/list]
« Last Edit: September 19, 2018, 12:12:07 am by TrekkiesUnite118 »

paul_met

  • Full Member
  • ***
  • Posts: 183
    • View Profile
    • Meduza Team
Re: Translating Saturn Grandia
« Reply #1 on: September 15, 2018, 05:27:25 am »
The Saturn version of the game supports the font 8x16 by default.



But this venture with the translation of the version for Saturn I think is useless now. The remaster version of this game will be released soon on PC and switch.

Tom

  • Hero Member
  • *****
  • Posts: 562
    • View Profile
Re: Translating Saturn Grandia
« Reply #2 on: September 15, 2018, 08:29:02 am »
I think this is still a valuable venture, even with versions on the PS1, PC, and NS. The remasters will probably use the old, censored script, and I think it would be well worth it to see the game running on its original hardware without any censorship.

kakerasorcerer

  • Jr. Member
  • **
  • Posts: 7
    • View Profile
Re: Translating Saturn Grandia
« Reply #3 on: September 15, 2018, 09:22:48 am »
Really cool work! Saturn version performs much better, so I'd love to play it.

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #4 on: September 15, 2018, 10:24:26 am »
The Saturn version of the game supports the font 8x16 by default.




Oh cool. Was there anything special you had to do when you edited that into the FDATA.BIN file?

But this venture with the translation of the version for Saturn I think is useless now. The remaster version of this game will be released soon on PC and switch.

Eh I don't think so. I mean the remasters, remakes, and re-releases of The Final Fantasy games or Chrono games haven't made translations of their originals useless over the years.

Tenkarider

  • Jr. Member
  • **
  • Posts: 35
    • View Profile
Re: Translating Saturn Grandia
« Reply #5 on: September 15, 2018, 10:31:31 am »
Just wondering... any chance that this can lead also to a translation of Grandia Digital Museum?

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #6 on: September 15, 2018, 10:55:13 am »
Just wondering... any chance that this can lead also to a translation of Grandia Digital Museum?

No idea. I'd have to look at the games files and see if there's any similarities in structure.

paul_met

  • Full Member
  • ***
  • Posts: 183
    • View Profile
    • Meduza Team
Re: Translating Saturn Grandia
« Reply #7 on: September 15, 2018, 11:20:28 am »
Oh cool. Was there anything special you had to do when you edited that into the FDATA.BIN file?
I did not edit any fonts.  When I saw the font 8x16 in the boot menu it became clear that the game supports it by default. It remained only to learn the coding  of latin characters.

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #8 on: September 15, 2018, 12:49:08 pm »
I did not edit any fonts.  When I saw the font 8x16 in the boot menu it became clear that the game supports it by default. It remained only to learn the coding  of latin characters.


Ah ok. So I guess there's a 3rd font set in the game somewhere that's 8x16 then. Do you remember what all you had to change to get the game to use that font instead of the 16x16 one? Or was it just changing the values to the values of the Latin 8x16 font?

paul_met

  • Full Member
  • ***
  • Posts: 183
    • View Profile
    • Meduza Team
Re: Translating Saturn Grandia
« Reply #9 on: September 15, 2018, 02:06:19 pm »
Ah ok. So I guess there's a 3rd font set in the game somewhere that's 8x16 then. Do you remember what all you had to change to get the game to use that font instead of the 16x16 one? Or was it just changing the values to the values of the Latin 8x16 font?
The Latin characters has a standard encoding. To display latin 8x16 characters at the beginning of the line should be a byte "03".

Kallisto

  • Sr. Member
  • ****
  • Posts: 459
    • View Profile
Re: Translating Saturn Grandia
« Reply #10 on: September 15, 2018, 02:44:36 pm »
if Grandia Digital Museum ends up being similar that would be a great benefit later on, but to tell you in advance I think the game had some kind of freezing issue that may or may not have prevented English translation at the time.

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #11 on: September 15, 2018, 09:12:53 pm »
The Latin characters has a standard encoding. To display latin 8x16 characters at the beginning of the line should be a byte "03".

Well that makes things easier to say the least:





So really all that would need to be done here is find out the control codes as well as any pointers in the files that determine text position and length. With that figured out putting in the English PS1 script would be rather simple I'd imagine.

Though I do wonder if there's a way to just modify the game to default to the 8x16 font instead of the 16x16 one so you don't have to waste a byte for each line of text with the 03 control code.

EDIT:

So I think I might be figuring out how the script pointers work based on comparing the PS1 versions, and then spot checking with the Saturn version.

So here is what I believe to be one complete text routine from the file BA38.MDT at x05B4:



This highlighted portion I believe is a header of some kind. It seems to change from text sample to text sample, so there's probably optional values that are in some text boxes but not in others.


Now this next highlighted portion is the important part. This value I believe represents how many bytes until the end of this text box. In this example it's 5A:


If we then move forward 5A that will bring us to the end of this block of data. The end of a block of text data seems to be tagged with this possible footer:


After this the next text box starts. I've been able to keep following this pattern and consistently keep finding what appears to be the correct number of bytes and the start of the next text box.

I'm going to keep testing this but so far it's looking promising.

Also, another interesting thing of note is that it seems that the English PS1 versions files seem to be close to if not the same size as the Japanese PS1 files. It looks like Sony removed unnecessary padding to keep the files close to the same size from what I can tell.
« Last Edit: September 16, 2018, 02:46:10 am by TrekkiesUnite118 »

Asaki

  • RHDN Patreon Supporter!
  • Full Member
  • *****
  • Posts: 166
    • View Profile
Re: Translating Saturn Grandia
« Reply #12 on: September 16, 2018, 03:51:19 pm »
This would be awesome, I've always wanted to try this game.

aorin1

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: Translating Saturn Grandia
« Reply #13 on: September 17, 2018, 01:57:38 pm »
Yes!
Thanks for starting this project, the Saturn version is the best!

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #14 on: September 18, 2018, 12:38:19 am »
Ok, so I think I've cracked the code of the MDT/MDP file headers. This information seems to hold true for both the Saturn and PS1 files, the only difference being that in the PS1 files pointers are Little Endian where as in the Saturn files they're Big Endian.
  • The header is 512 Bytes in size.
  • The header seems to consist of just a pointer table
  • Each entry is 8 bytes total in size
  • The first 4 bytes seem to be the place in the file where the data lives
  • The second 4 bytes seem to be the size of the data.
  • The entry that tells us where the text lives and how big it is seems to be at x60 every time.
  • There's an entry at x58 that seems to point to a 24 byte piece of data right before the text. Perhaps this is a header or something?
  • Consistently I'm seeing in the English PS1 version that any addresses that come after the address defined at x60 have their values increased by the increase in size of the text data, and any values that come before it are untouched.
  • I'm also seeing again padding being removed in the PS1 files of the exact amount added to the text.
  • Another oddity I'm seeing is that the last entry in the table at x1F8 seems to also have it's size increased by the increase in text size. But I'm not sure what that value represents or what data was added.
Some other oddities I'm seeing is that some entries don't seem to make sense as the addresses are beyond the size of the file, but they're still updated in the English PS1 version.


With this info I imagine I should be able to make a program that can do the following:
  • Parse the MDT files
  • Extract the script portions
  • Put the MDT file back together with a new script portion.
  • Recalculate the pointer table.
The hard part to automate in code I'd imagine would be to find a portion of padding in the file that can be removed to keep the file size the same.
« Last Edit: September 18, 2018, 12:44:32 am by TrekkiesUnite118 »

Tom

  • Hero Member
  • *****
  • Posts: 562
    • View Profile
Re: Translating Saturn Grandia
« Reply #15 on: September 18, 2018, 12:55:43 am »
I hope you don't just reuse the PS1 script, but if you do, you should at least redo the lines about "coffee."

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #16 on: September 18, 2018, 01:01:42 am »
I hope you don't just reuse the PS1 script, but if you do, you should at least redo the lines about "coffee."

Well my thought was use the PS1 script as a quick and dirty v0.1 kind of release. Just as a proof of concept that it can be done and works. Then if there's people fluent in Japanese who want to step forward and actually do a good translation once there's mature enough tools they can have at it. I did think about using the old Project Alent translation for some spots, but if memory serves correctly that was never 100% complete.

But I'd imagine for at least a v1.0 release using the PS1 Script I could go through and correct some errors and redo things like the coffee lines and what not. Though to be honest I'd probably need some help there as well to identify and correct them.

Kallisto

  • Sr. Member
  • ****
  • Posts: 459
    • View Profile
Re: Translating Saturn Grandia
« Reply #17 on: September 18, 2018, 03:09:58 am »
I was thinking once you do de-censor the script will we see the PS1 script also de-censored since you're working with both versions due to the similarities with the files?

Also will be possible to put in the English Dub too?

TrekkiesUnite118

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: Translating Saturn Grandia
« Reply #18 on: September 18, 2018, 07:41:51 am »
I was thinking once you do de-censor the script will we see the PS1 script also de-censored since you're working with both versions due to the similarities with the files?

Possibly. I was mainly focusing on the Saturn version but if someone wanted to update the script in the PS1 version it would be pretty easy to do. Though I think this might be already being worked on in the Grandia Redux patch if I remember correctly.

Also will be possible to put in the English Dub too?

It would be possible, but probably a bit tricky to figure out. The Saturn audio is ADX encoded, and it's all in one big file called GM1.STZ The PS1 seems to have this same set up too, but I'm not sure if it uses ADX encoding. My guess would be no though since at the time that was more of a Sega Console specific format until post Dreamcast days. So this file would need to be broken down into it's original individual audio files first. Next would be re-encoding the PS1 samples into ADX and putting the .STZ file back together correctly. That I see being tricky. I have an ADX Encoder/Decoder from years ago when I did music/voice modifications for Phantasy Star Universe, but I have no idea if it will be compatible with Grandia.
« Last Edit: September 22, 2018, 04:35:50 pm by TrekkiesUnite118 »

gadesx

  • Full Member
  • ***
  • Posts: 220
    • View Profile
Re: Translating Saturn Grandia
« Reply #19 on: September 26, 2018, 10:48:00 am »
There is a tool to translate grandia psx version by butz, the spanish translation used that tool.
Maybe can be useful for saturn version.
I have the tool, but requires an archive that I don't have.
The tool have a GUI to translate and modify pointers around the game. If someone have the complete tool I'm interested too.