Romhacking.net

Romhacking => Personal Projects => Topic started by: TrekkiesUnite118 on September 15, 2018, 03:12:59 am

Title: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 15, 2018, 03:12:59 am
(https://i.imgur.com/dZrgKcP.png)

Next, using tools made by CyberWarriorX on SegaXtreme, I was able to extract the font for both the Japanese and Korean Saturn versions:

(https://i.imgur.com/bDSQZ3B.png)

(https://i.imgur.com/8ZPrrtr.png)

And I was also able to get the English PS1 font:
(https://i.imgur.com/785QjEZ.png)

Using the tool TileMolester, I was able to stretch the English PS1 font to 16x16, and place it in the Saturn font file in ASCII order. This allowed me to use ASCII encoding to change the text to English:

(https://i.imgur.com/uS7zHdl.png)

Currently, I've been working on two things. First I've been making a Kanji table based on the games Font File. Basically this should at the very least make viewing the script easier in hex editors as the Japanese characters will be displayed correctly. I'd imagine it will also make dumping the script easier. Next I've been figuring out the control codes for the text boxes. So far I've been able to determine the following, which I hope is correct:

x09 = Pause in writing the text. Seems to correspond to pauses in voiced sequences.
0FXX = These seem to correspond to various different things. XX I've seen range from 01 up to 0A.
0F03 = This seems to control the Left Character Portrait.
0F21 = This seems to control the Right Character Portrait.
0A0C 00XX  = This seems to control which portrait is being displayed, and appears to be called when we change portraits as well.

So for example if you see this:

0F03 0A0C 00XX

It means change the Left Character Portrait to portrait XX. I was able to confirm this by changing the values of XX to arbitrary values which gave the following results:

(https://i.imgur.com/oHQ2bij.png)

So the area I think I'm going to get stuck on is the issue of dealing with the font size. I'd imagine some kind of ASM Hack will be needed to change the game to use an 8x16 font so the English PS1 font can be used. That's an area that I really have no experience in and it's simply a skill set I just don't have. There's an old thread over at SegaXtreme that goes into some details on how the Font Compression works, which might also shed some light on how text is rendered:

https://segaxtreme.net/threads/saturn-hacking-notes.4934/

After that however, I think it's just a matter of figuring out the rest of the control codes for the games scripts as well as figuring out formatting of the menu text files. Once that's figured out I think making tools to dump the script and inject new values shouldn't be too difficult to do.

Obviously I don't think this will be a trivial matter, but I figured I'd share my findings none the less.[/list]
Title: Re: Translating Saturn Grandia
Post by: paul_met on September 15, 2018, 05:27:25 am
The Saturn version of the game supports the font 8x16 by default.

(https://s5.postimg.cc/bt63qj7w7/image.png) (https://s5.postimg.cc/z7e32gxjb/image.png)

But this venture with the translation of the version for Saturn I think is useless now. The remaster version of this game will be released soon on PC and switch.
Title: Re: Translating Saturn Grandia
Post by: Tom on September 15, 2018, 08:29:02 am
I think this is still a valuable venture, even with versions on the PS1, PC, and NS. The remasters will probably use the old, censored script, and I think it would be well worth it to see the game running on its original hardware without any censorship.
Title: Re: Translating Saturn Grandia
Post by: kakerasorcerer on September 15, 2018, 09:22:48 am
Really cool work! Saturn version performs much better, so I'd love to play it.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 15, 2018, 10:24:26 am
The Saturn version of the game supports the font 8x16 by default.

(https://s5.postimg.cc/bt63qj7w7/image.png) (https://s5.postimg.cc/z7e32gxjb/image.png)


Oh cool. Was there anything special you had to do when you edited that into the FDATA.BIN file?

But this venture with the translation of the version for Saturn I think is useless now. The remaster version of this game will be released soon on PC and switch.

Eh I don't think so. I mean the remasters, remakes, and re-releases of The Final Fantasy games or Chrono games haven't made translations of their originals useless over the years.
Title: Re: Translating Saturn Grandia
Post by: Tenkarider on September 15, 2018, 10:31:31 am
Just wondering... any chance that this can lead also to a translation of Grandia Digital Museum?
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 15, 2018, 10:55:13 am
Just wondering... any chance that this can lead also to a translation of Grandia Digital Museum?

No idea. I'd have to look at the games files and see if there's any similarities in structure.
Title: Re: Translating Saturn Grandia
Post by: paul_met on September 15, 2018, 11:20:28 am
Oh cool. Was there anything special you had to do when you edited that into the FDATA.BIN file?
I did not edit any fonts.  When I saw the font 8x16 in the boot menu it became clear that the game supports it by default. It remained only to learn the coding  of latin characters.
(https://s5.postimg.cc/sty6qus13/image.png)
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 15, 2018, 12:49:08 pm
I did not edit any fonts.  When I saw the font 8x16 in the boot menu it became clear that the game supports it by default. It remained only to learn the coding  of latin characters.
(https://s5.postimg.cc/sty6qus13/image.png)

Ah ok. So I guess there's a 3rd font set in the game somewhere that's 8x16 then. Do you remember what all you had to change to get the game to use that font instead of the 16x16 one? Or was it just changing the values to the values of the Latin 8x16 font?
Title: Re: Translating Saturn Grandia
Post by: paul_met on September 15, 2018, 02:06:19 pm
Ah ok. So I guess there's a 3rd font set in the game somewhere that's 8x16 then. Do you remember what all you had to change to get the game to use that font instead of the 16x16 one? Or was it just changing the values to the values of the Latin 8x16 font?
The Latin characters has a standard encoding. To display latin 8x16 characters at the beginning of the line should be a byte "03".
Title: Re: Translating Saturn Grandia
Post by: Kallisto on September 15, 2018, 02:44:36 pm
if Grandia Digital Museum ends up being similar that would be a great benefit later on, but to tell you in advance I think the game had some kind of freezing issue that may or may not have prevented English translation at the time.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 15, 2018, 09:12:53 pm
The Latin characters has a standard encoding. To display latin 8x16 characters at the beginning of the line should be a byte "03".

Well that makes things easier to say the least:

(https://i.imgur.com/nHHSq2S.png)

(https://i.imgur.com/fNyOAxR.png)

So really all that would need to be done here is find out the control codes as well as any pointers in the files that determine text position and length. With that figured out putting in the English PS1 script would be rather simple I'd imagine.

Though I do wonder if there's a way to just modify the game to default to the 8x16 font instead of the 16x16 one so you don't have to waste a byte for each line of text with the 03 control code.

EDIT:

So I think I might be figuring out how the script pointers work based on comparing the PS1 versions, and then spot checking with the Saturn version.

So here is what I believe to be one complete text routine from the file BA38.MDT at x05B4:

(https://i.imgur.com/KIbP8AA.png)

This highlighted portion I believe is a header of some kind. It seems to change from text sample to text sample, so there's probably optional values that are in some text boxes but not in others.
(https://i.imgur.com/mPVcun4.png)

Now this next highlighted portion is the important part. This value I believe represents how many bytes until the end of this text box. In this example it's 5A:
(https://i.imgur.com/fh4mRyi.png)

If we then move forward 5A that will bring us to the end of this block of data. The end of a block of text data seems to be tagged with this possible footer:
(https://i.imgur.com/bGJT0aB.png)

After this the next text box starts. I've been able to keep following this pattern and consistently keep finding what appears to be the correct number of bytes and the start of the next text box.

I'm going to keep testing this but so far it's looking promising.

Also, another interesting thing of note is that it seems that the English PS1 versions files seem to be close to if not the same size as the Japanese PS1 files. It looks like Sony removed unnecessary padding to keep the files close to the same size from what I can tell.
Title: Re: Translating Saturn Grandia
Post by: Asaki on September 16, 2018, 03:51:19 pm
This would be awesome, I've always wanted to try this game.
Title: Re: Translating Saturn Grandia
Post by: aorin1 on September 17, 2018, 01:57:38 pm
Yes!
Thanks for starting this project, the Saturn version is the best!
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 18, 2018, 12:38:19 am
Ok, so I think I've cracked the code of the MDT/MDP file headers. This information seems to hold true for both the Saturn and PS1 files, the only difference being that in the PS1 files pointers are Little Endian where as in the Saturn files they're Big Endian.
Some other oddities I'm seeing is that some entries don't seem to make sense as the addresses are beyond the size of the file, but they're still updated in the English PS1 version.


With this info I imagine I should be able to make a program that can do the following:
The hard part to automate in code I'd imagine would be to find a portion of padding in the file that can be removed to keep the file size the same.
Title: Re: Translating Saturn Grandia
Post by: Tom on September 18, 2018, 12:55:43 am
I hope you don't just reuse the PS1 script, but if you do, you should at least redo the lines about "coffee."
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 18, 2018, 01:01:42 am
I hope you don't just reuse the PS1 script, but if you do, you should at least redo the lines about "coffee."

Well my thought was use the PS1 script as a quick and dirty v0.1 kind of release. Just as a proof of concept that it can be done and works. Then if there's people fluent in Japanese who want to step forward and actually do a good translation once there's mature enough tools they can have at it. I did think about using the old Project Alent translation for some spots, but if memory serves correctly that was never 100% complete.

But I'd imagine for at least a v1.0 release using the PS1 Script I could go through and correct some errors and redo things like the coffee lines and what not. Though to be honest I'd probably need some help there as well to identify and correct them.
Title: Re: Translating Saturn Grandia
Post by: Kallisto on September 18, 2018, 03:09:58 am
I was thinking once you do de-censor the script will we see the PS1 script also de-censored since you're working with both versions due to the similarities with the files?

Also will be possible to put in the English Dub too?
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 18, 2018, 07:41:51 am
I was thinking once you do de-censor the script will we see the PS1 script also de-censored since you're working with both versions due to the similarities with the files?

Possibly. I was mainly focusing on the Saturn version but if someone wanted to update the script in the PS1 version it would be pretty easy to do. Though I think this might be already being worked on in the Grandia Redux patch if I remember correctly.

Also will be possible to put in the English Dub too?

It would be possible, but probably a bit tricky to figure out. The Saturn audio is ADX encoded, and it's all in one big file called GM1.STZ The PS1 seems to have this same set up too, but I'm not sure if it uses ADX encoding. My guess would be no though since at the time that was more of a Sega Console specific format until post Dreamcast days. So this file would need to be broken down into it's original individual audio files first. Next would be re-encoding the PS1 samples into ADX and putting the .STZ file back together correctly. That I see being tricky. I have an ADX Encoder/Decoder from years ago when I did music/voice modifications for Phantasy Star Universe, but I have no idea if it will be compatible with Grandia.
Title: Re: Translating Saturn Grandia
Post by: gadesx on September 26, 2018, 10:48:00 am
There is a tool to translate grandia psx version by butz, the spanish translation used that tool.
Maybe can be useful for saturn version.
I have the tool, but requires an archive that I don't have.
The tool have a GUI to translate and modify pointers around the game. If someone have the complete tool I'm interested too.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on September 27, 2018, 09:27:12 am
There is a tool to translate grandia psx version by butz, the spanish translation used that tool.
Maybe can be useful for saturn version.
I have the tool, but requires an archive that I don't have.
The tool have a GUI to translate and modify pointers around the game. If someone have the complete tool I'm interested too.

Well I've been working on making some basic tools of my own. I've figured a bit more out such has how to parse the script out of the MDT/MDP files, how to break down the script file further into individual event sequences, etc. So with that figured out and digging into the files more it's becoming more obvious that the Saturn and PS1 script data is pretty much in the same format, just the non text data is byte swapped due to endian differences.

But if you have the tool I can take a look at it and see what I can figure out from it if you can link me to it.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on February 09, 2019, 02:14:17 am
So it's been a while since I've updated this thread here. I've made quite a bit of progress though. First I've successfully been able to convert the PS1 script over to the Saturn format and have it to work. This has made it so pretty much all the script is translated and for the most part working. An older video of this in action can be viewed here:

https://www.youtube.com/watch?v=lYIlDZtoD7Q

Next, I moved onto the items, menus, and enemies. After some work, I've successfully translated the following:


These have been done by pulling the data out of the Playstation version and injecting it into where it needs to go into the Saturn version. For the items this wasn't too hard as I just needed to put the English data into the file and adjust the pointers. For the monsters it was a bit more painstaking as there was no consistent pattern and all 200+ monsters had to be manually changed.

Menus I was able to somewhat automate it with a tool that looked for pointers/offsets in the BIN files for those screens, and then updated those values based on the new data I wanted to put in to replace the Japanese text. This worked successfully for the most part, with only a few files needing hand adjustments afterwards. I'll be committing these tools to github after I take some time to clean the code up a bit.

Here's some screenshots to show the progress:

(https://i.imgur.com/toJoiMQ.png)

(https://i.imgur.com/lBQLhbI.png)

(https://i.imgur.com/BUgt2Xe.png)

(https://i.imgur.com/aQ72KST.png)

(https://i.imgur.com/6UnqiWh.png)

(https://i.imgur.com/wwPzUpU.png)

(https://i.imgur.com/yeRaJmt.png)


However I have hit a snag. It seems that the English PS1 version created additional MDT files for some areas where the text was too big to fit into RAM. Basically the created a duplicate file from a parent MDT file, and put the overflow text into that one. This new file then takes the place of the original after certain story events have happened.

I've looked into the possibility of recombining these files but I'm not sure if it will work as we may hit the same problem. So I'm curious if it's possible to do the same thing the PS1 version did and create these clone files and find a way to make the game use them. However that's probably going to take some time to figure out.

February 09, 2019, 03:22:42 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Sorry for the double post, but I did some more digging and I have at least found what the game uses to know which MDT file to load when you change areas. At offset 0x06029A00 there's a table that lists all the files on the CD, with a value for what sector it's at, and how many bytes it is. The game references this when it wants to load a new file. How it knows which file to get out of this list though I don't know yet.

However, I have been able to play around and have the game load different areas for me by changing the values for one MDT file to be that of another. So by doing this I was able to go from the Port of Parm to New Parm and test out the magic shop screen:

(https://i.imgur.com/eVJX3l5.png)

Unfortunately this screen got modified in the English PS1 version, so some of those translations either wont fit or don't make sense, so I've had to improvise a bit here. I also tested out the buy screen and found I missed a word in the trade screen:

(https://i.imgur.com/eZIl8Tr.png)

Next, I realized I forgot to post some battle screenshots last night:

(https://i.imgur.com/EFDN2gW.png)

(https://i.imgur.com/5YCciuF.png)

(https://i.imgur.com/LFyxKiq.png)

(https://i.imgur.com/KayNalE.png)

(https://i.imgur.com/4RCGFQg.png)

(https://i.imgur.com/oeiMBFq.png)

Finally, just for fun I took Sue on a date to the Pirate Island. The game went a bit bonkers, I can only imagine because characters that should be in the party weren't there, and characters that shouldn't be in the party were there. The game eventually crashed when invisible Feena tried to speak after Justin spoke on behalf of a mermaid:

(https://i.imgur.com/nUfOc61.png)
Title: Re: Translating Saturn Grandia
Post by: aorin1 on February 11, 2019, 09:57:12 am
 :crazy: Great NEWS!!!
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on April 10, 2019, 02:03:34 am
I realized that I forgot to update this thread. After getting the event flags squared away I have the Disc 1 patch ready for beta testing. You can download it from my github here:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.052.zip

This is a very early release that is intended for Beta testing only. The goal here is to catch any hard crashes or severe bugs in the game, as it's still likely that could happen. While I've caught a few myself, I don't really have the time to fully play through all of Disc 1 multiple times trying various different things. Hence why I'm releasing this early patch.

So if you want to Beta test, download the patch and apply it to your image of the game. I would refrain from burning this to a disc just yet as crashes are still very likely, and I wouldn't want anyone to waste good quality CD-Rs. For now I'd stick to emulators. The emulator of choice for this is Mednafen. It seems to behave the closes to a real Saturn. SSF is a good second, though there are instances where a real Saturn and Mednafen crash, while SSF doesn't.

If you encounter a crash or bug, please provide the following information in either this thread or the SegaXtreme thread:

Aside from the possible crashing, the following issues are known:
Title: Re: Translating Saturn Grandia
Post by: dm0x on April 10, 2019, 11:29:40 am
The english dub likely wouldnt match the retranslated script anyway, I'd just skip on that. If you want the dub, then play PSX, imho.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on April 11, 2019, 01:43:59 am
The english dub likely wouldnt match the retranslated script anyway, I'd just skip on that. If you want the dub, then play PSX, imho.

There's no reason it shouldn't match as the script is pulled directly from the PS1 version. In fact it might end up fitting better than the Saturn Japanese audio does right now as it would have the additional pauses that the PS1 script expects so it can stay in sync.

In other news, I've fixed an issue where some additional text was not translated in the SVLD.BIN file. I've also committed the tool I wrote for translating the values in it. The Overwrite Save? is still in there though as I haven't put any thought into better wording at the moment.

(https://i.imgur.com/dXFYGf7.png)

On final note I have some good news concerning Grandia Digital Museum. Upon peaking into the disc the file structure appears to be the same as the original game, and the file formats look to be similar. So with some luck my tools should be applicable to Digital Museum as well.
Title: Re: Translating Saturn Grandia
Post by: Kallisto on April 12, 2019, 11:21:36 pm
That's great news! Here is hoping the English Dub works out well next. Speaking of the audio format issue, should take a look at what was available at the time the game was made, and narrow it down from there.

Worst case scenario it is completely custom made by the Devs. I was told that sometimes devs do use strange audio formats as a way to mask the real format. I'm not sure if that was a still a practice even way back in the 90s.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on April 15, 2019, 02:10:00 am
I've created a new patch that can be downloaded from github. In it I've included the following fixes:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.53.zip (https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.53.zip)

April 21, 2019, 03:37:00 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
A new patch version is out, this fixes a crash that was happening at the boss of Typhoon Tower. It also fixes some voice synchronization issues in the two intro scenes where Justin and Sue are introduced, as well as when Justin's Mom is introduced. It can be downloaded here:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.54.zip
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on April 24, 2019, 09:00:43 pm
New patch:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.55.zip

This fixes a crash that was happening after beating Typhoon Tower and trying to go back to Dight Village.
Title: Re: Translating Saturn Grandia
Post by: Rolly on May 02, 2019, 10:07:37 am
Fantastic job! I'm happy to download and take tests.
I will also try on real hardwarde.
Im a few days i spread the news about Your project on my portal ;)
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on May 02, 2019, 11:14:31 pm
New Patch:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.56.zip

This fixes an event flag bug in Gumbo that was preventing people from getting to the Twin Towers.
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on May 10, 2019, 07:14:01 pm
New Patch:

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.57.zip (https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.57.zip)

This should fix a hang up that happened when Leen leaves at the end of the Twin Towers. It also translates the end of Disc 1 screen and cleans up the title screen a bit:

(https://i.imgur.com/juDr2W4.png)

(https://i.imgur.com/Jtj59q5.png)
Title: Re: Translating Saturn Grandia
Post by: aorin1 on May 10, 2019, 11:59:00 pm
You're on fire.
Seriously, I'm so glad this is becoming real.
I read above you also mentioned that Digital Museum is translatable with the same tools, great news!
Title: Re: Translating Saturn Grandia
Post by: TrekkiesUnite118 on May 18, 2019, 04:30:01 am
So I now have all the Disc 1 Map Screens translated and running at 704x448. This screen originally ran at 352x448 which caused horrible scaling artifacts in the text drawn by VDP1. I've modified the game code to run this screen at 704x448 so now the text is drawn in it's native resolution:

(https://i.imgur.com/iRRyzT3.png)

(https://i.imgur.com/LJfIjAC.png)

(https://i.imgur.com/5ndQgsh.png)

(https://i.imgur.com/0wcy4QK.png)

(https://i.imgur.com/9GMo00X.png)

These should be in the next patch assuming I don't find some horrific bug from the resolution change.

May 22, 2019, 02:59:46 am - (Auto Merged - Double Posts are not allowed before 7 days.)
I've been able to pull the tiles for the characters names for the results screen from the PS1 version:

(https://i.imgur.com/6TSpmqN.png)
(https://i.imgur.com/CuBayVN.png)

Unfortunately Guido's name wont fit for the space I have available, so if I can't find away to extend it I may have to shorten his name to Gido.

May 24, 2019, 02:16:41 am - (Auto Merged - Double Posts are not allowed before 7 days.)
New Patch:
Disc 1 v0.58 (5/24/2019):

https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.58.zip (https://github.com/TrekkiesUnite118/GrandiaTranslation/blob/master/SS_Grandia_EN_Disc_1_v0.58.zip)

Also with the recent news from beta testers that disc 1 can be completed, I'm going to start focusing on trying to clean up the out of sync audio for the rest of disc 1 as well as any remaining uncompressed graphical text. Once that is done all that should remain is the compressed battle graphics, the FMVs, and the few sequences that are voiced with no text. At that point I'll be shifting focus to Disc 2.
Title: Re: Translating Saturn Grandia
Post by: Lord Igniz on May 26, 2019, 08:15:12 pm
Hey man,

Awesome work, and love that you're constantly updating the project. Grandia is one of the main games I always wanted to play though on my Saturn, but being a JRPG, I felt it would be too difficult to get through. This is also the reason my copy of Grandia II for Dreamcast remanins unplayed, lol.

Keep up the great work, the Saturn community is definitely excited for the work on this classic. Cheers! :beer:
Title: Re: Translating Saturn Grandia
Post by: Special T on May 27, 2019, 12:09:58 pm
Saturn is my favorite console and I'm glad to this game is getting translated. Thank you for your hard work!