Romhacking.net

Romhacking => ROM Hacking Discussion => Topic started by: DarknessSavior on January 15, 2010, 03:48:24 pm

Title: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 15, 2010, 03:48:24 pm
I'm interested in attempting to at least hack some basic text into this game. I was easily able to make a table file for the PSX version of the first game, yet this one proves more difficult.

I've already tried loading the entire ISO (only ~330 MB, so no big deal) into WindHex and using an SJIS table, no results. I opened it in Monkey-Moore, no working results (both with normal searching, and with wildcards in case the kana are 2 bytes each). I asked KitsuneSniper for help, who searched for the byte combinations of some SJIS values, and he could find random kanji (which to me, seems like random data rather than actual kanji), but nothing else.

Can anyone offer some insight as to some other methods to try to get table values?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rolen47 on January 16, 2010, 03:12:54 am
Only 330 MB? That doesn't sound right. Isn't that game supposed to be over 800 MB?

Edit:
I took a quick look at the game, and it looks like there are 2 bigfiles called "gsd1.bin" and "gsd2.bin" in the ISO and they are compressed   :'(. My guess is the text is in those files, and you can't do anything to them unless you figure out how to decompress and extract those archives.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 16, 2010, 11:00:32 am
Only 330 MB? That doesn't sound right. Isn't that game supposed to be over 800 MB?

Edit:
I took a quick look at the game, and it looks like there are 2 bigfiles called "gsd1.bin" and "gsd2.bin" in the ISO and they are compressed   :'(. My guess is the text is in those files, and you can't do anything to them unless you figure out how to decompress and extract those archives.
Well, truth be told I took a .cso that I had of the game and turned it into an .iso using a program I found, but I had the compression setting up as high as it goes, so it actually came out smaller than it was when it was a .cso. Interesting, huh?

Yeah, I noticed those files, too. I also noticed some .src files, which KitsuneSniper tells me are a dyslexic way of writing SCRipt, one for each game. Could be in there, too. I just need to find table values and then I can start tearing through files to find the text.

Edit: Well, I managed to make a table using the old-fashioned RPG method (naming the main character something, moving the save file somewhere, naming the opposite thing on the next file, and comparing the two, rinse and repeat). The table has the same values used over for various characters, or so I thought at first. Apparently, 81-83 and BC are used as control codes to determine which set of characters to use (81 is hiragana alone, 82 is the end of hiragana and katakana, 83 is katakana, and BC is English upper-case only). However, it seems as if the script is compressed. I checked both the ISO in it's entirety, as well as the various suspected script files. I wasn't even able to find any small words, let alone the script. >_<;

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Maxim on January 20, 2010, 01:51:39 am
Files are compressed with gzip algorithm. You can cut some compressed file from archive and open it in WinRar.
Hope that'll help you to find a script.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 20, 2010, 03:24:53 pm
Really? I can just try and open the .bin files with Winrar? Interesting. I'll give it a shot!

Edit: I tried doing so, and Winrar just said they were damaged files or files of an unknown type. Should I try renaming the files or something?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: KaioShin on January 20, 2010, 04:09:11 pm
The files inside the archieve are compressed with that. You have to extract them from the package first. To know where one file starts and ends you need to figure out the file format first. Should be easy though, standard VFS fare.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Maxim on January 21, 2010, 01:57:21 am
KaioShin said the right thing. :) Format of this archive is plain and easy.
It's just an archive with many gzipped files in there.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 21, 2010, 11:18:15 am
The files inside the archieve are compressed with that. You have to extract them from the package first. To know where one file starts and ends you need to figure out the file format first. Should be easy though, standard VFS fare.
So, what would I use to decompress these files? >_>;

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rolen47 on January 21, 2010, 12:48:28 pm
Well here's what you can do (this probably isn't the best way, but it works):

Open gsd1.bin in WindHex.

Search for the hex value: 1F8B08  (gzip files always start with this header)
Write down the offset (it's 14000)
Search for hex value 1F8B08 again.
Write down the offset (it's 10E800)

Now, Edit -> Dump Binary Data
Enter file name: va1.gz
Start Offset: 14000
Ending Offset: 10E7FF (you should subtract 1 from the offset you wrote down.)

and that will extract the first compressed file. Then you can use 7-Zip (http://www.7-zip.org/) or WinRAR to extract the gz file.





Then to extract the second file you do pretty much the same thing:

Write down the offset (it's 10E800)
Search for hex value 1F8B08 again.
Write down the offset (it's 1C8800)

Now, Edit -> Dump Binary Data
Enter file name: va2.gz
Start Offset: 10E800
Ending Offset: 1C87FF (you should subtract 1 from the offset you wrote down.)

Now just keep doing that for all the files. I think there's 226 files in that 1 archive... Errrr hope you don't go insane extracting them all. If you were a programmer you wouldn't have to do it this way.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 21, 2010, 01:13:12 pm
While that would be extremely helpful if there were only like, 10 files, like you said, there's 200+ files. Very insightful, though. Learn somethin' new everyday.

Is there some program I could use to detect each file and extract it?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: KaioShin on January 21, 2010, 01:39:08 pm
Is there some program I could use to detect each file and extract it?

Write one. Seriously, extracting achieves is a basic romhacking skill required for almost all disc-based systems. And it's really not that hard. It's the perfect exercise for one's first "serious" program. All you need to finish such a program is basic program flow control and basic file I/O functions (the later can be looked up in the documentation of your given language quickly).

Don't say you couldn't do it until you tried, it's seriously easy.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 21, 2010, 01:50:32 pm
Is there some program I could use to detect each file and extract it?

Write one. Seriously, extracting achieves is a basic romhacking skill required for almost all disc-based systems. And it's really not that hard. It's the perfect exercise for one's first "serious" program. All you need to finish such a program is basic program flow control and basic file I/O functions (the later can be looked up in the documentation of your given language quickly).

Don't say you couldn't do it until you tried, it's seriously easy.
Oh, sure. The only program language I'm even vaguely familiar with is Java. And that's something I just started taking a class in like, two weeks ago. Aside from that, I doubt I could code something like that in 65c816 (or if I could, that I have the skill needed to do so).

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on January 21, 2010, 02:25:47 pm
Grab a copy of Turbo Pascal and read through (most of) the help file - shouldn't take more than a couple of hours tops. Then look up the syntax for variable declarations, control structures and functions/procedures (which is really easy), and that's it. You're ready to start experimenting with files.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 21, 2010, 04:09:54 pm
Grab a copy of Turbo Pascal and read through (most of) the help file - shouldn't take more than a couple of hours tops. Then look up the syntax for variable declarations, control structures and functions/procedures (which is really easy), and that's it. You're ready to start experimenting with files.
That sounds fairly promising, even though I dunno what the hell that is (sounds kinda like a programming language, based on 'variable declaration'). I'll look into doing that this weekend.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: BRPXQZME on January 21, 2010, 05:14:53 pm
lol pascal
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on January 21, 2010, 05:28:29 pm
Oh Jesus, not gzip again! °0° At least I'm glad I've got it all implemented for FF7. This format is truly diabolic.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on January 21, 2010, 05:37:42 pm
@DarknessSavior:
I should add that when I said "read through the help file", I meant only the part that lists the constants, variables and functions of the already shipped units (SYSTEM, CRT, DOS, etc).


lol pascal
It's not standard pascal though.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: BRPXQZME on January 21, 2010, 05:43:22 pm
Yes, but all non-mainstream programming languages, API’s, and compilers—especially formerly-mainstream ones—are easy targets.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: KaioShin on January 22, 2010, 04:54:03 am
Is Turbo Pascal even object-oriented? I know Delphi is, but IIRC TP was before that. If he is learning Java anyway right now I wouldn't confuse him with another non-OO language. He can write that extraction tool just as easily in Java and since he has to learn it anyway (I assume it's for college or something) that'll also give him a huge boost in motivation to make progress with Java.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on January 22, 2010, 06:23:34 am
Is Turbo Pascal even object-oriented?
Yes, since version 5.5 (http://en.wikipedia.org/wiki/Turbo_Pascal).

Anyway, you don't need OOP for simple file manipulation (imo).
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on January 22, 2010, 09:44:11 am
As far as Java goes, the only stuff I know how to do is really simple stuff. Concatenation, simple arithmetic, and System.out.println stuff. We just started. Yes, it's for college, but I don't have to take it. I chose to take it, hoping that since it's "Intro to Programming" that it would help me out in the realm of ASM (that way I could at least start thinking like a programmer).

So I don't see myself writing some sort of extraction program in Java just yet. Or even by the end of the semester (I'm not 100% of the topics we're covering, but we're only doing the first six chapters out of this giant Java textbook, "Absolute Java").

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: BRPXQZME on January 22, 2010, 10:38:59 am
Gonna go out on a limb and recommend you just learn Python. I ain’t ever used it myself, but my experience from watching high-schoolers pick it up is that it makes stuff like this a cinch. Well-documented. zlib is part of the standard package, so you don’t even need to do anything funky to get it set up.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on January 22, 2010, 11:01:32 am
I wonder if java has a good native support for GZip. That would make the un/packer a lot faster to code, especially if it can work with buffers (:9).
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: BRPXQZME on January 22, 2010, 12:01:31 pm
Does it have native support? Yes. Is it good? ...depends on what you need. Some have felt the need to re-implement the functionality (http://www.jcraft.com/jzlib/), for example.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: golden on January 22, 2010, 12:24:22 pm
Gonna go out on a limb and recommend you just learn Python. I ain’t ever used it myself, but my experience from watching high-schoolers pick it up is that it makes stuff like this a cinch. Well-documented. zlib is part of the standard package, so you don’t even need to do anything funky to get it set up.
I recommend it as well, I'm using it for my projects' tools. If you want to, DarknessSavior, you can check out this post (http://www.romhacking.net/forum/index.php/topic,8509.msg142861.html#msg142861) which describes how something similar could be accomplished in Python (http://docs.python.org/3.1/library/).
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on January 22, 2010, 04:42:25 pm
Does it have native support? Yes.
I knew about this one. IIRC, they've used pure C code for that with DLLs or whatever other systems use.

Quote
Is it good? ...depends on what you need. Some have felt the need to re-implement the functionality (http://www.jcraft.com/jzlib/), for example.
:/ I'm not completely sure what's wrong with the default implementation, but as long as it decompresses from buffers it should be fine, especially for this kind of rearchivation programs.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Kitsune Sniper on January 22, 2010, 07:36:00 pm
:/ I'm not completely sure what's wrong with the default implementation, but as long as it decompresses from buffers it should be fine, especially for this kind of rearchivation programs.
It says so on the page.

"For example, we needed to full access to zlib to add the packet compression support to pure Java SSH system, but they are useless for our requirements." There's a bug in Java's zlib implementation that won't let them do this, so they made their own version. :p
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on January 23, 2010, 08:30:46 am
I read that part. Still, it doesn't explain if it's important or not for code using files rather than online streams.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on March 02, 2010, 07:58:09 pm
are you still working on it?
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Vegetaman on March 04, 2010, 05:02:36 pm
I'd suggest C#, as it makes this kind of manipulation ridiculously easy (at least, over C++). However, be very careful, as some commands (like filestream) can wind up being notoriously slow...
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 23, 2010, 12:44:49 am
DS, I'm beginning to think your hay stack might not be so large after all. If you go through the process of converting your PSP file into an .ISO file with the appropriate utility (maybe not for discussion here), you should find that the system of GZIP-compressed files inside gsd1.bin and gsd2.bin share a lot of names with the files in the PSX versions of these games. At least, this is what I've found with gsd2.bin and the PSX version of Suikoden II.

Unfortunately it's not guaranteed because there are more than a few files in the PSX version that are either missing or named differently in the PSP version, but it's worth a try perhaps. Can you recall the file name of any of the text files in the Japanese language version of Suikoden II-PSX? We can check real quick if fortune smiles upon your project here.

Disregard this if you've tried file name comparison already.  :D


EDIT: Snooped through a couple of these randomly; results mixed. I didn't find any similarity between VA01.bin (PSX) and VA01.bin (PSP), though I didn't check thoroughly. The files SHUGO.bin (PSX) and HP_SHUGO.bin (PSP) were much more interesting, sharing byte runs of significant length (greater than 0x30 bytes matched in one lucky data search). The PSP files tend to be much larger than their PSX equivalents based on what I've observed so far, so I'm wondering if I've been looking at graphics files that are uncompressed (after GZIP decompression) in the PSP files but still LZSS compressed in the corresponding PSX file. Are the .VA## files music related in the PSX version, by chance? If the soundtrack were remastered, that might explain why the files are radically different, and larger, in the PSP version. Complete speculation on my part though.

In any case, searching for the name of one of the text files you worked with in the Japanese version of Suikoden II in gsd2.bin might be worthwhile if you haven't done this already. Just using WinRAR to apply the GZIP decompression as Rolen47 suggested earlier works like a charm, I think. I remember seeing a recognizable ASCII string (I think it referred to another filename) in the uncompressed VA01.bin (PSP); it struck me as unlikely to appear if the decompression were inaccurate.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: PhOeNiX on April 23, 2010, 05:15:12 am
In Java should be relatively simple:
InputStream comprIS=new GZIPInputStream(new FileInputStream("your file name here"));

or if you want to work on buffers

InputStream comprIS=new GZIPInputStream(new ByteArrayInputStream(byte_buffer));

but i'd sudgest using BufferedStream, instead of loading the whole file in memory

InputStream comprIS=new GZIPInputStream(new BufferedInputStream(new FileInputStream("your file name here")));

//your code here

comprIS.close();
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 23, 2010, 02:11:34 pm
With regard to locating the game's text: judging from the English version of Suikoden II (PSX), VA01.bin contains the text for Riou's and Jowy's escape at the beginning of the game. Is that the case for the Japanese version of Suikoden II (PSX) as well? If so, I'd recommend GZIP-uncompressing the file tagged "VA01.bin" in the PSP version of Suikoden II (gsd2.bin) and seeing if your table works there. I didn't see anything immediately recognizable as one of the common Japanese text encoding schemes, but the English version uses a relative alphabet and I think you found that the Japanese PSX version of Suikoden II does as well; probably the case here. It'd be awfully cool if the exact same text encoding scheme were carried over from the PSX version to the PSP version.

I can't guarantee that the text is really in the VA## files, since I don't have access to a Japanese version of Suikoden II to text-compare with, nor a PSP for trial-and-error testing of the actual PSP files. Also, the VA## files in the PSP version and in the PSX version of Suikoden II seem very differently structured. There's many areas in the PSP version of the file that look like they could contain text, so I think the possibility is worth exploring, but even in best case scenario the project might involve more than simply swapping the English-language files from Suikoden II-PSX into Suikoden II-PSP. I suppose we could luck out and find that the PSP version files still have the same text storage structure as the PSX version files, and the differences come from any non-text info that might be stored in the PSP version files.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 23, 2010, 03:59:38 pm
I don't quite understand the whole decompression thing.

Also, I only figured out the table for Suikoden I. Table file could totally be different for II.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 23, 2010, 05:07:05 pm
Finally got my paws on a Japanese copy of Suikoden II, and there are some shared data in what I would guess are the right places. This is looking a bit more promising now! Take a look around address 0x84F8 of VA01.bin in your Japanese copy of Suikoden II-PSX -- does that appear to hold text according to the table file you have available? If you don't have everything handy at the moment, I'll see if I can figure out the Japanese text encoding scheme Suikoden II was using. If what I'm looking at right now is indeed dialogue, this project looks like it might be doable.


This is what worked for me as far as the PSP file decompression:

1. Convert your .CSO into a .ISO (with the appropriate utility, which isn't hosted here at RH.net to my knowledge, and probably for good reason). Running a search on "cso to iso" should net you what you need. Load your .ISO into a program like ISOBuster and you'll be able to extract individual files. The file I'm looking at is the meat of Suikoden II-PSP, which is "gsd2.bin."

2. gsd2.bin is an archive of GZIP-compressed files. Open it up in a hex editor and you'll find it starts with a table of some kind, probably a series of pointers to all the compressed files. Each file begins with the header Rolen47 mentioned earlier in this thread. You can extract the file via copy/paste in your hex editor, save it, and then decompress it in WinRAR. For example, there's a compressed file tagged VA01.bin that lies at address 0x14000 in gsd2.bin. Copy the data from 0x14000~0x6F000 and save it as its own file. Open up that saved file in WinRAR and extract it, and budda-bing, you might just be looking at the first portion of the game script!

I guess I'll see if I can figure out the text encoding scheme Suikoden II uses real quick. What program do you use for Japanese-language relative searches?

EDIT:
Anyone up for a quick Kanji-identifying exercise? I'm trying to develop a table for Suikoden II (PSX) - Japanese version, and it would help if I could identify the characters shown below in preparation for a relative search. Despite my best efforts I've been unable to hunt these down in the Shift_JIS table:
(http://img20.imageshack.us/img20/7295/textline.png)
Just having the first six Japanese characters in the white text display would be awesome. The very first bracket symbol should have a Shift_JIS value of 8175, so I'm looking for the six characters after that.

There's huge runs of bytes shared between the Japanese version of VA01 (PSX) and VA01 (PSP), so I'm growing more confident that the script's very easily accessible. Now I just need to prove it.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 23, 2010, 10:43:23 pm
For relative searches, I use monkey-moore. You have to input the Japanese character set by hand in the custom character thing, but it's the best relative searcher out there.

「気がはやいな、あいう。もう着がえたのかい?」

Not entirely sure if that second kanji is 着 or not. But it looks like it.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 24, 2010, 03:26:15 am
Thanks DS! Ugh, looks like this won't be easy. I'm fairly sure I've found the first line of text in the PSX version at address 0x6EA8 in VA01.bin (J-PSX). Two problems, unfortunately: this line of text isn't in the PSP version of the file ( :() and secondly the text encoding scheme looks pretty bizarre ( :().

Check out Jowy's first line here again, clearer pic this time:
(http://img143.imageshack.us/img143/4124/textlinejapanese.png)
In hexadecimal, this reads: DF 01 0C 73 5F 69 47 5A E4 05 00 E5 0E 68 48 02 26 73 49 55 5E 4B 47 E8 E0 00 00 00. The 05 00 should be an opcode that inserts the player's name (I chose "!!!" to keep the Japanese characters distinct from the player name here). DF and E0 should be opening and closing quote brackets respectively. But look at the meat of the dialogue...eugh. It doesn't seem to be encoded in any logical order, and there's just one byte per Japanese character instead of two; is this even possible? With one byte there's only enough room for Hiragana and Katakana, plus a select punctuation marks I think.

Well, actually, I take back the one-byte observation -- looking at this again, the character "い" repeated in Jowy's textbox only matches up if the first Kanji, "気," is interpreted as a two-byte value. Is it possible Hiragana and Katakana are one-byte values, with two-byte values reserved for Kanji and certain text display opcodes?


In any case, the line isn't in the corresponding PSP file, which is disappointing. Turns out the matching data I was looking at earlier in these files doesn't conform to the text encoding scheme either, so what I was examining was probably non-text data and a red herring. I'm not prepared to concede that the text isn't in these files in the PSP version yet, but it would appear the encoding schemes are probably different.               
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Tauwasser on April 24, 2010, 07:02:08 am
Well, actually, I take back the one-byte observation -- looking at this again, the character "い" repeated in Jowy's textbox only matches up if the first Kanji, "気," is interpreted as a two-byte value. Is it possible Hiragana and Katakana are one-byte values, with two-byte values reserved for Kanji and certain text display opcodes?

Very much so. However, I have also seen games on recent consoles (GBA) that will have a table of thirty or so kanji addressed in two-byte values and mapped to one-byte values for the text following it. Also, many games will build their table from first appearance of a character in dialog, so it doesn't need to be ordered, though it would be neater. So if you see inconsistencies in character addressing, you might as well check out if there is some loading mechanism that will swap characters. Also, the most-used kanji might have gotten one-byte values by default, even tho the rest got two-byte values.

cYa,

Tauwasser
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 24, 2010, 01:57:17 pm
*insert facepalm here* I should have tried this to start out with, and that would have saved a lot of time...

DS, your text is sitting right there in the PSP files in Unicode buddy! Little Endian style. :beer:  There's some opcodes assigned to unused values in the Unicode scheme I think, and these will need to be figured out of course. Check out some data pulled from near the very end of VA01.BIN (PSP):


(http://img143.imageshack.us/img143/4124/textlinejapanese.png)
 
0C 30 17 6C 4C 30 6F 30 84 30 44 30 6A 30 01 30 42 26 31 32 02 30 20 22
82 30 46 30 40 77 4C 30 48 30 5F 30 6E 30 4B 30 44 30 1F FF 0D 30


Someone might want to double-check me on this, but I think it's right (http://www.rikai.com/library/kanjitables/kanji_codes.unicode.shtml). The bytes "42 26 31 32" insert the player's name, and there might be a blank space opcode in there. The bytes "20 22" seem to be a carriage return command.


Unfortunately the file system for gsd2.bin doesn't seem to be readable by ISOBuster; I was hoping you'd be able to use a generic CD reader tool to extract the files, and rebuild gsd2.bin as if it were a generic CD. But since you're only dealing with 26 files, the more important concern will probably be re-inserting English text according to whatever framework and pointer tables the game engine might be using.

It seems there have been a lot of complaints about the English translation for Suikoden II especially? Does this mean you'd prefer to launch a re-translation project rather than use the English text in Suikoden II-PSX?
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Tauwasser on April 24, 2010, 02:02:53 pm
Someone might want to double-check me on this, but I think it's right (http://www.rikai.com/library/kanjitables/kanji_codes.unicode.shtml). The bytes "42 26 31 32" insert the player's name, and there might be a blank space opcode in there. The bytes "20 22" seem to be a carriage return command.

For a better table see the Unicode Code Charts (http://unicode.org/charts). Also, see the UniHan database (http://www.unicode.org/charts/unihan.html), where you will find most Han (CJKV) ideographs listed with presentational form pictures and all Unicode properties.

Also, your format is UTF-16 LE, sorry I didn't look at it earlier, but you posted a different string above.

cYa,

Tauwasser
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 24, 2010, 02:12:47 pm
Thanks Tauwasser, I hadn't seen the second one before. The use of the word "han" in that database has got my curiosity piqued. There seemed to be some sub-files tagged "han" along with files tagged "map" and "clut" in the PSP files. The "map" and "clut" files would seem to be graphics related at first glance, but I'm wondering what the tag "han" could mean. I always associated this word with Chinese and not Japanese, so I've been confused about it. In any case, the "han" files aren't anywhere near the Unicode text; they're in the same area as the "map" and "clut" files.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Tauwasser on April 24, 2010, 02:23:46 pm
Han is just used for Hanzi, the Chinese 漢字, because many other actual Chinese characters and derivatives as well as new creations are used in CJKV languages. However, your "han" files might simply be an abbreviation for some Japanese word that starts with はん and is relevant to maps, events etc.

cYa,

Tauwasser
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 24, 2010, 02:25:29 pm
It seems there have been a lot of complaints about the English translation for Suikoden II especially? Does this mean you'd prefer to launch a re-translation project rather than use the English text in Suikoden II-PSX?
I was hoping to do a re-translation of Suikoden I, actually. And if that worked out well enough, I would do the second one.

And by the text being "right in the PSP files", do you mean in the gsd.bin files, or the files within there?

I'd need to do alot of things to make this work out right. Gotta get a lower-case font in there (by default, Suikoden I has an upper-case English font, but no lower case one) for both normal text and battles. And possibly look into making the font dimensions smaller (if memory serves me, the English font in Suikoden I PSX was 16x16).

But thanks for looking into all of this.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on April 24, 2010, 04:07:25 pm
The file structure you'd have to go through to find the text is like this (still using Suikoden II as an example):

ISO file (converted from CSO file)
-> gsd2.bin
--> GZIP compressed file tagged "va01.bin" (carve this out of gsd2.bin either by hand or with an extraction/injection tool down the road)
---> va01.bin (the true file after WinRAR decompression)

The text seems to be toward the very end of each decompressed va## file and stored in straight Unicode with the exception of special opcodes for various functions. I'm not sure how sensitive the game engine will be to tampering at each level of the file structure. Rebuilding the ISO file and converting it back into a CSO should work theoretically, but custom tools might be needed for rebuilding gsd2.bin from the translated and recompressed files if the game engine wants to be picky.

I'm not in a position to take on more projects, but I'd love to see the PSP release playable in English someday. I saw some video of Suikoden I&II the other day and I was impressed that they apparently made new sprites for diagonal direction walking on the field maps.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on April 24, 2010, 08:07:16 pm
converting it back into a CSO
Unnecessary step.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Guadozoku on April 24, 2010, 09:22:15 pm
KaioShin said the right thing. :) Format of this archive is plain and easy.
It's just an archive with many gzipped files in there.
A ton of games seem to do this, someone should write a program that extracts all the gzip files for use in winrar.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on April 25, 2010, 04:43:11 am
Or you can use the command-line program gzip (http://www.gnu.org/software/gzip/) instead of WinRAR. ;)
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 25, 2010, 10:59:17 am
Or you can use the command-line program gzip (http://www.gnu.org/software/gzip/) instead of WinRAR. ;)
Would that actually work properly? If I were to throw one of the giant files into gzip, would I be able to get all of the smaller files out of it easily?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on April 25, 2010, 11:52:10 am
Or you can use the command-line program gzip (http://www.gnu.org/software/gzip/) instead of WinRAR. ;)

Would that actually work properly? If I were to throw one of the giant files into gzip, would I be able to get all of the smaller files out of it easily?

gzip works on streams, which may be read from a file or from standard input (http://en.wikipedia.org/wiki/Standard_streams) (e.g. "<file1.dat  gunzip -c  >file2.dat"). So if you give it one file or stream you'll get one file or stream in return.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 25, 2010, 12:04:50 pm
Or you can use the command-line program gzip (http://www.gnu.org/software/gzip/) instead of WinRAR. ;)

Would that actually work properly? If I were to throw one of the giant files into gzip, would I be able to get all of the smaller files out of it easily?

gzip works on streams, which may be read from a file or from standard input (http://en.wikipedia.org/wiki/Standard_streams) (e.g. "<file1.dat  gunzip -c  >file2.dat"). So if you give it one file or stream you'll get one file or stream in return.
I don't really get the streams thing. But if I'm just giving it one file (that contains multiple files), I'll only get one file back?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on April 25, 2010, 12:26:58 pm
But if I'm just giving it one file (that contains multiple files), I'll only get one file back?

That's what I said.


I don't really get the streams thing

Stream = channel for program input or output.

When you open cmd.exe and use the "dir" command, it lists the content of the current directory by writing text characters to StdOut. By default that's the console window. When you use "dir >tmp.txt", the output is written to the file "tmp.txt". (">>" appends to the file instead of overwriting it.) When you use "dir |more", the text is redirected to the program "more", which reads text from its StdIn, writes it to its StdOut and pauses after each page. So you can build entire pipelines (http://en.wikipedia.org/wiki/Pipeline_%28Unix%29).
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 25, 2010, 01:35:22 pm
But if I'm just giving it one file (that contains multiple files), I'll only get one file back?

That's what I said.

Would this one file be something like a normal archive that I can open and get all of the files from, or what?

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: creaothceann on April 25, 2010, 01:55:28 pm
No. gzip is not zip!

Think of gzip as a compression scheme. It really handles only one big block of data.


KaioShin said the right thing. :) Format of this archive is plain and easy.
It's just an archive with many gzipped files in there.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on June 22, 2011, 05:41:46 pm
Any more progress on decompressing the files?
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Sephiroth 1311 on June 30, 2011, 05:31:36 am
Hi there, today I stumbled upon a French translation project (http://www.romhack.org/viewtopic.php?f=23&t=3585) for Suiko I&II PSP, so I thought you might be interested in asking them for help. :)

Would be nice to have this game in English too.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on June 30, 2011, 08:39:43 pm
Ooh, why do I have the feeling an Italian translation might be in the works soon?  ;)

As I recall, we have a handle on how to g-zip decompress and probably recompress the necessary files, and dialogue text has been identified (not sure how that reflects on getting at miscellaneous and image-based text though). What's missing is a nice front-end utility for taking care of repointering text strings, though that could be worked on by hand in a pinch. The big challenge IMO is finding a PSP emulator with full debugging and tracing capabilities, not to mention waiting until the coast looks clear for PSP translation projects, with respect to US modders. Anyone interested in a translation project could make a big difference by investigating what tools the French project is currently working with.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Sephiroth 1311 on July 01, 2011, 06:49:54 am
Ooh, why do I have the feeling an Italian translation might be in the works soon?  ;)

As I recall, we have a handle on how to g-zip decompress and probably recompress the necessary files, and dialogue text has been identified (not sure how that reflects on getting at miscellaneous and image-based text though). What's missing is a nice front-end utility for taking care of repointering text strings, though that could be worked on by hand in a pinch. The big challenge IMO is finding a PSP emulator with full debugging and tracing capabilities, not to mention waiting until the coast looks clear for PSP translation projects, with respect to US modders. Anyone interested in a translation project could make a big difference by investigating what tools the French project is currently working with.
Actually, my group and I are working on the PSX version of the first Suikoden. We currently don't have plans for the PSP version, as long as an unofficial translation into a European language comes out. :D

Anyway, you're absolutely right when you speak about the lack of a decent PSP debugger. I was really startled when I saw those images from the French forum (I occasionally check it, since it's the biggest european forum about romhacking and translations), and I immediately thought of this topic.

I really hope they'll be willing to share some information with this community, an English translation would be a great present for the international Suikoden fanbase.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on July 01, 2011, 01:22:57 pm
Is there anyone here who has enough French experience to ask them for help as far as getting to the text and inserting that font and whatnot? Or should I risk looking like a fool and write up a large post and run it through Google Translate?

This is literally my dream translation. FFIV is my first RPG, so it'll always have a special place in my heart. But Suikoden I is my absolute favorite RPG. And I would give anything to translate these two games and bring them to the Suikoden-loving public.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on July 01, 2011, 02:22:43 pm
Erm, they do understand English.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on July 01, 2011, 02:31:55 pm
Erm, they do understand English.
I didn't want to make the stupid assumption that most Americans do: that everyone around them speaks English.

Damned if I do, damned if I don't. :P

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Gemini on July 01, 2011, 05:20:12 pm
It's a board where they translate games from English, so it's pretty obvious some if not most of them understand said language.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on July 02, 2011, 12:03:36 am
If you really want I can try to translate something for you.... But as Gemini said they most likely understand english
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on July 02, 2011, 11:36:12 am
Well, I've posted on their forums, and the guy doing the work on it has offered to let me contribute. I'm not sure of the details so far, but it looks like I'm going to wind up working with them and translating the games!

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on July 02, 2011, 10:43:24 pm
Glad to see they'll let you work with them!

It's a shame they won't just release a version with the original US script though.... but it will be nice to see what you  come up with!
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: FaustWolf on July 03, 2011, 02:07:24 am
Wow, way to go DS! Can't wait to try out the translation one day.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Sephiroth 1311 on July 03, 2011, 03:47:38 am
Good luck with your project! It's always nice when people share the information they got and help the whole RH community. If you succeed in completing your project, it'll be a great joy for the whole Suikoden community around the world (assuming that most of them is at least able to understand English).

I was also happy to read that their tool is able to export the PSX script and import it into the PSP version. This way, if my team does translate the PSX version into Italian, we will hopefully have an easy time converting our translation to the PSP version and, at that point, translate Suikoden II too. Which, actually, was officially translated, but it's one of the worst Italian translation I ever saw when dealing with videogames, so it does need to be remade from scratch.

And, of course, merci beaucoup to the French team!
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Azkadellia on July 03, 2011, 04:48:59 am
The US version of Suikoden 2 was badly translated also. Same goes for Suikoden 1. Misspellings, grammar errors, among other things. They couldn't even get the character's names straight.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on July 03, 2011, 03:29:42 pm
Well, I figured I would let you guys know that the guys behind the French translation have given me the tools I need to translate the scripts. They're stored on their servers, and once I'm done translating them, they can insert them into the game.

So far, I've got what seemed like all of the shop translations done. And I just started on some of the random NPC text in Gregminster.

The program allows me to see a progress report on how much I've done. So far? 0.33%. But I've just begun!

Also, I don't plan on altering the English script if it is unnecessary. If the Japanese matches up with the English, I'll just clean it up as much as possible and make it flow a little better. But if it's off somehow, or I think it needs some rewording, I'll do that too.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on July 04, 2011, 05:14:59 pm
I'm glad to see you're not planning to stray too much from the original script. I hate literal translations
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on July 04, 2011, 05:25:38 pm
I'm glad to see you're not planning to stray too much from the original script. I hate literal translations
The original PSX translation -is- pretty literal. Those're the parts I plan on playing with a bit. :P

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: StorMyu on July 05, 2011, 02:24:57 pm
It's a shame they won't just release a version with the original US script though....
As said here by Kipy
http://romhack.org/viewtopic.php?f=23&t=3585#p95093 (http://romhack.org/viewtopic.php?f=23&t=3585#p95093)
(Don't worry he wrote it in english)
Quote from: Kipy
We actually made tools that are able to associate the PSP's japanese texts to the english ones (from the PSX), even if they're not coded and ordered the same way. :)

that therefore means we can put the US texts easily in this version.
But for legally reasons, we're not allowed to do it. (official us texts are copyrighted materials).

+ if you want to see some screenshot.
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Rudy on July 05, 2011, 04:53:41 pm
As said here by Kipy
http://romhack.org/viewtopic.php?f=23&t=3585#p95093 (http://romhack.org/viewtopic.php?f=23&t=3585#p95093)
(Don't worry he wrote it in english)
+ if you want to see some screenshot.
Thanks, I was already keeping track of the thread (plus I also speak french so I can somewhat follow it lol)
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: bazinga59 on April 02, 2012, 05:58:11 pm
is this project dead?
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on April 10, 2012, 08:34:12 pm
Nope. Project's still up and running, I just checked with the project head myself.

I personally haven't done much work on the English script (it's sitting at about 2% of a whopping 1.2 MB script -- most of the system messages and Gregminster text is done). But kipy says the French script is almost finished, and then he's going to dump the side-text (items, spells, equipment, etc).

I'm gonna start hitting this pretty hard once my semester is over, which is only about two weeks away. Also, Niahak has joined the translation team for the English version. :D

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: bazinga59 on April 10, 2012, 10:47:58 pm
Nope. Project's still up and running, I just checked with the project head myself.

I personally haven't done much work on the English script (it's sitting at about 2% of a whopping 1.2 MB script -- most of the system messages and Gregminster text is done). But kipy says the French script is almost finished, and then he's going to dump the side-text (items, spells, equipment, etc).

I'm gonna start hitting this pretty hard once my semester is over, which is only about two weeks away. Also, Niahak has joined the translation team for the English version. :D

~DS

awesome thanks for the update!  ;D
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: Shin` on September 16, 2012, 08:45:56 pm
Sweet jesus, how did I miss this?

Is the project still active DS?

If there is anything I can do to contribute let me know. :)
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: DarknessSavior on September 16, 2012, 09:20:50 pm
Sweet jesus, how did I miss this?

Is the project still active DS?

If there is anything I can do to contribute let me know. :)
It's still active, as far as I know. I haven't done any work on it personally in the last few months or so, though. I've been busy trying to finish up other projects and doing life-y things.

~DS
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: jmjohnson85 on January 04, 2014, 10:20:35 am
Was just searching to see if anyone was working on this game.  Very happy to see DarknessSavior tackling it - you have my support!
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: c-c-c on September 30, 2016, 11:36:02 am
It's still active, as far as I know. I haven't done any work on it personally in the last few months or so, though. I've been busy trying to finish up other projects and doing life-y things.

~DS

soo... what ever happened to this project?   :-\
Title: Re: Trying to make a table for Suikoden I and II (PSP)
Post by: KingMike on October 02, 2016, 11:02:46 am
If anything happened, Darkness would've updated in the last four years. :P