News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: [PSX] Brave Fencer Musashi - Messy texts  (Read 16557 times)

Dark Fulgore

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
    • Tool-assisted Flawless Playthroughs
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #20 on: January 24, 2013, 11:08:33 pm »
Well it seems you are still 1 step ahead of me.
I got the old version (and a newer) to start on win xp and win7 x64 but when i try to expand cd, it crashes.
Did you add some dll to the folder of the exe?? Can you please zip them and send them to me??

Check my previous post for the upload link. I uploaded myself, just extracted and when it crashed while using the Expand CD option, I used XP Compatibility Mode and done, worked.

By the way the newer versions seem to have lost the PSone games and concentrate on RE.
A little thing I noticed is that you are still working with the whole image, you usually don't do that.
Get isobuster or cdmage to extract the single files.

I already did that and found this SC0177.BIN is the one with the Musashi dialogue I posted.

The list.cd you can let it alone for the moment, it seems a collection of 8 pointer tables but i not sure yet for what.
Main.cd contains all assimilation texts, save screens, game over, reload etc... texts and mostly already decompressed.
If you search the whole image for Musa, you will find a mix of decompressed and compressed text but if you search for <Musashi>, you will find mostly uncompressed text (yes, there are parts that are decompressed already so you could study the format there but you have to play the game till there ;p)
If you searched the image for a text and you want to know in what file it is, take the offset in the image, convert it to decimal, divide it by 2352 and check (with cdmage or any program that show you the LBA of the files) where it is.
Let's make the example of your text above, the offset in the image is x1EE1B10 = 32381712 > divide by 2352 = 13767.73....
If we check the table CDmage give us for the LBA, we have SC01.CD starting at 6161 and ST01.STR starting at 15965 so the text you posted above is inside the file SC01.CD (xEDB6A7 to be exact).

Cool math!
Tried by myself and found that the first dialogue at the shops are in SC03.CD and they really are! What the hell!?
As you said, they aren't in any logical order or probably those "fixed texts" are all in SC03.CD. I will check this out.

The very first sentence on the game (<Steward Ribson> There both be no effect!!) is in the same archive (SC01.CD) at xF0909E meaning that the text inside the files is not in the right order just to make things more fun ;)
I can not really help you with the LZSS program because i don't know wich one you are using.
What i told you before about compression is correct but there is more and i belive this kind of compression is called huffman.
Not yet tested but take a look at your hex screen, just before your maked part (4 bytes before) there is an FF, let's start from there.
FF = the next 8 bytes are uncompressed so 01 18 <Musas (write like this because more easy to follow)
the next byte is 7F, (not confirmed) it's looking like it's in yF format meaning that the next y bytes are uncompressed so hi> 0A 19 14 17
F4 00 it's probably one of those *go back and copy* but i didn't figured out the format yet. Because the next is a FF, i believe this will copy only 1 byte.
here the next FF so 25 01 Shoot 04
here there is another of those go back and copy and i believe that the last byte it write here will be a FF so i believe that could be it copy and write the above 0A 19 14 17 F4 00 FF
because the last byte writte in a FF, the next will be uncompressed
and so on.

I can only depend on you with this compression problem.  :-[
But I will look at those non-compressed, those simple dialogues without voice and animation from Musashi.

If you can send me some of those splitted files you got from that program (especially those first texts in the game) i can check out the decompression tools).
Did you check that they are not already decompressed??

Here's the exact file where the Robinson text appears.
They still compressed and there's a lot of files. SC01 has 86 files and SC02 has 45 files, all compressed. I mean, the compressed texts you know still compressed.
Thanks again, Auryn! This compression is the major problem of this, right? Because I want to edit the main screen image and put your name there, man. Really, I have no knowledge at this, it's my first time trying this with a PSX game and you are helping me a lot, explained everything with every detail. I'm really thankful for your time and effort, brother.
Thanks everyone using their time to tech me those things, giving me advices and stuff.
Thanks so much, guys!

BioFAT should be more stable with the next few revisions. MarkGrass, the author, is currently using bits of my code to replace the SDL libraries; this should hopefully fix all the dependency issues and most random crashes. As for the compression, I'm afraid you'll have to use a specific compressor made for this game. It could be using something already implemented in other games, like Okamura's LZSS from FFVII, but it could also be something totally different like in Xenogears or Chrono Cross. SquareSoft was the master of custom compressions back in the PSX days.

Those compressions is what kills most of the projects people starts? Or there's something more complicated than that?

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #21 on: January 24, 2013, 11:11:50 pm »
The compression look damm familiar but I didn't pingpointed yet wich one it was :p
Anyway, 2 little things:
first what we told you before about the "control codes", even if you worked on the compressed file, when you found the control codes to change the text color, they was correct because those bytes was uncompressed.
So what it means?? You can place a  0102 in front of a word to make it red and put an 0101 after it to change the color back to black.
If you can reconize more control codes, you can change the game even further.
Just a proof of this 2 things, look at this:
http://youtu.be/uWtwvboH0Dk
See the first speaker name changed and made red and listed as the voice acting for the first 2 sentences are wrong and repeated 2 times :)
it's looking like 88XX is voice acting (usually before the speaker name)
new line is 0A19 usually followed by 14 xx (maybe a little pause)
there is another byte before the speaker that i was thinking is maybe the pic on the speach bobble but the game crashes on epsxe if changed: anyway they are 0218 for Steward, 0418 for the butler, 0318 for the princess, 0118 for Musashi and 0018 for All together.

Here a little present, the first part of the dialogue uncompressed (no i didn't crack that yet, I got it from a savestate ;) ).


With this you can start study those control codes:)

Dark Fulgore

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
    • Tool-assisted Flawless Playthroughs
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #22 on: January 25, 2013, 12:32:38 pm »
The compression look damm familiar but I didn't pingpointed yet wich one it was :p
Anyway, 2 little things:
first what we told you before about the "control codes", even if you worked on the compressed file, when you found the control codes to change the text color, they was correct because those bytes was uncompressed.
So what it means?? You can place a  0102 in front of a word to make it red and put an 0101 after it to change the color back to black.
If you can reconize more control codes, you can change the game even further.
Just a proof of this 2 things, look at this:
http://youtu.be/uWtwvboH0Dk
See the first speaker name changed and made red and listed as the voice acting for the first 2 sentences are wrong and repeated 2 times :)
it's looking like 88XX is voice acting (usually before the speaker name)
new line is 0A19 usually followed by 14 xx (maybe a little pause)
there is another byte before the speaker that i was thinking is maybe the pic on the speach bobble but the game crashes on epsxe if changed: anyway they are 0218 for Steward, 0418 for the butler, 0318 for the princess, 0118 for Musashi and 0018 for All together.

Here a little present, the first part of the dialogue uncompressed (no i didn't crack that yet, I got it from a savestate ;) ).


With this you can start study those control codes:)

Hahaha! I tried to change some voices, made Musashi say "Give it a stomp!" at the first dialogue. But I changed it directly into the ISO file. Did you changed the RED word in the SC0101.BIN file? There's a way to change in the emulator debugger and affects the game? Because watching the memory everything is easier to spot, but when looking at the file, they are compressed.

Found the value that changes the texts box. There's a value of 02 and 06. 02 is the normal box and the 06 is the yelling box.



There's a 09 before both examples, but there's another dialogue that you can change the box but there's no 09 before.

Here's the change:


But I can't find the value of the initial box, tought. I'll take a look at the dialogue pic and take a better look at the decompressed image you sent me.
Thanks, brother! See ya!

EDIT:

Just made a quick translation on a uncompressed text, didn't exceed the original size and changed the red words to the ones I needed.



Hahaha, looks so cool see this in my language. :D
I hope I can keep the jokes!
« Last Edit: January 25, 2013, 12:50:49 pm by Dark Fulgore »

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #23 on: January 25, 2013, 02:37:57 pm »
Quote
Check my previous post for the upload link. I uploaded myself, just extracted and when it crashed while using the Expand CD option, I used XP Compatibility Mode and done, worked.
I was working on Win XP, tested on win7 with compatibility mode and it worked.

Quote
I already did that and found this SC0177.BIN is the one with the Musashi dialogue I posted.
Well, later about this later.


Quote
Cool math!
Tried by myself and found that the first dialogue at the shops are in SC03.CD and they really are! What the hell!?
As you said, they aren't in any logical order or probably those "fixed texts" are all in SC03.CD. I will check this out.
A psx cd is composed of 2352 bytes for each sector (not the movies) where 2048 are real data and the rest is for the CD itself (see EDC/ECC in google).
The image of the cd is build reading all those bytes and sectors one after the other and an LBA is the number of the sector, so not really a big trick there :p


Quote
I can only depend on you with this compression problem.  :-[
I can only figure out how it works, then you will need somebody that with make you a tool to decompress if there is none that is working online.


Quote
They still compressed and there's a lot of files. SC01 has 86 files and SC02 has 45 files, all compressed. I mean, the compressed texts you know still compressed.
Thanks again, Auryn! This compression is the major problem of this, right?
If you asked me this yesterday but i am not sure anymore. It was looking too easy to be a Squaresoft game and taking a look at those splitted files, I see a little problem but i have to check more about it.
Just the short version, the first byte in SC01.CD is x56 = 86 and then followed by 4/4 combination that look like offset/size pairs and the number of bytes of this pointer table look everything ok.
At x800 should be the start of the first file and it starts with "PAC" so if you search SC01.CD for "PAC", you should find 86 instances but you will find more than 200. There are sure some PAC that are not the magic word but there are still more than 150 PACs in that file.
Meaning that you maybe have to split those splitted files again and in fact if you search for PAC in any of those splitted files, you will find some instances.


Quote
Because I want to edit the main screen image and put your name there, man. Really, I have no knowledge at this, it's my first time trying this with a PSX game and you are helping me a lot, explained everything with every detail. I'm really thankful for your time and effort, brother.
Thanks :)


Quote
Those compressions is what kills most of the projects people starts? Or there's something more complicated than that?
Yes and no. Many people think that translating a game is the same as translating a txt file and mostly don't have patience or the intelligence to figure out those schemes (archive and formats).
Is it the most diffucult?? Well I would not say that in this case. they are maybe customs archives or compression but very near to "standart" formats. If you encounter games with custom tables, custom format, custom compression and maybe copy protections or even custom file format (see Chrono Cross, Vagrant story where you see only 2-3 files in the cd but the cd is full of files adressed by LBA and not by "name" as usual). Yes, Gemini, I know my last sentence here it's actually wrong but let it be like this for this new guy :)

Quote
Did you changed the RED word in the SC0101.BIN file?
No I changed it in the SC01.CD file and stay just "ahead of the compressed part" :p


Quote
There's a way to change in the emulator debugger and affects the game? Because watching the memory everything is easier to spot, but when looking at the file, they are compressed.
You mean to change the ram in real time in the game.
There was ways to do it with cheat modules and pc connected to real hardware but sadly mine all broke down :(
From what i know the only way would be to edit the save state but you need to recompress it every time so almost as long as editing the files.



Quote
Just made a quick translation on a uncompressed text, didn't exceed the original size and changed the red words to the ones I needed.
You can probably use the 00 between the sentences if you need just look out to let at least a 00 00 between one sentence and the next one, look out for text overflow on screen and don't change the starting point of those sentences in case there are some pointers around you didn't see yet.

Nice pic, it's looking like you are getting the job done there.
Changing the bubble or changing the sounds is not really necessary for the translation but if you know what they do, you can exclude them and find out what others do.


LostTemplar

  • Hero Member
  • *****
  • Posts: 906
    • View Profile
    • au-ro-ra.net
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #24 on: January 27, 2013, 09:58:54 am »
I took a quick look at it and it looks like a relatively standard LZSS compression.

First, there's a flag byte whose bits (starting from the LSB) indicate if the next token is a reference or a literal (1 = literal, 0 = reference; e.g. 0xff -> the following 8 bytes are all to be copied verbatim to the destination buffer). This is pretty straight-forward, however, I couldn't fully figure out the reference part.

A reference seems to be composed of two bytes; an offset and a length. However, there seem to be multiple interpretations. Sometimes, it's simply "copy len bytes from buffer_start + offset + 3", but then there are offsets like 0xf8 right at the beginning that point nowhere, whether it's interpreted as signed, relative, or absolute. Furthermore, some lengths don't make sense, for example there's a reference that says length = 0x0a although only 4 bytes are copied.

It would be pretty easy if one were to look at the assembly code, I guess, though.

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #25 on: January 27, 2013, 11:42:45 am »
I didn't have time to do something this weekend and i will not have time the whole week.
Anyway thanks LostTemplar.
Those special cases you talking about (like 0xf8 right at the beginning that point nowhere / reference that says length = 0x0a although only 4 bytes are copied.) sound absolutly like the FF7 compression.
If I remember well the first one: xf8 pointing nowhere (I believe it points before something is written, right??) should just write 00 for the reference length.
For the second case, are you sure it's only 4 bytes?? is it possible they are repeated??
From what i remember in FF7, there was cases where the length was longer than the window meaning simply that the window would be repeated fully or partially again and again.
What i mean that if we are in the same offset in the compression and length would be 4, output would be:
ABCD
but with length x0a it would become:
ABCDABCDAB

Sadly qhimm wiki is still down because there would be everything about it in there :(

Maybe Gemini can confirm that this decompressor work for FF7 ( i got it from a old post of him):
http://pastebin.com/cEXZUYRj
« Last Edit: January 27, 2013, 12:08:15 pm by Auryn »

LostTemplar

  • Hero Member
  • *****
  • Posts: 906
    • View Profile
    • au-ro-ra.net
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #26 on: January 27, 2013, 12:17:55 pm »
Code: [Select]
77 B6 1C 0A 61 62 50 02 74 73 w¶  ab  ts (should be "whereabouts" after decompression)

77=w, B6 is the next flag byte. So B6 = 10110110, first bit is 0, it's a reference: 1C 0A. At offset 1C there's "here doth [...]"; because the result should be "whereabouts" and the "ab...ts" comes directly after, it only copies the "here", which is 4 bytes. However, other times (e.g. directly after, the "ou" of "abouts") the length is just as it is stored. Pretty weird everything.

I'm pretty sure it's not the format that's decompressed in that FF7 code... (which would be just 12-bit offset and 4-bit length; here it seems like 8-bit offset and 8-bit length... though I'm not overly sure).

RetroHelix

  • Full Member
  • ***
  • Posts: 148
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #27 on: January 27, 2013, 05:39:48 pm »
Don't know if http://web.archive.org/web/20111108072536/http://wiki.qhimm.com/FF7 has the information to verify your thoughts. Just wanted to let you know about it.

Dark Fulgore

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
    • Tool-assisted Flawless Playthroughs
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #28 on: January 27, 2013, 06:03:45 pm »
Wow, that's a lot of LZSS information. Are guys close to an answer? Anything I can do to help? I wish I could know something more deeper about this issue.  :-[

I was translating some of the texts when Musashi assimilates a new ability and noticied a new color, instead the light red used in most of the dialogues, I found a real red used when the tutorial says to press the O button. And then made some tests and found those colors:

01 00 appears to be white or nothing.


01 01 is the standard black text.


01 02 is the light red used in most of important words.


01 03 is pink.


01 04 is green, used to represent the BP in some texts.


01 05 is red.


01 06 is blue.


01 07 is brown or golden.


01 08 is gray.


01 09 is a more intense red.


01 0A is lime green.


01 0B is yellow.


01 0C is a more intense blue.


01 0D is a more intense pink.


01 0E is light blue.


I can't even remember where those colors are used, if they do.
I liked a lot those colors, need to see where they are used or maybe I can insert some of them by myself.  ;D

Vehek

  • Full Member
  • ***
  • Posts: 206
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #29 on: January 27, 2013, 07:51:08 pm »
Here's what I have so far on its LZSS implementation, based off looking at it in a debugger and examining some of its compressed data. I don't know where the file size (for either the compressed form or decompressed form) is yet.

Ring buffer length is 1024 bytes.
Threshold is 1. So at minimum, a reference can copy 2 bytes (breaking even with the size of the reference). 10-bit offset, 6-bit length.

The starting position in the buffer appears to be 1.

Auryn

  • Hero Member
  • *****
  • Posts: 650
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #30 on: January 28, 2013, 02:36:09 pm »
Thanks RetroHelix, that is the site i had in mind and more exactly the LZS page.

LostTemplar, please take a look at it, there are some formula in that site, maybe something similar apply to BFM.


Dark Fulgore

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
    • Tool-assisted Flawless Playthroughs
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #31 on: February 02, 2013, 02:55:19 pm »
I'm translating some of the assimilation ability texts and I'm adapting the text as I can.

The Gunshot ability:





(Already fixed the red dot at the end.)

The Perfume ability:



The Stun ability:





I'm translating without special characters, there's a "simple" way to add more characters or they normally are more complicated?
« Last Edit: February 02, 2013, 03:00:20 pm by Dark Fulgore »

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #32 on: February 02, 2013, 05:57:30 pm »
You just need to ensure that the font has the needed glyphs and that the character set includes it.

Josekenshin

  • Jr. Member
  • **
  • Posts: 3
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #33 on: March 28, 2013, 03:22:06 pm »
Hi

Also I want to translate the game into Spanish.

CUE has a decompression tool. The files are PAC and SQV.

http://www.romhacking.net/utilities/939/

The tool decompresses files. CD and also decompress this files in a folder called .dir





I see the text, but I need to recompress this files.  :-\

Josekenshin

  • Jr. Member
  • **
  • Posts: 3
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #34 on: February 26, 2014, 09:58:06 am »
Sorry for asking again. I would like to translate the game in to Spanish and  I haven´t  idea how to compress the files, extracted by the tool of CUE.

Thanks.

Dark Fulgore

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
    • Tool-assisted Flawless Playthroughs
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #35 on: February 26, 2014, 09:17:29 pm »
I would love to translate Musashi to PT-BR, but I don't have the necessary programming skills to bypass that compression thing. :(
If some good soul could make an extractor for us, that would be our salvation. :)

Vehek

  • Full Member
  • ***
  • Posts: 206
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #36 on: March 01, 2014, 06:16:55 pm »
As mentioned above, CUE made an extractor.
What are needed are:
a) A compressor.
b) A tool to put the files back together, saving the time of manually editing the .CD files in a hex editor.

Since I don't know much about writing compression routines, I tried modifying Okumura's algorithims last year. The results are compatible with CUE's extractor but not with the actual game. Something goes horribly wrong when the game tries to read my recompressed file. It ends the decompression early and gets stuck in a loop of decompressing the file.
I tried looking into how it uses the file length from the PAC header, only to find that it had no effect on the decompressed size. Sigh...even more mysteries that need to be resolved before the game's archive files can be reconstructed.

It looks like it'll be a pain to adjust string lengths in this game. Text pointers can be rather scattered. Some dialogues don't even have full pointers. Their offsets are hardcoded into the location's coding. Not event code, but assembly!
For example, when Musashi notices the soldier with the Gunshot ability:
Code: [Select]
0017e4dc: 3c04801b lui r4,0x801b
0017e4e0: 248417f0 addiu r4,r4,0x17f0

Those uncompressed assimilation/death strings? Their pointers aren't in main.cd, but rather in the compressed location files. Not just one set of pointers, but one in every location. (I don't know if it's actually "every" or not, but probably a lot.) Looks like you'll have to stick to the way you've apparently been doing it, fitting the translated text within the original string length.

Vehek

  • Full Member
  • ***
  • Posts: 206
    • View Profile
Re: [PSX] Brave Fencer Musashi - Messy texts
« Reply #37 on: August 05, 2014, 11:44:51 pm »
If anyone wants to know, I've since done a trace on part of the decompression routines, and I found what actually signals the end of the compressed data. The game stops decompressing when the start-offset of a reference is 0. That's why my prior recompression attempts didn't work; they were filled with references that had offsets of 0.
I also found the font a few months ago. View the "001" file in FILE_010.dir of MAIN.CD when dumped by CUE's extractor. In that file, I think the font graphics start at 0x4CBC. It's 1BPP format.