News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - theflyingzamboni

Pages: [1]
SOLVED: Apparently xdelta could do that the whole time. D'oh!

I've been working on various mods for The Legend of Dragoon and created my own sort of Frankensteined patcher for the game. Right now it uses xdelta3 to create and apply diff patches to small component files, which has worked fine so far. But I've reached a point where I have two mods that need to patch the same file, and there's no way to do that with xdelta without going down the road of multiple patch versions. And that way lies madness and terrible user experience. Also, one of these mods expands the sizes of some of these files, so simple offset replacement won't work.

So is there a patch format and command line patching utility that allows for:
  • insertions/deletions and changing file size like xdelta
  • patch stacking

As an added bonus, if possible, I would like to be able test the patches before applying them to make sure they aren't going to modify the same bytes. So if someone knows of a patch format that fits these requirements, I'd also be interested in the specifications and how they could be compared.

Programming / Recompressing a file to original size
« on: December 05, 2018, 01:56:49 pm »
In The Legend of Dragoon, some of the files are compressed via an implementation of byte-pair encoding applied to blocks of <=2048 bytes in size. For each block, the byte-pair dictionary assigns value pairs to keys until there are no pairs occurring at least 5 times, or until all valid keys are full. The only ordering that is important is that the pair that occurs the most frequently is the next to be encoded.

My issue is that in some cases, the size of the compressed code has to remain the same, to avoid upsetting absolute pointers in the file the compressed code is embedded in. Currently, I get around this by randomizing the dictionary sort order on a block-by-block basis. When two byte pairs in a block occur with the same frequency, I use different sub-sorts. For example, I sort them in ascending order by the first byte in the pair, and further sub-sort that by the second byte in descending order. I have a total of 5 different sort types, and for a given block, each one may give a different compressed size. This way, when I'm compressing multiple blocks, it can repeatedly attempt to compress the file until it hits on a combination of sorts that results in the original compressed file size.

There are two questions I have:
1. Is there a way to do this that is more efficient than pure randomization? If I'm compressing a file 100 blocks long, I could attempt compression 100+ times and still not hit on a winning combo (and running through every permutation is out of the question). So like, if the file is consistently compressing to too small a size, then weight a sort order that produces a larger compressed block on average? Or loop up to five times on each block and try to get each block to equal its original size, with a variable tracking deficits/surpluses that need to be balanced out by future blocks? Or something?
2. Right now, if I manage to get lucky on a tricky file and hit on a combination that works, the next time I run the program it will randomize it all again. Is there a good way to keep track between instances of running the program of a working combination of sort orders, so that the next time I run the program, it does it in one try?

Sorry if my descriptions are confusing, I'll attempt to clarify points if someone has questions.

ROM Hacking Discussion / Acceptable to include patchers in download?
« on: November 13, 2018, 04:41:18 pm »
Just a quick question. I created an exe to quickly install a hack I made. Thing is, it relies on 3 other existing utilities in addition to a custom one: xdelta3, cdpatch (from the old cmdpack), and psx-mode2. I'm trying to make this as simple to install as possible, and asking people to go download 3 additional programs seems a little much. So my question is, is it acceptable to include the executables for these three programs in my download, so people can just run the installer without the extra hassle? Obviously I'd include credits for the programs in the README, but I'm not sure if this is considered okay in this context.

ROM Hacking Discussion / PSX - Editing unusual TIMs
« on: August 06, 2018, 12:40:00 pm »
I'm having TIM troubles. I'm trying to deal with character textures, but the format doesn't match the usual one that all of the TIM exporting/inserting/editing programs I've found expect.

Here's the header:
Code: [Select]
10 00 00 00 08 00 00 00 0C 08 00 00 00 02 F0 01 40 00 10 00Type: 4 bpp
CLUT size: 2048
Palette Org X: 512
Palette Org Y: 496
# colors in CLUT: 64
# CLUTs: 16

But according to everything I've found, 4bpp images should only have 16 colors per CLUT. TimViewer seems to assume this, and treats it as 64 16-color CLUTs instead of the other way round. Tim2View at least recognizes the correct specs. The way they're organized, the first color is black (transparency), then the colors are in the next 15 indices, and the rest are black transparencies.

This causes the problem that any program I use to edit the image exports 16-color palettes (and keeps reordering them). I've tried the GIMP plugin, low-color bmps with, and usenti, but I'm not having any luck preserving the palette. The one time I managed to replace the texture without the game breaking (using GIMP), the palette was still corrupted, leaving transparencies in weird places because certain pixels got changed to black.

The other issue is that the first 5 CLUTs have non-zero values in the first 16 indices, which means there are 5 UV maps (am I thinking of this correctly?). The last 11 64-color CLUTs are all 00s. But every time I edit the TIM, it chops off the last UV map. So when I save a new TIM, I end up with a CLUT that's 512 bytes long instead of 2048, and contains only 4 color-filled CLUTs, instead of 5. I got the one palette-corrupted texture to work by adding in 1536 bytes of 00s, but that still meant I was missing one of my UV maps (or rather it was fully transparent).

Is there any way I can work with these files to preserve them correctly? Also, how do you work with colors in a multi-palette TIM, when you can't see the true colors as they appear in the game?

I'm thinking specifically of NoOneee's Legend of Dragoon patches. The way they made their Undub patch for instance requires both a US and a Japanese copy of the game. It extracts the appropriate files from the Japanese game image, and inserts them into the US one.

What I'm curious about is whether it's okay to use the Japanese version of the file to create a patch for the US version, like you would for a file you modified yourself. Then people could just apply the patch to their US version without needing the Japanese version. Is that acceptable, or not because the user would effectively obtain access to code from a second game version they don't necessarily own?

ROM Hacking Discussion / Help with PSX compression
« on: April 16, 2018, 03:49:47 pm »
In The Legend of Dragoon, the text for menus, items, weapons, etc. is stored in files with a .OV_ extension, and is compressed somehow. Here's a snippet:

That's the weapon list. It reads, in order:

Broad Sword Bastard Sword Heat Blade Falchion Mind Crush Fairy Sword Claymore Soul Eater Axe Tomahawk Battle Axe Great Axe Indora's Axe Rapier Shadow Cutter Dancing Dagger Flamberge Gladius Dragon Buster Demon Stiletto Spear Lance Glaive Spear of Terror Partisan Halberd Twister Glaive Short Bow Sparkle Arrow Long Bow Bemusing Arrow Virulent Arrow Detonating Arrow Arrow of Force Mace Morning Star War Hammer Heavy Mace Basher Pretty Hammer Iron Knuckle Beast Fang Diamond Claw Thunder Fist Destroyer Mace Brass Knuckle

I looked it up in the RAM dump. It decompresses to this in-game, with a massive pointer table at the end of each text block that points to absolute positions in RAM:

It's some sort of dictionary compression, I just don't know how it works. For instance, there's a sequence at the end of this text block in RAM that's 0xFFA0000000000100 repeated five times, and that spot in the original file has 0xB5 repeated five times, so 0xB5 is clearly a dictionary key to that particular sequence of bytes. The game includes the I guess standard English dictionary table sequence of 0x00-0x7F, as well, so I know that's the start of the dictionary.

Does anyone know what type of compression this is and how I can decompress it?

UPDATE: I will be using this topic for updates to all of my non-Script Overhaul LOD hacking-related activities now, as well. See this post and this post for the most recent updates!

Hey now, Dart, that disc took a lot of work... (and can be downloaded here:

ORIGINAL: I've been working on this for a while now, but I feel like I'm finally far enough in this project to post a topic for it here.

This project is intended as a fix for the (often amusingly) poor official localization of The Legend of Dragoon. To be clear, I'm calling this a "script overhaul," not a retranslation, as it is based off of the English version of the game. The aim isn't to create a more technically accurate translation of the game, but to edit and improve the script that English-speaking players already know and love, in spite of its foibles.

This is also more than a simple grammar fix. Some of the changes I make to individual lines are quite substantial, but I always strive to make sure that the original intents of the lines and personalities of the characters are retained.

In general, what I'm trying to do is:
  • Fix poor grammar and punctuation
  • Better word choice
  • Remove bowdlerizations
  • Rewrite clumsy sentences
  • Improve dialogue flow
  • Generally add more "color" to dry dialogue, and make sure characters say things that humans might actually say
  • Expand some NPC dialogue, so that they have more to say (that is, the dialogue seen when returning to locations later in the game)
  • Quality of life upgrades, like making item/spell descriptions more explicit, and displaying chest contents even when inventory is full

I am doing this for all the text in the game (eventually). That includes the main story, NPCs, environmental objects, combat, and in-engine cutscenes. For now, the project is just me, with some editing help on my altered script by a friend of mine. The script on the first disc alone is well over 40,000 words, so it won't be finished for quite some time. Progress is steady, though, if not very fast.


Script Overhaul
In Progress:
  • Disc 2 script: 0%
To do:
  • Disc 3 script: 0%
  • Disc 4 script: 0%

Other LOD mods
To Do:
  • Extend NoOneee's Undub mod to include all voice audio: 90%
    • Include pre-rendered FMVs: 100%
    • Include remaining Dragoon voices: 0% (currently no way to replace LODXA00.XA)

Legend of Dragoon Modding System (LODModS)
In Progress:
  • Open-source LODModS tools v1.0a: 10%
To Do:
  • Hacking tools documentation: 1%
  • A lot, probably: ???%

Example Images
Italics proof of concept:

World map text change proof of concept:

ROM Hacking Discussion / PSX - Relocating pointer table
« on: February 27, 2018, 02:50:44 pm »
So I've run into an issue where the pointer table is located after the text it points to. I would like to be able to expand the text as needed, which means I need to be able to move the pointer table. But to do that, of course, I need to find and alter the commands that indicate the pointer table offset.

I've partially traced what's happening in the disassembler using pSX debugger and PCSXTrace:

Code: [Select]
800161c8 00041400: SLL     00000014 (v0), 140000fe (a0), 10 (16) -> this 140000fe is used to calculated several different offsets
800164b4 8e030008: LW      00000014 (v1), 0008 (800bc070 (s0)) [800bc078] -> load base address
800164b8 00021383: SRA     00fe0000 (v0), 00fe0000 (v0), 0e (14) -> calculate relative offset of pointer table
800164bc 00621821: ADDU    801a6d3c (v1), 801a6d3c (v1), 000003f8 (v0) -> add offset to base address to get pointer table location
800164c0 00081080: SLL     000003f8 (v0), 00000000 (t0), 02 (2)
800164c4 00431021: ADDU    00000000 (v0), 00000000 (v0), 801a7134 (v1)
800164c8 8c420000: LW      801a7134 (v0), 0000 (801a7134 (v0)) [801a7134] -> load first word at pointer table location
800164cc 00000000: NOP   
800164d0 00021080: SLL     ffffff6f (v0), ffffff6f (v0), 02 (2) -> pointer

I think what I need to do is change that 0x03f8 to the new relative offset of the relocated pointer table, but since it's calculated by a bit shift, there isn't really any flexibility. Plus, I have no idea how to find where in the disc image that command is stored into memory from.

I'm at a total loss right now. Does anyone have any experience with moving pointer tables? Is this something that can be done on PSX?

ROM Hacking Discussion / Help rebuilding PSX ISO with cd-tool
« on: January 31, 2018, 02:51:41 pm »
So I'm working on a text hack, but after inserting some of my script, the file that contains all the text has exceeded the original file size, so CDMage won't work. From what I've read, the best way to deal with this is using cd-tool to rebuild a new ISO.

I've read the documentation that comes with the program, but I'm not finding the explanations particularly clear. There also isn't a practical example of a script for the ISO rebuilding process, and I can't find anything like that anywhere.

Is there anyone with experience using cd-tool who can help me out? A more in-depth explanation of how to use the program, or ideally, a working lua script that someone has used for a game so I could see how it works?

Newcomer's Board / Cartographer and bit shifting
« on: April 21, 2016, 03:28:24 pm »
So I've looked around but can't find a solution to my problems using Cartographer with The Legend of Dragoon ROM.

There are a couple of problems dumping the script from this particular ROM that I've run into:

1. It uses relative pointer tables, but the hex values in the ROM are left shifted by two bits at runtime. Considering pointers seem to be 32-bit and the text occurs in relatively small chunks should not even require more than 16-bit pointers, I don't know why they do this, but that's how it is. Is there any way to have the pointers calculated on the fly when script dumping with Cartographer, while still keeping the original pointer values for later insertion? I guess I'd need Atlas to do something similar when updating pointer values.

2. After every segment of text (so after the end character $FFA0), there are either 8 bytes defining the dimensions of the text box that the text is displayed in. These bytes have their own separate relative pointer table that occurs immediately after the pointer table for the text. I'd need to dump and reinsert these bytes with updated pointers as well, since they're interleaved with the text. I don't think Cartographer supports having multiple pointer tables though, so the characters get dumped alongside the rest of the text, without their own pointer references.  Is there some way to use multiple pointer tables with their own separate base addresses, and have them dump the bytes sequentially? So [text][box dimensions][text][box dimensions].

3. The text box dimensions always start at the start of a full 32-bit word, so if the preceding text end character ends at the 16-bit point, it pads out the the remaining two bytes with $00 to keep everything word-aligned. Would the best way to deal with this to define both $FFA0 and $FFA00000 as end characters?

I haven't looked as much into Atlas yet, but I imagine some of these questions will apply to that too.

Note: There's a lot of lead-up, but this is mainly about how to deal with text and pointers.

This is something I've wanted to do for years, as I have enormous nostalgia for this game. Anyone who's familiar with it knows that the translation is atrocious, and every time I'd replay it I'd think about how I'd write a particular line. I tried several times over the years, but could never locate the text in the iso I had. I have yet to read about anyone else doing so either, though I've seen several people try.

Anyway, I decided to give it one more go, and this time I found the encoding table they used in the game files! So I have that, and now I can search and read all the dialogue. As a test, I even changed a letter in the .iso, and this worked fine. What I want to do though is be able to shorten and lengthen lines of text as need when rewriting. So a full translation hack, translating from LoD English to real English.  :P

To start, I have .iso images that I downloaded, not BIN/CUE files from the original discs. I'd use my own rips, but the final movie on the last disc is corrupted on my copy.If I convert the iso to bin/cue, the cue just has a single track instead of the file architecture. I don't know if this is a problem. I was able to save all the files from the iso using jPSXdec though, so hopefully I'm okay. The structure looks like this, if that helps.

Next, the hex structure, in case it's helpful:

So DRGN21.BIN contains all of the background texture assets, plus the dialogue, all mashed together in a confusing jumble. The text is mostly consecutive within a "scene", but the scenes are jumbled weirdly (and occasionally there is duplicate dialogue for some reason). Text is preceded by one of several headers: two for initiating text boxes (two different types), one for new line, one to close out a text box, and some other less major stuff. After the full text for a text box closes, there's a short block that sets the width and height of the text box itself. This is followed by the header for the next text box. The text itself uses two bytes, where the second is always 00, so there is a space between all the characters. Like so:

I considered the idea that it used sequential text, but I don't think that's possible. The game occasionally has choices, so it needs conditionals. Not to mention some things are out of order even within a scene. In fact, in that screenshot I think there's an unused line. So somewhere there are pointers. But I don't know which file they're in, how to find them, how to calculate knew ones, or anything like that. Any suggestions on how I go about doing that? I've read about pointers, but I still find myself completely lost here.

A couple of side question:
1) If I add or remove bytes of text, doesn't that change the offset of everything downstream from that point? So would I have to calculate new pointers for every single thing in there every time I change length? Because since it's a huge file that contains images as well as text, I don't think that's feasible.
2) Why is it bad to edit the iso instead of the component files on the disc image?

Any help would be greatly appreciated. Including assistance on this project, if anyone's interested.

Pages: [1]