News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - theflyingzamboni

Pages: [1] 2 3 4 5
1
Alright, so I don't have anything new to show right now, but it's been a while and I wanted to post an update on the state of these projects, in case anyone is following this thread. Overly wordy, because that's just how I do with these things. Despite lack of updates, I have actually been working on stuff when I have the chance.

First off, the Script Overhaul, which I assume is what most people are interested in: Unfortunately, any further releases are still in the far distant future. However, I have (finally) started working on the Disc 2 script! No timeline, but after *checks* 9 months since the Disc 1 beta released, I have begun working again on my actual original mod project. Progress should also be smoother in the future, due to what I've been doing instead, which is a lot of setup work.

Part of this setup work has been continuing my other major project, coding my LODModS toolset, so that I don't have to keep interrupting script work with fixing bugs or creating new tool features necessary for modding. At present, I am slowly but surely going back through my tools, bugfixing, optimizing, improving, and, perhaps most importantly, documenting my code. Which is necessary, because I plan to release it at some point as an open source beta on github so that a) other people can create translations into their own languages or create other LODModS-style mods, and b) so other modders can potentially contribute features to the toolset (it's in Python), because I am spreading myself pretty thin on these projects with the time I have available (not much).

The other part of this setup work was dumping and sorting the full script into a readable state, which I finished with a couple weeks ago. This was the most important pre-editing step. With all 4 discs dumped and sorted, I won't have to stop after each disc to get things together for the next, so the process will be smoothed considerably. Like the toolset, I will make my sorting fields publicly available for modders. Also learning from disc 1, I have set everything up in such a way that I can make all my edits in a sorted spreadsheet created automatically from the script dumps and insert scripts from that, instead of the horror-show of typing all the dialogue into a Word document, editing it, then copy-pasting and formatting lines into text files. Because I really had no idea what I was doing.

So that's what I've been doing the past few months. As for the future, expect things to continue to be pretty slow. My free-time situation changed drastically this April, so I can't work on modding as much as before. My plan at the moment is to continue working simultaneously on Disc 2 and coding. I am also considering doing a story-only pass on the script and ignore random NPC dialogue entirely for the moment, rather than a full overhaul disc-by-disc. That way, the overhauled story script could be available sooner, while work then continues on the less important (and less read) incidental dialogue.

If you read all this text, then congratulations, you really are an RPG fan.  :laugh:

2
Given those instructions, you shouldn't need any assembly knowledge. If you know any kind of programming language, you can write a function to decompress those graphics, just by translating those instructions into code.

Simplest thing to do in general terms is open the file in binary read mode, and go through it byte-by-byte. Read the first 8 bytes (after the header if it has one), and store them in an array/list/whatever, depending on the language you're using, so it can be indexed when the proper instruction is read (the 0x07-0x0e instruction). Once you've done that, read each byte in order, and split it into nibbles (half-bytes).

nibble1 = byte >> 4
nibble2 = byte & 0x0f

You then want to have some kind of switch or if/else if control flow structure with a case for each of the opcodes. You'll test the first nibble against the cases to get into the logic for the matching opcode, then you'll execute that logic, reading additional bytes as necessary. The instructions are pretty straightforward for each opcode, and can easily be translated into code. As you read real 4-bit values (not opcodes or parameters), append them to a bit array. Loop through the file until you hit the end decompression opcode, at which point you have a decompressed file in your bit array, which you'll write to a new file.

3
Programming / Re: Adding support for Shift-JIS in hex editor HxD
« on: March 14, 2019, 09:02:39 pm »
If I can make a sort of related feature request, would you be willing to add support for user-defined encoding tables, like WindHex does? HxD is my go-to hex editor for most things, but I do all my hacking on a game that defines its own encoding. So any time I need to search text, I have to use WindHex, which is a rather clunky old hex editor that feels much worse to use, but has the advantage of letting users load encoding tables defined in text files. I would love, love if you added this feature to HxD so that I didn't need a secondary editor.

4
Awesome work, thank you!!!

Is there any chance you could also remove the mod chip protection so that it is playable on a real console? I'd hate to play it on an emulator.

The current anti-mod patches have a ton of trainers that I don't care for, add an ugly intro, and interfere with things like the PS-IO.

If you can't or don't want to, don't worry, still an awesome effort!
I don't think that's the way it works (and I'm not sure what you mean by "anti-mod patches" and "trainers").

PSX discs were printed using a special burner that you just can't replicate with a regular PC disc burner. There is nothing that I could do to the disc image that would make a CD-R playable on a physical console when you burned it to disc, because it's how the data is burned to disc that prevents you from playing burned CDs. I found a good explanation here: https://www.youtube.com/watch?v=XUwSOfQ1D3c

You can still play a burned disc if you want, but you'll need to either mod-chip your console or use the disc swap method (assuming you have a PSX, I have no idea what to do on a PS2 to play burned games).

5
Micro-update: The encounter rate bugfix (standalone and Mod Pack) has been updated to v1.2, and now includes a no-encounter patch version. (Note that no, half, and double encounters do not currently affect world map encounters.) Links in the OP.

Very much looking forward to this!

Tried playing again about a year ago and the dialogue was so bad, I couldn't keep playing.

Playing through disc 1 now. Much better so far. Thanks!
Glad to hear it!

6
Just realized yesterday that changes to chest text that were intended for the 0.9 release never made it in. Oops. It's just a simple quality of life change to tell the player what a chest contains even if their inventory is full when they try to open it. As of v0.92, these changes are now included, and all relevant download links have been updated.

7
A couple of quick announcements:

LODModS patcher version 1.3 (http://www.romhacking.net/utilities/1445/) is out now, with significant changes to the user interface (no more manually editing the config file!). Additionally, checksum validation has been disabled for xdelta, meaning that patches targeting the same areas of a file can be compatible, so long as they aren't targeting identical bytes. Everything mod available now is compatible.

Additionally, the directory structure of the patches folder has been changed, and all LODModS mods have been updated accordingly (Script Overhaul, Mod Pack, Full XP, Encounter Rate Fix). This means that older mod versions won't work with patcher v1.3, and vice versa, so everything will need to be redownloaded (all links in OP).

Second, I just released an Expanded Inventory mod that raises the consumable item cap from 32 to 64 (http://www.romhacking.net/hacks/4360/). As with the other mods, it requires patcher v1.3 or greater. It has also been included in the most recent update of the LODModS Mod Pack (http://www.romhacking.net/hacks/4348/), and is compatible with all other LODModS mods.

8
I disagree with that logic entirely.

Not every translation will be, or should be, entirely literal. Non literal translations can be acceptable, but what matters is why it is non-literal. Removing something because it violates someone's taboo or might offend some hypothetical person is a bad reason for translating something non-literally. It was bad when the religious right did it in the 90s and it's bad when people on the opposite end of the spectrum do it today (not that I'm accusing anyone here of belonging to either of those two groups ;))

Additionally, the current climate has made a lot of people very sensitive (for good reason) to anything that looks even slightly like censorship. Perhaps someone might be upset by the original name of the enemy (although I can say, based on a lot of unfortunate personal experience, that any Jewish person who spends enough time online to get interested in romhacking 20-year-old games will have seen things a million times more offensive than whatever the name of this enemy may be), but I can guarantee that some people will be offended if they think that a translator is deciding which parts of the original developers' creative vision they are or are not allowed to see.

That sort of reasoning is incredibly easy to turn around.
Quote
I disagree with that logic entirely.

Not every translation will be, or should be, entirely censored. Non censored translations can be acceptable, but what matters is why it is non-censored. Leaving something deeply offensive in because it violates someone's idea of creative purity or might offend some hypothetical person is a bad reason for deciding to translate something literally.

Additionally, the current climate has made a lot of people very sensitive (for good reason) to anything that looks even slightly like hate speech. Perhaps someone might be upset by a small change to the original name of the enemy (although I can say, based on a lot of unfortunate personal experience, that any retro-gamer who spends enough time gaming to get interested in romhacking 20-year-old games will have seen changes a million times more extreme than altering whatever the name of this enemy may be), but I can guarantee that some people will be offended if they play a translation where the enemies they're slaughtering are named after a religious sect that still faces threats of violent hate crimes.

The factors for and against a change like this need to be weighed for each individual change, and shouldn't be decided by blanket ideas of censorship and creative purity. And when I put "someone might be upset because this game seems to demonize a real group of people" and "someone might be upset because one word was changed from the original Japanese in an old game they like" on the scales... the second carries zero weight, even in what I would consider this generally inconsequential issue, where the odds of either of those possibilities is low. A video game is not a museum or history book, it's not necessary to keep every single wart of the past on general principle for the future's edification, especially since the original content is still available.

So I stand by my original post. If OP feels like the implications of the original name warrant a change, then change the name.

Side note: "People have seen worse" is also a terrible argument for anything, and generates all kinds of logical absurdities. "This bad thing is okay because it isn't the worst thing" can be applied to argue that basically anything is acceptable.


9
If you think it's a problem, take others' advice and change it. Some fans get too hung up on everything being exactly literal in translations. That's why devs 'localize,' they don't just verbatim translate, because some things just don't translate well. Personally - and I know many will disagree - I would consider a translation that takes no effort to localize to be mediocre at best.

Considered another way, it won't hurt anyone's experience if the name isn't a perfectly literal translation, but has at least a small chance of hurting someone's experience if it is, so why not change it?

10
i was basically suggesting you to create multiple patches for those cases, but let the utility hide that from the end user.

perhaps disabling validation will work for your use cases, but i believe there is always the risk of one of the patches referring to a region that another patch alters.

Ah, I see. Yeah, that was one thought I had, but I'm really trying to avoid the multiple patch version thing. That can easily get out of hand if you have to add, say, a third overlapping patch. Suddenly you need a whole lot of different versions.

And yes, I agree there are definite risks, but right now, I think they're worth it. My intent is to create a system for modding this game that other modders can use, which I think is feasible, considering there are only two of us actively developing mods for LoD. Getting new people to utilize a specific system is easier than asking vets to change their ways. As part of this, I can try to encourage modders to be fully aware of what they are changing and share that info, so other modders will be able to report any compatibility users in their documentation. I'll probably try to keep a compatibility masterlist available somewhere as well.

So right now, the patcher stops itself from running if it detects overlaps in the patch list. I'm going to change that to a warning about compatibility, instead. Hopefully, between that and good documentation, the pitfalls can be avoided, if they're ever encountered at all. Perhaps I can have the patcher run a simple version check on the clean disc images first, just to keeps some manner of safeguard in place.

11
ROM Hacking Discussion / Re: Something like xdelta that can stack patches?
« on: February 07, 2019, 10:14:46 am »
If checksum validation is your only problem, xdelta allows you to disable it completely.
:banghead: :banghead: :banghead:

I dunno how many times I've looked at the option list for xdelta, and somehow I never noticed that was there. The patches stack fine with that switch set. Thank you, that's exactly what I needed to know.  :cookie:

12
ROM Hacking Discussion / Re: Something like xdelta that can stack patches?
« on: February 06, 2019, 05:22:14 pm »
how about a custom patcher that hides the complexity from the end users?
I'm not quite sure what you mean?

If you're talking about a utility to handle the patch application process, I've already created one. Just requires the user to specify some directory paths, and handles all the file extraction/insertion/decompression/compression/patching behind the scenes.

I'm just hoping to find a patch format that fits my needs a little better than xdelta to use with it. I get that there are good reasons for file validation when patching, but I'm already running into situations where I've got bytes I need to change for different mods in close proximity.

In this specific instance, I have an inventory capacity mod that alters a couple values in a single 2048-byte decompressed block, but that same block also contains a couple pointers for my text mod. I'm hoping there's a patch format that lets a user stack these two sets of changes, which isn't possible if there's checksum validation, because once one is applied, the other will fail.

If you're suggesting that I design my own alternate patch format as an alternative to xdelta, that might be a little complex for me, and I'm trying to avoid that.

13
ROM Hacking Discussion / Re: Something like xdelta that can stack patches?
« on: February 06, 2019, 11:33:08 am »
"to create and apply diff patches to small component files"
The file/file system aware approach is the method most would use to get to your position. Usually in the form of a batch file, something that can rip files from a given system's ISO/ROM and then return it after patching.
You can drop down a level lower and do pointer, or possibly content, aware stuff if you must (though you will likely be coding it yourself). You usually see this for high level game editing tools but no reason you could not do it here.
This is what the patcher I wrote does. Other than STR and XA files, the game has two main types of data- and code-containing files: MRG files and BPE files. The MRG files are composed of a bunch of smaller data files indexed by an LBA table at the head of the MRG file. Some MRG files are composed of MRG files themselves, up to a few layers deep. Since I can just change the address and file size in the LBA table, these are the files I can alter the size of if I need to. In my case, it's just text overflow, rather than code.

The BPE files are byte-pair encoded by block, usually 2048 bytes per decompressed block, sometimes less. When I mod these, I decompress only the blocks I'm interested in, then recompress them. Due to where many of these files are loaded in RAM, I don't change their sizes.

So the process I use is: extract game files from disc -> decompress blocks/extract subfiles to smallest possible LBA-table-referenced unit -> patch -> recompress/reinsert subfiles -> insert game files into disc

Stacking theoretically dynamic/size altering patches though is mathematically tricky, even more so when you consider most such dynamic/relocation scans are computers doing something like a compression algorithm and not humans assigning meaning to sections of code.
I think the BPE's are the only ones where I need to stack patches, so I actually doubt I'll need to stack size-altering patches.

What I need is something that can potentially stack static patches or change file size, but does not have to be able to do both at the same time.

"to make sure they aren't going to modify the same bytes"
Which will work until you repoint something and thus render something else redundant. If you are dealing with pointers that is not even a rare case. If you know you are not going to do this and thus have a limited case then so be it.
I consider this scenario unlikely, if I'm understanding you correctly.

14
SOLVED: Apparently xdelta could do that the whole time. D'oh!

I've been working on various mods for The Legend of Dragoon and created my own sort of Frankensteined patcher for the game. Right now it uses xdelta3 to create and apply diff patches to small component files, which has worked fine so far. But I've reached a point where I have two mods that need to patch the same file, and there's no way to do that with xdelta without going down the road of multiple patch versions. And that way lies madness and terrible user experience. Also, one of these mods expands the sizes of some of these files, so simple offset replacement won't work.

So is there a patch format and command line patching utility that allows for:
  • insertions/deletions and changing file size like xdelta
  • patch stacking

As an added bonus, if possible, I would like to be able test the patches before applying them to make sure they aren't going to modify the same bytes. So if someone knows of a patch format that fits these requirements, I'd also be interested in the specifications and how they could be compared.

15
Alright, major project updates!

First, I'm pleased to announce the debut of the Legend of Dragoon Modding System, or LODModS, a new, simple-to-use system for applying mods to LoD! Up until now, mods for the game have tended to each use similar but differing installation methods, forcing users to have to rename disc images and move files around to apply them all. Further, no thought was given to mod compatibility, nor was there any sort of documentation on the subject to help users figure out how to deal with mod conflicts.

LODModS is my solution to these and other issues, by designing mods to be installed by a single patcher (Frankensteined out of custom code and freely available utilities), and reducing their file footprint where possible. The patcher is easy to setup and use, only requiring the user to specify a handful of folder and file names in the config file before running the executable. Currently, only the patcher is available, but once the full toolset is in a more complete state, I'll release that too so that other modders can generate mods designed to work with the system.

Second, I have also released the v0.9 update to my Script Overhaul Disc 1 beta. In addition to a few small typo corrections, this update contains improvements to all of the item/spell/system/battle menu text, generally focused on readability and making item and spell descriptions a bit more explicit (and accurate). For instance, Dragoon spell descriptions now have the actual damage percentages. These changes are applied to all 4 discs. The installation method has have course been updated to use the LODModS patcher.

Third, while testing other mods for the game, I discovered that NoOneee's Encounter Rate Bugfix is actually itself bugged. Because of this, I went ahead and made my own fix for the original bug using a different algorithm. The encounter rate should now be roughly the same regardless of movement direction and speed, as originally intended. Additionally, I have included alternate versions to halve and double the rate.

Finally, to make full use of LODModS, I have updated all currently complete LoD mods to work with LODModS, including both my own work and mods by other authors (with permission), and bundled them into a single LODModS Mod Pack. In some cases, I have also extended or otherwise updated those mods (details in the readme files). The Mod Pack contains my Full XP and Encounter Rate Bugfix mods, NoOneee's Undub and Rose's Blood mods, and Zychronix's Half HP mod. Aside from my still-incomplete Script Overhaul mod, this is currently a one-stop-shop for all of LoD's mods.

Links are here, and also in the OP:
LODModS Patcher: http://www.romhacking.net/utilities/1445/
Script Overhaul beta v0.92: http://www.mediafire.com/file/587n5o4ue5gkcs5/LODSO_Disc1_beta.zip/file
LODModS Mod Pack: http://www.romhacking.net/hacks/4348/
Full XP: http://www.romhacking.net/hacks/4072/ (mainly archival, also in Mod Pack)
tfz's Encounter Rate Bugfix: http://www.romhacking.net/hacks/4347/ (mainly archival, also in Mod Pack)

16
By the way... I'm not very PC savvy so I'm unclear how to intepret the readme installation instructions. Something about renaming a disc image file? But I couldn't find that. I'm playing it on an emulator EPSXE and using a rom. The rom I got came with a .bin and a .cue file, but neither is a disc image as far as I can tell. Your assitance (or anyone's assistamce) would be most welcome.
Didn't have notifications turned on, so I didn't see this comment before. DrewUniverse is correct on both counts. Disc image means it's a snapshot of the data on the disc, which is what BIN/CUE and ISO files are. A "ROM" is similar, but refers to an image taken from something like a cartridge, though I realize it's often used as a general shorthand for game rips. I'm so used to the terminology that it didn't occur to me that others are not. I'll try to be more explicit in the instructions on the next release. But yes, if you haven't done it already, just rename the bin to the name given in the readme, follow the other instructions, run the patcher, and then rename it back to whatever it was originally called (or update the name in the CUE file).

Next update, the process will be slightly different. No file-renaming necessary, but it will still require some work to set up. I'm using a configuration file where people will just type in the path to their game, and the file names for the disc images (ISOs, BINs). Might be slightly easier to understand, I hope.

17
Programming / Re: Recompressing a file to original size
« on: December 19, 2018, 02:43:44 pm »
i think im still just misundestanding here but i understood that you have a linear sequence of compressed blocks and you simply took one bock (or few blocks) from the middle of the sequence, and made it smaller and then padded the end of that block with zeroes.

my though was to take all the blocks, and modify what you need, and then linearily arrange them back, then reinsert the entire sequence of blocks to the game and pad the end of the last block only with zeroes. (from a programmatical standpoint, doing this should not be too hard as you could propably fairly easily implement a way to fetch the compressed size of the block you compress and write it's size in it's place.)

this way the entire sequence would stay intact and unless the compressed sequence actually contains something that makes it seek anything specific at the end of of the sequence, by all means it has no reason to not work.

either way, it's good to hear you got it working (whatever it was that caused it to not work to begin with)

Ah, a failure of explanation on my part. What I meant was what you said. That I would have a file with say, 30 blocks, decompress blocks 14-18, and when I compressed them back into place, if the full file was smaller I would pad the end of the file with zeros (in this example, after block 30), not the end of the segment of blocks 14-18. By the metric of "does this create a functioning file," it works.

However, if you read further down in my last post, I try to explain the reason that's not really the solution I'm looking for, as it creates problems down the line for mod compatibility. Hopefully, the explanation is helpful. It's hard to explain my reasoning, since it's tied up in a full modding process that I've been working on, with a lot of parts tying together, that's difficult to describe briefly. That's why my question was focused on resolving the problem presented, rather than asking for alternative solutions that would circumvent the problem (though I mentioned a couple sort of in-between solutions in the EDIT of my last post.)

(To address another part of what I think you might be suggesting, decompressing into 30 files of 1 block each would present a problem for the text inserter, since all the text and pointers would be broken up across multiple files. I have one where the pointers are scattered in locations across the entire file  ::)) Likewise, in cases with compressed textures, the texture file would get broken into pieces by this method.)

18
Programming / Re: Recompressing a file to original size
« on: December 19, 2018, 12:53:29 pm »
(Apologies for the tl;dr, but I'm trying to explain an entire process and chain of logic here, to answer the inevitable questions of "Why not just do this?")

So zero padding the end was the first thing I tried, but it caused the game to crash. Just tried it again and it worked though, so I guess the bug was something else that I've since fixed.

Regardless, I still have a reason to do things the way that I am, and that's mod compatibility. Obviously, the compressor I wrote can deal with entire files at once, but dealing with them in segments improves mod compatibility. If I patch the entire file, then only the one patch is viable. But if I patch individual segments only, then any future mods that use that file will be compatible so long as they don't target any overlapping blocks.

Additionally, the way I handle segments, I write the start and end offset of the segment to a metadata file when I decompress. That way, when I compress it again, I can just copy the data before and after the segment I'm actually compressing, which is a lot faster than always decompressing and compressing every block (especially in one case where I'm targeting 6 blocks out of about 450). But of course, if I am doing multiple segments in a batch, if I compress one and it alters the file size, suddenly the metadata for the other segments is incorrect. I realize this could be resolved by using a looping workflow of decompress/extract single segment/file -> insert script -> compress/insert segment/file, but that's a lot less convenient for working on mods than a workflow of decompress/extract all segments/files -> insert all scripts -> compress/insert all segments/files. Plus, these tools are intended for more general use beyond text insertion as well, where this might be even less convenient. Basically what I'm saying is that somewhere I have to sacrifice some degree of convenience, and workflow isn't where I want to do it. So I'm trying to do something that's computationally more difficult, but simpler for the person trying to mod the game.

I have at least solved the issue of the end user of the mod having to wait on a randomized process, though. When the compressor hits on a successful sequence of sort orders for the blocks, it appends that sequence to the metadata file. During patch creation, this sequence is attached to the xdelta file, then stripped off during patch application and attached to the metadata again at the user's end, guaranteeing it will succeed first time, every time.

That still leaves me looking for a less naïve implementation of the compressor to speed up the process of finding a successful sort order sequence, if anyone has any ideas.


EDIT: I just had a couple thoughts on a compromise solution. What if, when I do a batch compression of multiple segments within the same file, I just do it in reverse order from the highest-numbered set of blocks to the lowest? That way, if one results in a smaller file, it won't impact the location of the next segment to be compressed?

Attacking from the other end, would it be worthwhile, instead of using start and end offsets from a static metadata file to copy the data on either side of the segment, to run the decompressor from within the compression process to generate the correct offsets at runtime? It would cause the compressor to take a performance hit, but the compressed size of a segment would no longer matter beyond sometimes needing to be no larger than the original.

I can see advantages and drawbacks to both. Anyone have thoughts?

19
Could it be compressed block-by-block, instead of the whole file at once? That would cause the same combinations of letters to be defined differently when compressed, if they occurred in different blocks.

20
Programming / Re: Recompressing a file to original size
« on: December 05, 2018, 06:48:59 pm »
What is there in the original compressed section? It is uncommon to see an end of file/compression marker (I would usually expect that sort of thing in the header or decompression instruction/function) but it is not entirely unknown a concept.

I would not expect executable code to magically be there, however if it is then maybe rather than simple 00/FF padding go with NOPs or something. Though I just looked at http://www.mrc.uidaho.edu/mrc/people/jff/digital/MIPSir.html and NOOP is encoded as 0000 0000 0000 0000 0000 0000 0000 0000 apparently.
Yes, 0x00000000 is NOP. I describe how the original compressed section is arranged in my previous post. I think it's not so much that the compressed code has an EOF marker, more that it ends on a header for a subsequent block that does not exist. I was just hypothesizing why it breaks when padded. I'm not really sure why it happens. The other reason I don't want to pad is that I'm not always trying to compress the entire file. To maximize mod-compatibility potential, I don't patch the entire compressed file. I decompress only the blocks that contain the code I'm changing, patch that, then compress those blocks back into place in the original. If I pad those blocks, the game will be reading garbage as the next decompressed block size. So that's why I want a non-padding solution.

Pages: [1] 2 3 4 5