Romhacking.net

Romhacking => Personal Projects => Topic started by: ArcanaXIX on June 05, 2021, 11:41:06 PM

Title: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on June 05, 2021, 11:41:06 PM
This is my first ROMHack but it's been a lot of fun while also extremely frustrating. My primary goal is to replace all instances of a particular character's talk sprites (both the full-scale talk sprites and the smaller mini talk-sprites that accompany thought bubbles, etc), but this game is pretty un-documented from what I've found. So I've been mostly working from scratch. My "I'm dreaming" vision for a "complete" version of this would somehow implement a button or something that lets you turn this on/off (revert to the original as desired), but my primary focus is just getting it to work to start with.

Specifically, I want to play as the cat the whole time.  ;)

After a lot of trial and error, I recently made some tangible progress - I've found a kind of "master list" of the sprites in cpac_3d.bin (note: I found an ancient thread somewhere in my initial research that implied cpac_2d.bin was where I might find these, but that was not correct, all talk sprite data is in cpac_3d.bin. cpac_2d.bin appears to mostly contain background assets), around memory address 04ED4280. They are of the following format:

## ## ## 00 ## ## 00 80

(I found them while poking around in CrystalTile's tile viewer because the 00 80 created these distinct black and salmon bars viewed with tile form GBA 8bpp. I may get into this later but that tile form is what you are going to want if you ever want to look at the background assets in cpac_2d.bin.)

Basically I found that if you copy the hex code of whatever sprite/asset you want and paste it over the hex code of the asset you want to replace, every single instance of that asset in the game will be replaced.

I actually have a full list of the hex codes pointing to all of the talk sprites. I can post a list later if there is interest. What concerns ME, are the protagonist's talk sprites, values listed here:












9C4003007C190080neutral smirk
185A030044190080neutral frown/serious
5C730300E4180080hand on chin/thinking
408C0300F8180080concerned/shocked/sweating
38A50300501A0080the angular gag "shocked" face
18D90300BC090080ghost with sunglasses
D4E20300D81B0080shrug with one palm out
ACFE0300CC190080mouth slightly open/gape/slight surprise
7818040098180080Facing down, flat line lips, serious and melancholy
103104009C180080angry/emotional yelling

And the code for the cat, which I want to use to replace all of these, is:

AC 49 04 00 54 10 00 80

(In addition for these full-size sprites, I also wanted to replace the mini-talk sprites, but thankfully that was easy. I replaced 502A0D00E4020080 at memory address 04ED45E8, with 342D0D0064020080.)

The good news is that I playtested the first 14 chapters of the game and confirmed that this works perfectly.





The bad news is, as anyone who has played this game knows (and this is unfortunately impossible to discuss without game spoilers so I apologize)... there is another character who uses those same sprites from chapter 15 onward, and well, I have unfortunately confirmed that this janky way of doing it turns him into a cat too:



Moreover, both characters - protagonist and this guy - use almost the full range of sprites in the chapter, too. There is virtually no easy way about this as it currently stands.

My other problem is that I had hoped to eventually give the cat a full range of original emotions/reactions to replace the originals, but that is going to be extremely difficult because the source images are in a format I can't make heads or tails of. All of the data in this game does not seem accessible using any of the NDS tools I've tried - they all find the main containers (e.g. cpac_3d.bin, cpac_2d.bin, etc...) but not the individual files within, likely because the format is unrecognized. I've been able to get as far as I have mostly by blacking out random blocks of hex code/tiles and loading the ROM to see what changes. Unfortunately, I'm hitting the limits of what I can do with that approach. For example, I narrowed down that the location for the sprite of the protagonist "thinking" is from ~04F0D340 - 04F0EC40, because when i messed with the data in that portion the sprite got messed up; but I have no idea how to actually make sense of it as a file and CrystalTile doesn't see any individual files within cpac_3d.bin.

At any rate, since I am probably going to need to look at the code for the specific scenes where BOTH characters are present, I have started looking at all the bits in this ROM with names like "st01/st01_game000_Expand.xml.lz". First of all, these are all compressed, so in order to read them I needed to export them from CrystalTile and decompress them with Batch77 and then re-compress them to re-insert and check my changes. I'm still digging through these, but so far, this is what I can gather on their format--


What I really need to understand is where these "m##_####" variables(?) are defined, and if I can figure that out, what part of that tells it what sprite to use. My assumption is that they must be defined in the specific stage .xml files because otherwise there should be no reason they can't load outside the ones already included. I've been trying to mess around with other bits and pieces of these files outside of the legible text but so far I haven't narrowed it down that far, and more often than not messing with it too much just causes the ROM to crash before it can load the scene at all.

So this is where I am currently at. Still trying to make heads or tails of how the data in this game works so I can try and figure out how to make it do what I want. Incredible how a simple fix that works for 70% of the game has turned into such a headache for the last few chapters.

Will certainly post updates here if I manage to find anything out that leads me toward a breakthrough. I know this is a very very niche mod for a very very niche game, but I am determined to figure this out. Any input is welcome also for other tools that may help, since as I mentioned this is my first attempt at ROMHacking and I'm learning as I go. :)
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on June 13, 2021, 12:29:53 AM
It's been about a week and I have made significant progress! I found what I've been looking for and have been able to implement it for most of chapter 1 and am confident it will work for the remainder of the game. It's going to take a while to actually DO the remainder of the game, and I still have extra/additional features I want to work on after I get that much done, but this is still huge.

The big breakthrough: "m##_####" variables, which I correctly realized indicate both dialogue AND associated talk sprite, are defined uniquely for every specific localization.

By which I mean that the talk sprite for any given line of dialogue is separately hard-coded for English, French, German, Spanish, and Italian on a case-by-case basis. There is nothing in the greater game###.xml file for each stage that can change this, you must edit the individual language .xml file(s) (e.g. en.xml, sp.xml, it.xml, fr.xml...). So I can change every single talk sprite in English and French will still look like normal.

(As such I am going to focus on just implementing this in English for now. I make no guarantees for any other languages down the line, but I'll try to provide some clear documentation so others could replicate this if desired.)

Once I realized this I was able to narrow down my search and have identified the following properties of talk sprites in the localization files (st##/st##_game###_Expand.en.xml, specifically):


The hex codes relevant to my project that point to Sissel's various sprites are as follows:

[26 - no sprite]
27 - default smirk (left)
28 - default smirk (right)
29 - default neutral/frown (left)
2a - default neutral/frown (right)
2b - thinking hand on chin (left)
2c - thinking hand on chin (right)
2d - sweating (left)
2e - sweating (right)
2f - angular "shocked" gag face (left)
30 - angular "shocked" gag face (right)
31 - ghost with sunglasses (left)
32 - ghost with sunglasses (right)
[33 - no sprite]
34 - shrugging with palm up (left)
35 - shrugging with palm up (right)
36 - mouth agape, slight shock (left)
37 - mouth agape, slight shock (right)
38 - thin line mouth, looking down, melancholy (left)
39 - thin line mouth, looking down, melancholy (right)
3a - angry emotional yelling (left)
3b - angry emotional yelling (right)
3c - cat (left)
3d - cat (right)

(As for the mini talk sprites, the default - as mentioned above - is 01, whereas the cat's is 08. I forget the sunglasses ghost one, I believe it is one of 0d or 0f but will need to double check when I get that far.)

This has given me a very straightforward method of replacing talk sprites that I've implemented for st01/game000, game001, game010, game011, game012, and game013 so far.


The specifics for what to find and replace are as follows:

For Mini Talk Sprites


* In cases where Sissel is using Yomiel's face (i.e. most of the game), simply FIND 1BFF0100 and REPLACE WITH 1BFF0800.
* In cases where Sissel is using the ghost with sunglasses, FIND 1BFF0D?00 and REPLACE WITH 1BFF0800. (I will come back and edit this when I remember what the actual code is, it may not be 0D.
* If there are any cases where Sissel and Yomiel both use this in the same scene it will need to be manually vetted.


For Full Talk Sprites (Left)


* In cases where Sissel is using Yomiel's face:












FINDREPLACE
08FF270008FF3C00
08FF290008FF3C00
08FF2B0008FF3C00
08FF2D0008FF3C00
08FF2F0008FF3C00
08FF340008FF3C00
08FF360008FF3C00
08FF380008FF3C00
08FF3A0008FF3C00

And:












FINDREPLACE
19FF270019FF3C00
19FF290019FF3C00
19FF2B0019FF3C00
19FF2D0019FF3C00
19FF2F0019FF3C00
19FF340019FF3C00
19FF360019FF3C00
19FF380019FF3C00
19FF3A0019FF3C00

* In cases where Sissel is using the ghost with sunglasses, FIND 08FF3100 and REPLACE WITH 08FF3C00, and the same with FIND 19FF3100 and REPLACE WITH 19FF3C00.
* In cases where Sissel and Yomiel both appear in the scene, hopefully one will be on the left and the other is on the right so you can just do one or the other, but if not it'll have to be manually vetted.


For Full Talk Sprites (Right)


Same deal as the left, but replacing all the remaining codes and pointing them to 3D instead of 3C.

* In cases where Sissel is using Yomiel's face:












FINDREPLACE
08FF280008FF3D00
08FF2A0008FF3D00
08FF2C0008FF3D00
08FF2E0008FF3D00
08FF300008FF3D00
08FF350008FF3D00
08FF370008FF3D00
08FF390008FF3D00
08FF3B0008FF3D00

And:












FINDREPLACE
19FF280019FF3D00
19FF2A0019FF3D00
19FF2C0019FF3D00
19FF2E0019FF3D00
19FF300019FF3D00
19FF350019FF3D00
19FF370019FF3D00
19FF390019FF3D00
19FF3B0019FF3D00

* In cases where Sissel is using the ghost with sunglasses, FIND 08FF3200 and REPLACE WITH 08FF3D00, and the same with FIND 19FF3200 and REPLACE WITH 19FF3D00.
* In cases where Sissel and Yomiel both appear in the scene, hopefully one will be on the left and the other is on the right so you can just do one or the other, but if not it'll have to be manually vetted.

SO. I know that's a lot but I just wanted to document that for clarity because it's very manual and it needs to be done for all 145 .lang.xml files in the ROM for every language you want the change to be reflected in.

That said; I've confirmed it does exactly what I expect it to in the scenes I've tested so far. The only exceptions are the log book and "rewind time" scenes, which still populate human Sissel (rewind pictured):



Furthermore, you MUST use the 08 and 19 preceding pointers and not just find/replace all instances of FF ## 00 because it will otherwise catch some false positives, resulting in uhhh miscellaneous text getting replaced with the letter "y":



At any rate, this appears to be working for now. I'm going to work through all of the en.xml files, and only when I'm completely finished with those will I try to pin down where the sprite information for the log book and "rewind time" might be if they aren't already covered by then.

The good news is, I am considering implementing this as a mode to select in place of the "language select" screen, since you need to do this one language at a time anyway and sprites appear to be language-independent. That is the next step on the road once I get these all done. :)
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on June 28, 2021, 02:59:58 AM
I'm still working on replacing all of the 08FF##00, 19FF##00, and 1BFF##00 pointers in all of the scene XML files, but I wanted to post an update because I have some unexpected good news! I was afraid that doing this in scenes with both Yomiel and Sissel would replace Yomiel's too, however it seems Yomiel has different associated codes for the same sprites! This means that - at least, as far as I've found - there is no danger of accidentally swapping Yomiel when done this way. I'll of course need to verify this with certainty, but as of right now, it's looking promising that replacing all of the above codes will not influence Yomiel.

(Other good news: with a few exceptions, in most cases Sissel only appears on the left side, so in most cases I just replace the left codes and then double check the scene to see if I missed any rights. Right-Sissel most often comes up in scenes with Missile, as in many of those Missile is on the left and Sissel on the right.)

I still have not found where the sprite for the log book on Sissel is defined, nor the "let's rewind time". My current focus is to finish all of the .en.xml files first, create a patch, then playtest that and make sure it doesn't either miss anything, swap any false positives, or cause unexpected crashes. Once I've done that much, I'll try to figure out where those two might be hiding. I believe it is likely still in the same format as the other m##_#### files, but that could be incorrect.

This also bodes well for the prospect of more unique mods down the line. If Yomiel has a different "palette" than Sissel even for the same sprites, then if I can find where that is defined, it might make it easy to swap all of them at once instead of this hands-on process, AND it might potentially pave the way to point Sissel's to a whole new rainbow of cat talk sprites. That would be much harder because I still haven't been able to figure out what format the sprites are in to begin with, but it would make it theoretically possible.

So, at the moment, no further progress to share, but I have a slightly better understanding of the ROM and hopefully will have a working release relatively soon.  8)
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on July 08, 2021, 10:07:41 PM
I figured out how to automate the hex code edits so I was able to finish all of these changes. I'll go into the steps below for documenting purposes but first an overview of where I'm at:


My first priority is to finish playtesting and verify that all of the gameplay talk sprites appear as expected. Once that's finished, I will look more into where these 3 exceptions may be pulled from. I am considering releasing an initial version without these if it takes too much longer to find, but if possible I'd really like to have a more complete initial version up before putting it out there.

These are the 3 problem screens:







I'm hoping they're indicated per-localization as well (since the rest of the game sprites are, I can't see why they wouldn't be, but... well, I've learned not to take anything for granted), so that eventually I can implement "Cat Mode" as a selection to opt out of for certain scenes if desired. We'll see.

As for the process! I was able to largely automate it, but I am playtesting because of course nothing is ever 100% easy and straightforward.

The Good News:

For the vast majority of the game, Sissel and Yomiel have different hex code pointers indicating their sprites, so running a script to replace all of Sissel's will leave Yomiel as-is.

The Bad News:

In some endgame scenes (specifically, it appears the ones where Sissel has taken on the ghost form and no longer uses Yomiel's sprites), Yomiel uses the hex code pointers that had previously been reserved for Sissel. So the script will turn Yomiel into a cat at the end.

The Workaround:


TL;DR - first turn all Human Sissels into Cat, then turn all Ghost Sissels into Cat, then look for examples where Yomiel was coded as Human Sissels and revert those as needed.

The tool that made this possible was XVI32: http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm

First I made individual scripts for "replace human" and "replace ghost". "ADR 0" between each replacement tells it to go back to the beginning of the file for each pattern to make sure it gets all of them.

Replace Human:

ADR 0
REPLACEALL 08 FF 27 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 29 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 2B 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 2D 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 2F 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 34 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 36 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 38 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 3A 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 28 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 2A 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 2C 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 2E 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 30 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 35 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 37 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 39 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 08 FF 3B 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 19 FF 27 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 29 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 2B 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 2D 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 2F 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 34 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 36 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 38 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 3A 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 28 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 2A 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 2C 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 2E 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 30 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 35 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 37 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 39 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 19 FF 3B 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 1B FF 01 00 BY 1B FF 08 00

Replace Ghost:

ADR 0
REPLACEALL 08 FF 31 00 BY 08 FF 3C 00
ADR 0
REPLACEALL 08 FF 32 00 BY 08 FF 3D 00
ADR 0
REPLACEALL 19 FF 31 00 BY 19 FF 3C 00
ADR 0
REPLACEALL 19 FF 32 00 BY 19 FF 3D 00
ADR 0
REPLACEALL 1B FF 0D 00 BY 1B FF 08 00

Then I was able to run these on every single .en.xml file that I had exported from CrystalTile2 and decompressed with BatchLZ77. I did it the lazy way by dumping all the .decompressed files in the same folder as the scripts and the XVI32 executable, so that all I had to do was open that folder in an elevated command prompt, and run:

FOR %f IN (*.decompressed) DO START /W xvi32.exe %f /S=scriptname.xsc

I did this once for the human swaps, moved the edited files to a specific folder, then did it again for the ghost swaps and moved them to a different folder to then re-compress and re-insert.

Where I'm currently at:


I have a lot more down-the-line items but I'll flesh those out once the basic idea is confirmed working. We're getting there!
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: fmlatghor on July 10, 2021, 11:51:49 PM
Nice to know that people are still writing hacks for gt. I have some information I have gathered about cpacs that might be useful.

The cpac format is used in other nds games by Capcom such as Apollo justice and Resident Evil: Deadly Silence for the same purpose as gt, to store graphical data. There are scripts to extract the data files contained within the cpac files...notably for gt there is a script (requires quickbms: https://aluigi.altervista.org/quickbms.htm (https://aluigi.altervista.org/quickbms.htm) ): http://aluigi.altervista.org/bms/ghost_trick.bms (http://aluigi.altervista.org/bms/ghost_trick.bms). You may also find related tools in https://github.com/Missingmew/phoenixtools (e.g. apollo-cpac). Well, once you extract the data files I'm not exactly sure what you do with them.

I assuming this is the "ancient thread": https://www.vg-resource.com/thread-16501.html (https://www.vg-resource.com/thread-16501.html) ? From what you're saying, I'd assume the "cpac_2d.bin" "cpac_3d.bin" names are just conventions used by Capcom -- because if you've ever read some interviews with Shu Takumi and Hironobu Takeshita(?), the 3d graphics were actually rendered in 2d....so it'd be odd if the ROM even contained 3d data. (See https://www.siliconera.com/ghost-trick-director-explains-why-he-made-the-lead-character-a-ghost/ (https://www.siliconera.com/ghost-trick-director-explains-why-he-made-the-lead-character-a-ghost/) )


And I cannot thank you enough for writing this. It really told me a lot about gt. I have been struggling to figure this game out as well.
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on August 02, 2021, 06:04:10 PM
Hey, apologies I meant to respond earlier to acknowledge this reply but I wanted to wait until I also had more progress to report. Unfortunately I haven't made that much progress so I've decided to just reply anyway and acknowledge what you've said!

(I'm attending grad school this fall so life has gotten busier and I've had less time to muck around in the Ghost Trick ROM. The project is not abandoned, I will keep coming back to it, but I haven't finished my playtest run-through which I want to finish before I try to figure out what's up with the couple exception screens.)

Thanks for linking those resources! I think I had previously tried using one of the cpac extraction tools, but at the time I was using it on cpac_2d.bin because I thought that's where the talk sprites were and also couldn't make heads or tails of the data. Now that I have a better understanding of the ROM and where everything lives in it, it might be worth giving that another go and seeing if I can make some more progress.

Again, really appreciate the resources and input, and apologies for taking so long to follow up! I wanted to have more to show for it but, alas, life happens. I'm not giving up though!  ;)
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: CrazyMLC on August 02, 2021, 08:55:30 PM
I've actually recently been looking at the ROM for Ghost Trick, toying with the idea of fixing some of the formatting and typo errors. Piggy-backed off of a GitHub project I found called ghost-tripper (https://github.com/CatTrinket/ghost-tripper), and have been fixing up and finishing the text decoder. Ended up coming up with the same idea of a NG+ starting off with your memories/identity intact.

Maybe we can compare notes?
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: fmlatghor on August 03, 2021, 02:41:55 PM
Quote from: CrazyMLC on August 02, 2021, 08:55:30 PM
I've actually recently been looking at the ROM for Ghost Trick, toying with the idea of fixing some of the formatting and typo errors. Piggy-backed off of a GitHub project I found called ghost-tripper (https://github.com/CatTrinket/ghost-tripper), and have been fixing up and finishing the text decoder. Ended up coming up with the same idea of a NG+ starting off with your memories/identity intact.

Maybe we can compare notes?

Please share anything you've found. I actually know of that project, too. It has multiple tools (some are ones written by others - nlzss), including one to extract the sprites -- though I have not given it a good look. And speaking of text encoding that'd be useful because there is someone I am working with who would like to translate the game to Dutch.

Quote from: ArcanaXIX on August 02, 2021, 06:04:10 PM
Hey, apologies I meant to respond earlier to acknowledge this reply but I wanted to wait until I also had more progress to report. Unfortunately I haven't made that much progress so I've decided to just reply anyway and acknowledge what you've said!
My motto is "it's never too late!".
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on August 04, 2021, 12:59:22 AM
Quote from: CrazyMLC on August 02, 2021, 08:55:30 PM
I've actually recently been looking at the ROM for Ghost Trick, toying with the idea of fixing some of the formatting and typo errors. Piggy-backed off of a GitHub project I found called ghost-tripper (https://github.com/CatTrinket/ghost-tripper), and have been fixing up and finishing the text decoder. Ended up coming up with the same idea of a NG+ starting off with your memories/identity intact.

Maybe we can compare notes?

Oooh yes! I've tried to document everything I've found in here but it's probably a bit stream-of-consciousness, I would definitely appreciate any resources or things you've found.

I've done a lot of digging around but mostly aimed at finding talk sprite pointers etc., but I've stumbled across plenty of adjacent knowledge on my journey. Is there anything in particular that would be helpful for you? I have a lot of probably incomprehensible .txt files on my computer but I could try to clean up a few and make them more legible. I uploaded two documents but they're pretty focused on sprites/images, but as they are written by a noob perhaps you might find more useful information in them than I realize: https://www.romhacking.net/?page=documents&game=2439

Let me know if there's anything in particular that might be useful for you. I have a lot of notes but most of it's just repetitive testing crap.

For my part, right now my biggest question is where the "let's try that again/rewind time" scene is defined in the game's code as well as where the sprite pointer for the entries in the "people" book are, and finally the "looks like this phone's dead" scene. Those are the only 3 scenes left where my sloppy sprite pointer re-write didn't successfully cat-ify Sissel (I'm pretty sure at least - still need to finish playtesting to be certain).
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: CrazyMLC on August 04, 2021, 06:02:39 PM
Most of my work has gone into understanding the contents of the "st##" folders, specifically the language files. For one thing, I think "st" stands for "stage" as the folders tend to be for certain areas. I've also identified a large number of the functions; I can print out many of the .en files without any hex codes present.
You can see my progress here (https://github.com/CrazyMLC/ghost-treader).

Ultimately my plan is to be able to convert any of these files into a human-readable script, edit that, and then be able to reencode them into a form the game can use.
That way it'd be feasible to write up new dialogue for a NG+ mode. (or write a dutch translation for example)

Quote from: ArcanaXIX on August 04, 2021, 12:59:22 AM
For my part, right now my biggest question is where the "let's try that again/rewind time" scene is defined in the game's code as well as where the sprite pointer for the entries in the "people" book are, and finally the "looks like this phone's dead" scene. Those are the only 3 scenes left where my sloppy sprite pointer re-write didn't successfully cat-ify Sissel (I'm pretty sure at least - still need to finish playtesting to be certain).
I'm not sure where some of the scenes you're mentioning are, but the st## files can be a bit frustrating. They aren't always laid out consecutively, and can have scenes strewn about in seemingly random files.

From what I'm seeing, the most intriguing possibility is that if there's any Sissel sprites that Yomiel doesn't use, those could be replaced to make new expressions for our feline friend.

edit: I think you can find the phone/rewind scenes in the system folder. Both system_0000.en.msg.xml.bin and system_0000.en.msg.xml.lz contain them. (game probably uses the .lz, but I don't know why there's redundancy for it)
The "people" book sprite is probably in the database_000_Expand files, (that's where the text for it is, and it has some scripting in there too) but it's hard to tell where exactly it is without more work decoding the scripting language.
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on August 13, 2021, 10:26:18 PM
Quote from: CrazyMLC on August 04, 2021, 06:02:39 PM
From what I'm seeing, the most intriguing possibility is that if there's any Sissel sprites that Yomiel doesn't use, those could be replaced to make new expressions for our feline friend.

I had this thought as well.... unfortunately, Yomiel uses the full range. There were a couple other sprites that I wanted to check and see if they were redundant (young Lynne has 2 crying sprites iirc and I think Amelie has a couple similar sprites? but I didn't check to see if they are truly the same or slightly different)-- otherwise would need to understand how to define a new pointer somehow, if that's even possible; OR potentially if there's one that Yomiel doesn't use /THAT/ much maybe replace those instances with another one before using that slot for a kitty expression.

Quote
edit: I think you can find the phone/rewind scenes in the system folder. Both system_0000.en.msg.xml.bin and system_0000.en.msg.xml.lz contain them. (game probably uses the .lz, but I don't know why there's redundancy for it)
The "people" book sprite is probably in the database_000_Expand files, (that's where the text for it is, and it has some scripting in there too) but it's hard to tell where exactly it is without more work decoding the scripting language.

Oh, fantastic, thank you for this tip! Going to take another look at these and see if I can find them.
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: fmlatghor on September 06, 2021, 04:25:58 PM
Hello! CrazyMLC and I have gathered in #gt on the GBATemp IRC network. Feel free to join us ArcanaXIX, or anyone else that wants to join!


irc.gbatemp.net
plain text - 6667
TLS/SSL - 6697

More info: https://wiki.gbatemp.net/wiki/IRC
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on November 07, 2021, 01:31:42 AM
HELLO I AM SORRY AGAIN FOR RADIO SILENCE - school has been kicking my ass, but I finally got the time to look into the leads given regarding the rewind time/phone/people scenes.

I finally got around to installing an IRC chat client, I can't guarantee I'll be on all the time but I'll try to check in from time to time, if that's still active! I know it's been several months.

I think I have figured out why this is giving me such a headache, but I have not solved the underlying problem. The good news is, I'm 99% confident that all the find-replace-ing I did above for all of the dialogue worked, and Sissel is uniquely a cat for pretty much the whole game, these particular scenes aside.

Here's what I learned about system_0000.en.msg.xml.bin and database_0000.Expand.en.xml.bin:

1. There is a reason that all of my testing before this yielded absolutely no fruit. For some reason known only to God and Capcom, the compressed (.lz) versions of these files, as far as I can tell, do fuck-all. Those are the ones I had been replacing before, but I have since learned that they can be ignored completely, and the .bin versions are what matters.

So if, for example, I replace the system_0000.en.msg.xml.bin file with the system_0000.fr.msg.xml.bin file, then the game runs exactly the same except the "rewind time" scene's dialogue is suddenly in French. You get the picture. Same with replacing the database file and finding that suddenly all of the "people" log book entries are in French.

This is actually fantastic news because it means if (BIG IF) I can figure out how to swap the sprite pointers here, I can still implement "cat mode" as a specific language that can be toggled on/off!

2. The sprite that accompanies the "rewind time" entry and the people log books is coded completely differently from all of the "m##_####" pointers that I found earlier, which was why I was having a hell of a time finding them. In fact, it is NOT - as far as I can tell - a singular hard-coded pointer.

I'm not explaining this well so I will illustrate with an example.

In st13_game021 (the endgame, final 4-minutes-to-death sequence), if you fail and are prompted to rewind time - in normal gameplay, Sissel is a ghost in this "rewind time" scene. Because, well, you know, it's endgame and that reveal has happened already.

Now, I have been doing a kind of jank way of hopping around to endgame scenes on a new save file. I've edited the st01_root file so that the first legible line in it after all the jargon code reads "st13_game021," instead of st01_game000, effectively telling it to jump straight to endgame on a new save file. This obviously screws some stuff up but it's a good way to get a decent lay of the land.

OF NOTE: when I did this, and then if I just waited out the timer to fail during that endgame sequence, prompting the "go back in time" scene? Sissel still has Yomiel's sprite.



(Ignore the weird text, That was me changing random values to confirm that it was pulling from the system_0000 EN file as I hoped it was... and also to help me recognize that it is pulling the SAME scene from different gameplay scenarios. Basically I was screwing around with the text to see if I could modify the sprite pointer [I couldn't] and whether it truly was the exact same scene being pulled in early game and endgame [it was].)

I *also* happened to open up the people book (which is in French because I didn't bother reverting that after experimenting with .bin vs .lz), and found that Sissel was still in Yomiel-form there too.



TL;DR of the above point is that when jumping to st13_game021 - literally endgame - from a new game, both Sissel's entry in the people book and the "rewind time" scene have him in Yomiel-form. (This is also in a modified ROM where I have swapped all dialogue pointers to the cat already, but as we've established that appears to be a separate thing.)

But, if you open the people book after teleporting directly to, say, st14_game050.... the scene during which Sissel transforms from using Yomiel's image to the ghost in normal play....

[after a ton of dialogue...]



Sissel's entry turns into a ghost. This, I am 99.9% sure, corresponds to the "new info added" event triggered by that scene.

This is all a very awkward way of phrasing the following key take-away:

The rewind time and people book scenes display a different sprite for Yomiel based on certain progression flags. I have not yet figured out anything about what those are or how they work, all I know is that if you don't have the flag specifically telling the game "transform Sissel into the ghost now," then it will use his human form.

My hypothesis is that the st##_game## XML files tell the game when to update, and then the book and rewind scenes display the appropriate sprite based on that flag. I still have absolutely not even the faintest idea how that's implemented in the code, or how to tell it to pull a different sprite (if that's even possible as-is...). It may be a higher-level modification than the individual dialogue indicators I was doing before, but I don't know that for certain.

Anyway, I spent basically my whole day today getting this far, and while I unfortunately don't have anything actionable to show for it, I'm happy to have at least figured out why this has been such a headache. I think the next step will be trying to figure out either:

a) How that "turn Sissel into the ghost now" flag is set, or

b) where in the hell is the data that tells it what sprite to display based on that flag.

I don't know when I'll have more time to dig into this, but I'm at least glad that I have a better understanding of why these were giving me such a hard time. And I'm super thankful for the external help that's guided me to where I can ultimately find some answers. I appreciate all of the input here so much.

EDIT: I somehow also completely missed your Github project, I absolutely want to take a deeper look at your work when I have the time. Unfortunately time is not something I have a lot of at the moment, but I want you to know I have not forgotten about this project or your contributions and thoughts!
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on May 25, 2022, 11:07:53 PM
Hello everybody!

I sincerely apologize for the long bout of nothing, but basically grad school = no time for ROMHacking. The good news is that summer vacation = LOTS of time for ROMHacking.

I've finally had the chance to sit down and troubleshoot this some more, and while I still have not found what I'm looking for, I am beginning to get a much better understanding of the game and I have a new theory to test.

WHAT I HAVE LEARNED ABOUT Database_0000_Expand.en.xml.bin :


I am completely certain of this, because I can delete literally all of the data in it except for 2 lines that make the game crash otherwise and while no text loads and transitions get wonky, the visuals - other than the character-specific animations - look fine.

This was a bit of a win actually because I had kind of wanted to replace Yomiel's corpse there with the cat too. So.. I did! This was accomplished by just brute forcing some hex code in the database_0000 EN file.



[It's animated too but could not be bothered to record a gif as proof atm.]

More details on that to come, but first, I want to talk about my remaining suspicions about the talk sprite here.

As I said - it is 100% NOT in the localization file. I also went ahead and over-wrote the .lz version with junk too just to confirm that doesn't do anything (it doesn't).

Thought maybe it's in the original JP file (just database_0000_Expand.xml.bin, not the .en. version). There's a bit of a reason I thought that, but rest assured, not the case. I overwrote that with junk too and it had no adverse effect. The menu pulls up crystal-clear and pretty, just has no text.

This is what that menu looks like with basically everything in the database_0000 files removed:



I do have a strong suspicion on where to look next, and if I'm right, it means I won't be able to easily implement "cat mode" as a "language" to be selected; it's gonna have to be permanent. Maybe. We'll see.

Because I think the code for that menu, including sprites, is part of the game code that is not language-specific.

Here is my general understanding of how this game works [very, very general] --

It has "directories" for each location (st01 = junkyard, etc), and each directory has a root file that has the general progression for that location. They point to individual "game###.xml" files with the code for each individual scene. For example, the st01 root file takes you first to st01/st01_game000.xml.

For each game###.xml file there is a huge one that I'm pretty sure is largely just the original JP code, and several smaller additional ones for each language. What's interesting about this is exactly what the game pulls from each. Most of the code and progression etc is obviously in the main .xml file, but it looks to the language-specific .xml files for the "m##_###" which are BOTH the text AND associated talk sprite. So if someone says something, somewhere in the game code it is saying to look to blahblah.[LANGUAGE].xml to pull the sprite and text information.

Which means that for 99% of the game I was just replacing the code for all of Sissel's sprites to point to the code for the cat IN THE ENG FILES ONLY, and I have a script for that so it went pretty quickly. Only annoying part was putting the recompressed files back in.

HOWEVER; when you pull open the menu, I think that action is probably coming from the scene's code in the big JP xml file somewhere and not the random smaller localized files. I think I need to delve into one of those files (will specifically look at st14_game050, because something in that file causes the game to use the ghost sprite for Sissel for the people entry and rewind time scene) and actually bother to try and understand this game's logic a little deeper.

This might not be the right conclusion to draw; but what is DEFINITELY true is that the main sprites for the people log are definitely not in database_0000 -- I think they are wherever the images for that menu UI are, because neither disappeared when I started screwing with that file.

What's interesting is that in most scenes the background animations are NOT localization specific (well, I mean, that makes sense); it's odd that they are for only the people log, though.

Current state of the hack:
* All in-game talk sprites representing Sissel are the cat (I accomplished this like a year ago and yet here we still are with no release :,,,) )

* Sissel's "Me" entry in the people book has the cat in the background and slightly modified text

* Both of these are language-specific changes as the game pulls these assets from the language-specific files

* Still have not figured out how to replace the talk sprites for "rewind time," "phone line broken," and "people" scenes. Have a hunch these may be pulled from the non-language-specific underlying code, and what I wrote way back in November implies it is influenced by progression flags. Need to better understand game code to comprehend what's going on here. Time to finally look into the main .xml files and code for certain scenes.

I want to share my findings on how I accomplished the animation swap but I think I need to call it a day for now. I may instead write and submit a document resource on here with the details instead of adding it to this thread, which probably doesn't need more spaghetti hex code. If I do create that document I'll mention it in a later update.

The short version is that animations appear to be preceded by "FD FF" as a header of sorts, and I think they're called as ".mtbl.xml" files. I replaced "FD FF F8 00" [Yomiel's corpse] with "FD FF 78 0E" [kitty Sissel animation, finding this was such a pain in my butt you have no idea] and then with the space left I specified placement and which frames from that animation would play. More specific details in a separate document. I PROMISE.

[Haven't looked more into the system files; my assumption is that whatever I can figure out for the database re: the "people" record will probably have a similar connection with the system files for the "phone"/"rewind" scenes.]

More updates very soon!!
Title: Re: [NDS] Ghost Trick "NG+ Mode" (Warning: Game Spoilers Discussed)
Post by: ArcanaXIX on August 08, 2022, 01:03:04 AM
HOLY MOLY YOU GUYS OKAY. I have good news, and I have bad news. Re: the only 3 scenes left where I haven't successfully changed the sprite for Sissel, which are the "phone line down" scene, the People log/database, and the "rewind time" scene.

GOOD NEWS: I'm fucking stupid.

So, literally last year, I had run my "find+replace sprite pointers" script on the system_0000.en.msg.xml.bin file (among others) just out of desperation/curiosity. However, when I found that it did not seem to impact any of the scenes I was interested in, I reverted it just in case I had accidentally changed something I didn't want to change.

What I did NOT realize, until today when I was testing something ELSE, is that desmume's save states apparently also save the relevant data in "system_0000.en.msg.xml.bin" and therefore changing that file in the ROM then loading up a save state did not actually reflect any changes to that file.

So uhhh once I realized that, I ran the script on the file again, and.......



ONE OF THESE 3 SCENES HAS BEEN TAKEN CARE OF! It turns out that the "phone line down" is in fact coded the same as the other sprite pointers and my script takes care of that.

UNFORTUNATELY; the database and rewind time scenes are still a no-go. I am really, really trying to figure out what is going on with those. I DID realize however that the text I changed in the database ("Me, the Mystery" to "Me, a Meowstery") actually needed to be changed 16 times because the game apparently pulls a unique title for each database entry even if the title text is exactly the same. So that's fixed now, but I am still pretty certain the database file only contains text+animations and no portrait sprites.

I am beginning to suspect the sprites used for the database and rewind time are somehow coded per chapter, because loading any chapter from a completed save game loads the database at a particular state. And on the main menu page, Yomiel is used for Sissel on all but the last two entries.

Honestly if it were just the database I would go ahead and just publish it already, but the "rewind time" scene is bothering me enough I really want to try to get that one taken care of before I do. It comes up a lot in gameplay so I think it would be really jarring to have Yomiel's face keep popping up there when everything else is the cat.

I have a suspicion that the key to one is the key to the other, but anyway, at least now that I've knocked out the phone line I am narrowing my sights on the final issues with this hack. I AM SO CLOSE.

(Oh, I also published a document with my experimenting/findings on changing the animations. I included instructions to replicate exactly what I did in the database for Sissel's sprite as well. Linky: https://www.romhacking.net/documents/887/ )

Rundown on current state of the hack/method so far:

- Have a script that replaces all of Sissel's sprite pointers for all of his Yomiel and Ghost portraits to the cat. For certain scenes these changes had to be reverted because in the late game Yomiel uses these sprite pointers and Sissel only uses the ghost portraits. From playtesting, everything looks good and accurate in most cases now.

- Also ran that script on the system_0000.en.msg.xml.bin file which has updated Sissel's sprite for the "phone line not working" scene.

- Changed Yomiel's corpse to an animation of Sissel the cat in the People database and updated Sissel's title description from "Me, a Mystery" to "Me, a Meowstery."

- All that remains, pending playtesting to confirm nothing else is broken, is to figure out how to update Sissel's portraits in the People database and the "rewind time" scenes.

- NICE TO HAVE: It would be cool to also replace the Yomiel sprite on the bottom screen of the database with Sissel the cat as well, I haven't found where that is pulled from either.


I won't say "more soon" because historically that's been a lie, but I am so happy to have FINALLY MADE SOME PROGRESS on one of these extremely and frustratingly elusive scenes. I AM GETTING THERE!!!!