News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Editing text in Shadowrun (SNES)  (Read 6412 times)

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Editing text in Shadowrun (SNES)
« on: January 18, 2021, 10:54:00 am »
I'm using the BSnes debugger to try and understand how text works in Shadowrun(USA) for the SNES. I thought I'd find text strings in the ROM like in other games, but here it looks like the words and sentences are pieced together on the spot which is really confusing. Example:



In the debugger:



I downloaded the text dumper for Shadowrun SNES available on this site, but all it tells is this:

Quote
[0x0F1136][Dances with Clams does not work for money. He merely requires a donation of 2,000 nuyen to the Tribal Balldance Fund. I sense great need. Do you wish to make a donation?]

Is 0x0F1136 an offset in the ROM, or something else? Because I looked at it in the hex editor, and it doesn't seem like anything to me.

I'm at a loss here. Help?

Jorpho

  • Hero Member
  • *****
  • Posts: 4883
  • The cat screams with the voice of a man.
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #1 on: January 18, 2021, 12:57:29 pm »
I thought I'd find text strings in the ROM like in other games
I don't know what made you think that since it says right on the text dumper page:
Quote
The script is compressed using the Huffman algorithm, using 16-bits codeworks to store 1-2 characters (each character used has the upper bit active), and 0×0A is the end of line character.

Quote
Is 0x0F1136 an offset in the ROM, or something else?
Something else, I guess? The shadowtxt.c source code calls it the "codework", which suggests it is related to Huffman codewords.
https://en.wikipedia.org/wiki/Huffman_coding
This signature is an illusion and is a trap devised by Satan. Go ahead dauntlessly! Make rapid progres!

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #2 on: January 19, 2021, 09:04:45 pm »
compressed

I see, that's why I didn't recognize the string in the ROM.



^Turns out that 0x0F1136 is the offset for that string after all. I changed a few bytes in it and it produced changes in-game. Of course, the resulting text was just gibberish. The logic of Huffman compression seems simple, but if I don't have the "tree" with the codewords for that string how can I effect the changes I desire? All I want is to change the "2,000" into "1,500", like so:



I edited the uncompressed characters in the RAM to display the above image in-game. That was easy. How do I edit the compressed text in the ROM to display that?
« Last Edit: January 19, 2021, 09:37:35 pm by the_E_y_Es »

phonymike

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #3 on: January 20, 2021, 12:44:45 am »
The way I would do it, is dump the entire script to plain text, and then change the decompression routine to simply copy the uncompressed text to RAM. If the script dumper could create a file compatible with atlas, you could reinsert the uncompressed script into space at the end of the ROM (expand it) and atlas would update the pointers.

update: There seems to be lots of effort into editing the script to this game but no actual hacks to show for it. This post by RetroHelix links to a zip file with tons of info.

I looked into the code a little bit but couldn't find a pointer table. If you update the script, but your text is longer than what the game already contains, then it will throw off all the text after that. So you recalculate the pointer table. Unless this game somehow works differently than that.

Also on datacrystal there's a ton of information, as well as a python script compressor.
« Last Edit: January 20, 2021, 05:22:21 pm by phonymike »

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #4 on: January 20, 2021, 10:11:30 pm »
Hello there, I looked into the datacrystal and found this in the "ROM Map" section:

Quote
E8000 to  E8577 = Huffman-coding tree

That should contain the codewords I assume. Don't know exactly how to go about deciphering them but I will look into it tomorrow.

update: There seems to be lots of effort into editing the script to this game but no actual hacks to show for it. This post by RetroHelix links to a zip file with tons of info.
Ah yes, I'm familiar with those files and the persons who provided them (The Admiral and DragoonZero from gamefaqs). There's a lot of useful info in those.

I looked into the code a little bit but couldn't find a pointer table. If you update the script, but your text is longer than what the game already contains, then it will throw off all the text after that. So you recalculate the pointer table. Unless this game somehow works differently than that.

I found a Brazilian-Portuguese translation and the author does mention pointers for the text. What he did was expand the rom and build his own text separately because he could not adapt the original to the translated one.

Also on datacrystal there's a ton of information, as well as a python script compressor.
I downloaded that compressor but could not run it (using the command prompt). I must be doing something wrong. I wonder if it can compress text data? That would be great...

Jorpho

  • Hero Member
  • *****
  • Posts: 4883
  • The cat screams with the voice of a man.
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #5 on: January 20, 2021, 10:53:11 pm »
I downloaded that compressor but could not run it (using the command prompt). I must be doing something wrong.
Well, y'know, if you want someone to suggest what you're doing wrong, you're going to have to give some more details about what you did and what happened.
This signature is an illusion and is a trap devised by Satan. Go ahead dauntlessly! Make rapid progres!

phonymike

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #6 on: January 22, 2021, 11:12:38 pm »
I did a little searching around in the ROM, and it seems at $80:947A (0x147A unheadered ROM file) there's a pointer table to the conversations. The pointer table is 0x116 bytes in length, and 2 bytes per pointer. You need to take the pointer, and add bank $8D to get the snes memory address. For example, the third pointer 0x6F 0x90 gets byteswapped, and add 0x8D resulting in snes mem pointer $8D:906F (0x6906F unheadered ROM file).

The conversations are shown in conversedump.txt. The conversations have a structure you can see in shadowconverse.c, with a pointer to the character photo, and pointers to possible dialog strings. At least I think this is how it works.

So there's a pointer table for the conversations, and each conversation contains pointers to compressed text strings.

Keep in mind these script dump files may show ROM file locations with an extra 0x200 byte header.
Code: [Select]
66      42     E8966    Tread carefully whelp, you have much to overcome.
This is location 0xE8766 (snes mem $9D:8766) using an unheadered ROM.

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #7 on: January 25, 2021, 05:09:33 am »
So there's a pointer table for the conversations, and each conversation contains pointers to compressed text strings.
I looked more closely at the Datacrystal ROM map. Could this be it?

Quote
5980 to   5E47 = Text pointers (Lines 1000-1605, mostly)

And looking at the script dump:

Quote
1342     53E     F1336    Dances with Clams does not work for money. He merely requires a donation of 2,000 nuyen to the Tribal Balldance Fund. I sense great need. Do you wish to make a donation?

I think the F1136 offset is for an unheadered rom because when I change any bytes in that location it effects that exact dialogue line.

So if this line is number 1342, its pointer should in that range (5980 to 5E47), correct? I'll look both at 5980 and 5780 just in case and see if I can find the pointer for that specific line.

I appreciate the detailed explanation, thanks man. It's starting to make sense to me now.

phonymike

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #8 on: January 27, 2021, 11:22:56 pm »
I modified one of the programs to dump these lines, I count 612 of them from the pointer table.

Download it in my next post.

Code: [Select]
[348] 0xF1136
Dances with Clams does not work for money. He merely requires a donation of 2,000 nuyen to the Tribal Balldance Fund. I sense great need. Do you wish to make a donation?

I show it as line 348 (decimal) in the pointer table. Since each pointer is 2 bytes, multiply 348 x 2, and add it to the start of the pointer table 0x5980 and you get pointer 0x5C38. The values at that location are 0x36 0x91. Byteswap them and add 0xE8000 to get the rom location 0xF1336.



The script dump tells us the location 0xF1336 and that's really helpful to know, but you're only allowed to change that line as long as it's shorter than the original text. The pointer table gives us a lot more control. You could make this line longer, and the next line shorter, and just change the pointer to the second line to be farther than it was originally.

The only problem with this pointer table is that it only allows text to be located within a certain range. This might be a different routine than the conversation routine. Maybe these lines could be inserted uncompressed and the routine changed to point to expanded rom space.
« Last Edit: January 29, 2021, 11:23:29 pm by phonymike »

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #9 on: January 29, 2021, 08:08:41 am »
Your method makes perfect sense, I can find the pointers to any of those lines now. As you said, changing the pointer of the next line to a further address leaves more room for the line I want to edit. The problem is I'd have to rework the next line in a way that is functional and still makes sense in-game. So that's another can of worms...

In theory, if I can change the "2,000" to "1,500" while still keeping within the confines of the original pointers (equal or shorter than the original text, like you said), it would be the perfect solution. In that case, though, I'd still have to figure out what bytes of compressed text become "1,500" when uncompressed. To that end, I tried looking into other compressed text strings that contain "1,500" when uncompressed in-game. Example:

Code: [Select]
[363] 0xF142B
The power be with you for 1,500 nuyen!

I tried copying some of the bytes from that string, in hopes of getting that 1,500. None of them worked. It's like each string of compressed text uses its own codewords, I'm not sure...

Your other suggestion sounds more feasible, maybe I should expand the rom and write my own uncompressed text separately. If I were to change the routine that points to that original line (making it point to my custom line in the expanded space), would the other lines in the game still work normally? IOW, can I change the routine only in regards to that specific line?

Thanks so much for your help!

phonymike

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #10 on: January 29, 2021, 11:20:09 pm »
I made a lot of progress as far as these "lines" are concerned. The pointer table at 0x5980 points to 612 different lines of text. The "conversations" are different because they are more like chunks of pointers, that vary in size. So the conversations will be a little more difficult, but I hope atlas can deal with it.

I was able to decompress the "lines", and use atlas to insert them into the ROM at 0x101000, and update the pointer table at 0x5980. I also wrote some (messy) asm to copy the uncompressed strings into the memory location for text (0x7E21A0). The only problem is the game doesn't show the text. There looks to be a little more to the text decompression routine than just placing the text at 0x7E21A0, maybe as each letter is decompressed it updates some pointer for something.

I included everything except xkas.exe (v0.06), atlas.exe (1.11), and the Shadow Run ROM. Put those in the main folder, and run make.bat. To extract the script (it's already extracted), in the script_dump folder there's a dump_lines.bat. It puts the output in the previous main folder.

The cool thing is the code for the conversations and the lines come from different areas, so I'm able to filter the "lines" by setting the carry flag. The conversation text does not have the carry flag, so it is decompressed and displayed as normal. So my code can tell the difference. I haven't tested it much at all though, but it at least works for the first few text dialogs.



Hopefully I can get it working in the next several days. If I make more progress, I'll make a projects thread so more people can find this work and hopefully expand to a full Shadow Run script project. I've only seen the text for "lines" and "conversations". I don't see script triggers like giving the player 2,000. The script could say anything, but the game will still only give 2,000.

Download

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #11 on: February 02, 2021, 05:31:22 am »
This is great!  I gathered the files in one folder like you said, and ran make.bat; but I forgot to expand the rom first, so that atlas can insert the lines at the new address 0x101000. I'm going to download a program to expand it, then try again.

I can't wait to edit some of the text in this game, this is going to be awesome.

I don't see script triggers like giving the player 2,000. The script could say anything, but the game will still only give 2,000.
Don't worry, I got that covered! In this case, the actual amount that is subtracted (2,000 nuyen) is in a different place in the ROM, at offset DD9BE, unheadered. There I can change it to 1,500. :)

phonymike

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #12 on: February 02, 2021, 11:02:30 pm »
The make.bat copies the original rom to "sr.sfc", so the original is unmodified. Then xkas does this:

Code: [Select]
org $BFFFFF ;pad to 16 mbit
db $00

which expands the sr.sfc by putting a 0x00 right at the 16 mbit position, file address 0x1FFFFF. No need to expand  :thumbsup:

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #13 on: February 03, 2021, 08:08:35 pm »
Oh right, I got the sr.sfc file and it's 2MB. Looks like you were able to run it on bsnes; it just hangs on my end. It's weird because bsnes is supposed to support expanded roms.

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #14 on: June 21, 2021, 10:21:09 am »
I've found a Portuguese translation of the game; it might just be the only translation of Shadowrun (SNES).
 
https://github.com/DougRpg/shadowrun

https://joao13traducoes.blogspot.com/2018/06/snes-shadowrun-dougrpg.html

The author says that instead of modifying the original text pointers he decided to relocate the text to an expanded ROM space (the patch expands it to 6MB). I'm assuming the text is also uncompressed in this expanded space. That means it would be very easy to modify.

If I could run the patch on bsnes I could also view the memory in the debugger, so it would be possible to find the text strings there, then in the ROM itself, and finally modify them to my needs. But I haven't been able to run it so far.

The author says that to run the patch on bsnes it is necessary to change the emulator's "manifest" to make it support 6MB ROMs. How can I do that?

DougRPG

  • Full Member
  • ***
  • Posts: 151
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #15 on: June 23, 2021, 05:37:27 pm »
Hi, I was able to translate this game and you need to be a little more creative to accomplish that.
Here in rhdn you can find a code to extract all the game's dialogs. So the dialogs are together in a single block, and with that code you can extract and find the starting address of every dialog. Right?
Ok, nothing new here.
The problem is that there isn't a single pointer table in this game. You can find some small tables for some related dialogs, but for a lot of dialogs the pointers are hardcoded and spread all around the code. So changing every single pointer would be a nightmare for a long time.
So my solution? Simple, every dialog is compressed with Huffman, right? So the game has a Huffman routine that receives the dialog pointer to extract, right? Remeber this fact. Now I put all my translated dialog in a single block in an expanded area. I know the pointer for each of these new dialogs because I created this new block. So after that I went in all the starting addresses for the original dialogs (I have them when I run the extractor code, remember?) and put there the 3 byte address for the new translated dialog (now in the expanded are). So I overwrote the first 3 bytes of each compressed dialog with the new address (the original dialog is now corrupted). Finally I changed the Huffman routine to, at the beginning, get the first 3 bytes of the dialog, that now is the pointer to the new translated dialog, and get the data from there. So I added a little indirection. And I removed the Huffman and put a new algorithm of my own.
This way I didn't change a single pointer of the game, only added an indirection.

The only problem related to this solution was for the dialogs that had less than 3 bytes of compressed data, because this way I couldn't add the indirection pointer. Fortunately there were less than 5 of such dialogs, and for those I had to do the indirection manually.

But the text was the easiest part. The real problem was the dialog windows, where they are all hardcoded and totally unrelated from the dialogs.
« Last Edit: June 23, 2021, 06:38:46 pm by DougRPG »

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #16 on: June 23, 2021, 07:01:01 pm »
Hi, I was able to translate this game and you need to be a little more creative to accomplish that.
Here in rhdn you can find a code to extract all the game's dialogs. So the dialogs are together in a single block, and with that code you can extract and find the starting address of every dialog. Right?
Ok, nothing new here.
The problem is that there isn't a single pointer table in this game. You can find some small tables for some related dialogs, but for a lot of dialogs the pointers are hardcoded and spread all around the code. So changing every single pointer would be a nightmare for a long time.
So my solution? Simple, every dialog is compressed with Huffman, right? So the game has a Huffman routine that receives the dialog pointer to extract, right? Remeber this fact. Now I put all my translated dialog in a single block in an expanded area. I know the pointer for each of these new dialogs because I created this new block. So after that I went in all the starting addresses for the original dialogs (I have them when I run the extractor code, remember?) and put there the 3 byte address for the new translated dialog (now in the expanded are). So I overwrote the first 3 bytes of each compressed dialog with the new address (the original dialog is now corrupted). Finally I changed the Huffman routine to, at the beginning, get the first 3 bytes of the dialog, that now is the pointer to the new translated dialog, and get the data from there. So I added a little indirection. And I removed the Huffman and put a new algorithm of my own.
This way I didn't change a single pointer of the game, only added an indirection.

The only problem related to this solution was for the dialogs that had less than 3 bytes of compressed data, because this way I couldn't add the indirection pointer. Fortunately there were less than 5 of such dialogs, and for those I had to do the indirection manually.

But the text was the easiest part. The real problem was the dialog windows, where they are all hardcoded and totally unrelated from the dialogs.

Hi there! Thanks for chiming in.

I see, so the original pointer leads to the starting address of the original dialogue, and there the game reads the first 3 bytes in that sequence; these 3 bytes are a pointer to the new translated dialogue in the expanded rom space. That's pretty clever! So in fact you are still using all the original pointers, like so: old pointers -> new pointers -> new dialogue. Nice.

Maybe I should copy your method of replacing the first 3 bytes of every dialogue with a pointer to expanded space. That still leaves the compression issue, though. Huffman hates me. I've spent many hours trying to change a simple "2" into a "1", to no avail. I could have all the expanded space in the world, and I still wouldn't be able to make my changes.

The absence of a consolidated pointer table, the hardcoded pointers, the dialogue windows. The compression. It's nauseating.

I'm not sure what path I should take to accomplish this. Didn't think changing a couple of characters would be such a complex task.

Apologies for the rant. Your explanation does clear up a lot of things, and I appreciate it. :)
« Last Edit: June 23, 2021, 07:19:34 pm by the_E_y_Es »

DougRPG

  • Full Member
  • ***
  • Posts: 151
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #17 on: June 23, 2021, 09:07:49 pm »
Hi, you can't change some compressed chars changing some bytes like you described some posts above. Huffman works with bits, not bytes, so you have codewords of different bit sizes and they start and finish anywhere inside a byte.

If you want to change the price of only a few dialogs you can do it "manually". You put these dialogs somewhere in the Rom, uncompressed. At the beginning of the Huffman routine you jump to a new place where you'll put your code. You compare the current dialog pointer with the pointers you want to monitor. If not them, jump back and do the normal flow. If the current pointer is one of the desired dialogs, just copy the uncompressed dialog to the address the Huffman routine copies them and bypass the Huffman decompression.
This is a very good exercise to learn some romhacking and I think you can do it.
Any question just ask.

the_E_y_Es

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #18 on: June 24, 2021, 06:41:44 pm »
Right, so I can just create exceptions in Huffman routine for the pointers that interest me. I've never written code in SNES assembly but if it's anything like MIPS I think I can manage.

I'm starting with that line on my first post, "Dances with Clams does not work for money...". The pointer to that line is located at offset 5C38 in the ROM. The RAM address for that same pointer is at 0x80DC38. I'm using an unheadered USA rom.

I placed a read-breakpoint on 0x80DC38 and it fired off as expected when hiring the shadowrunner. Now I will look at the disassembly and try to determine where the Huffman routine begins. I also have to find some free space in the ROM where I can place my code. And of course, I have to learn what instructions I will need to perform the comparisons. So there's a lot to do.

I will post my progress here. I appreciate your continued support!

DougRPG

  • Full Member
  • ***
  • Posts: 151
    • View Profile
Re: Editing text in Shadowrun (SNES)
« Reply #19 on: June 24, 2021, 09:28:58 pm »
A little help: I read my code here and the Huffman routine starts at $81aec0, but the routine is called from $80fe57. The dialog is located at address $cf.
So you'll need to do something like this:

Code: [Select]
seek(0x80fe57)
  jml (empty_region_address)
 
seek(empty_region_address) 
  rep #$20
  lda $cf
  cmp #$1234 //Change to the address you want to monitor
  beq handle_this_dialog
  cmp #$5678 //another dialog. Change to the address you want to monitor
  beq handle_this_another_dialog
  lda [$cf] //optional. If you want to do your own stuff, read the data byte and proceed with you own algorithm and forget Huffman.
  jml 0x81aec0 //optional. Otherwise jump to $81aec0 to continue normally with the huffman routine.


handle_this_dialog:
  //here you will copy the new string to $21a0. Why $21a0? Homework. Do some debugging on the Huffman routine.