News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Dumping/inserting byte values?  (Read 1004 times)

Psyklax

  • Hero Member
  • *****
  • Posts: 717
    • View Profile
    • Psyklax Translations
Dumping/inserting byte values?
« on: March 03, 2018, 05:27:27 pm »
I'm working on a game that doesn't use typical pointer tables for the most part, so there are two things I need: first, something that can dump the addresses of all instances of a given value to a text file (so you could search for 7F and get a text file with every instance of 7F). Second, something that takes a text file with a list of addresses and values, and replaces the values at those addresses with the values given.

Now I mention it, that sounds exactly like an IPS file... maybe I can make my own IPS files for that purpose, but it doesn't solve the problem of extracting values (in my case, I want a list of every control code that ends a string, so I can build a text file for the second thing I mentioned).

Does anything like this exist? All my searches have come up blank. :(

EDIT: I was right - with a combo of a text editor, a spreadsheet and a hex editor, I made an IPS file for inserting updated pointer tables. Can't believe I actually put these in manually... :o Still, now I need something that does the ever-so-simple task of spitting out a text file with addresses of particular values...
« Last Edit: March 03, 2018, 06:13:23 pm by Psyklax »

danke

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 2040
    • View Profile
Re: Dumping/inserting byte values?
« Reply #1 on: March 03, 2018, 06:41:07 pm »
Have you tried Cartographer and Atlas?

FAST6191

  • Hero Member
  • *****
  • Posts: 2368
    • View Profile
Re: Dumping/inserting byte values?
« Reply #2 on: March 03, 2018, 06:47:59 pm »
"dump the addresses of all instances of a given value to a text file"

A hex editor will hopefully do that.
In hex workshop I would get a window on the bottom right (but it could vary) where all the results are.
A right click there would start giving options for copying and selecting all.
I used to do this with some DS text formats that end of section markers (+1 then getting you the start of the next section).

Equally "binary grep" is a thing and might do what you want.

Psyklax

  • Hero Member
  • *****
  • Posts: 717
    • View Profile
    • Psyklax Translations
Re: Dumping/inserting byte values?
« Reply #3 on: March 03, 2018, 07:33:30 pm »
Hex Workshop is a paid product though, right? I prefer to stick with HxD as it's free, lightweight and does everything I need EXCEPT this. :)

Seriously, if I knew a tiny bit of high-level programming, I could write this program myself, but knowing how to do it in 6502 assembly isn't going to help me produce a text file. :laugh: And no, I've never tried Cartographer or Atlas, but I doubt that they will do this very simple task.

RedComet

  • Hero Member
  • *****
  • Posts: 3160
    • View Profile
    • Twilight Translations
Re: Dumping/inserting byte values?
« Reply #4 on: March 04, 2018, 12:53:33 am »
If you have Python available, you can just run this script that I threw together and it'll do everything you need:

https://pastebin.com/nTfD3Rry
Twilight Translations - More than just Dragonball Z. :P

Psyklax

  • Hero Member
  • *****
  • Posts: 717
    • View Profile
    • Psyklax Translations
Re: Dumping/inserting byte values?
« Reply #5 on: March 04, 2018, 05:33:16 am »
If you have Python available, you can just run this script that I threw together and it'll do everything you need:

https://pastebin.com/nTfD3Rry

I want to try that, but I've no idea how to use it. :D I have python.exe, so I made a batch file to run your .py file with it, but how do I add the input file and the value I'm looking for? It just exits immediately when I try it.

Actually, it gives a syntax error:

Code: [Select]
search.py, line 33
   print hex(address)
           ^
SyntaxError: invalid syntax

EDIT: Okay, a quick Google and I figured you were missing the parentheses on "print",  so now it runs. But when I include "filename.nes 7F" it says:
Code: [Select]
    value = int(sys.argv[2], 0)
ValueError: invalid literal for int() with base 0: '7F'
Hmm, think I'll keep messing around with it...

EDIT2: Oh crap, "int" means integer - a decimal integer! So I had to convert 7F into decimal (127) and suddenly it works! Thank you so much! :D I'll be using this in the future. Now for me to learn more about how to code in Python...
« Last Edit: March 04, 2018, 05:53:32 am by Psyklax »

FAST6191

  • Hero Member
  • *****
  • Posts: 2368
    • View Profile
Re: Dumping/inserting byte values?
« Reply #6 on: March 04, 2018, 06:42:53 am »
Yeah hex workshop is paid.

A few years back I went through all the hex editors I could find, still recommend having a copy of either hex workshop or 010 editor (the two best for general purposes which include some ROM hacking among it all, obviously all non ROM hacking editors suck at tables/custom encodings).

However I also figured you could probably get things done if you got a bunch of the free editors together.
HxD I dismissed from the list, despite it being then and now my "of course windows does not come with even a basic hex editor" choice when sitting in front of another's computer.
Still the other parts of the list were
https://sourceforge.net/projects/hexplorer/ (which I see is being updated again, going to have to go through it all again it seems).
http://www.romhacking.net/utilities/1051/
http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm (it looks basic but scripting it has is beyond some of the commercial stuff)

Between all those you can get the functionality associated with one of the paid ones. Can't recall which would have the functionality you want here though.

mziab

  • Sr. Member
  • ****
  • Posts: 251
    • View Profile
    • mteam
Re: Dumping/inserting byte values?
« Reply #7 on: March 04, 2018, 10:22:30 am »
Actually, it gives a syntax error:

Code: [Select]
search.py, line 33
   print hex(address)
           ^
SyntaxError: invalid syntax

This is actually one of the incompatible changes between Python 2.x and 3.x. A lot of people are used to the old syntax, myself included. For the record, Python 2.7 is still available and supported, so you can just install the old version if you're dealing with a lot of legacy scripts. But in this case there seems to be no need, if that's the only issue you had.

Quote
Code: [Select]
    value = int(sys.argv[2], 0)
ValueError: invalid literal for int() with base 0: '7F'
Hmm, think I'll keep messing around with it...

EDIT2: Oh crap, "int" means integer - a decimal integer! So I had to convert 7F into decimal (127) and suddenly it works! Thank you so much! :D I'll be using this in the future. Now for me to learn more about how to code in Python...

Actually, you don't even need to convert it to decimal. Since the base is set to zero, Python will try to auto-detect it. The trick is to prefix the hex value with 0x. Then it will properly decode it. Otherwise it will treat it as decimal.

Psyklax

  • Hero Member
  • *****
  • Posts: 717
    • View Profile
    • Psyklax Translations
Re: Dumping/inserting byte values?
« Reply #8 on: March 04, 2018, 02:33:54 pm »
The trick is to prefix the hex value with 0x. Then it will properly decode it. Otherwise it will treat it as decimal.

Ahh, I didn't think of adding 0x to it, I may try that in the future. But yeah, since I'm posting, I might as well explain what I did in case anyone else needs to know. :)

So in this game, each line of dialogue ends with a control code, so by using that lovely script to tell me where every one of those is, then using OpenOffice Calc to add 1 to each address, I can get a list of every line of dialogue, and more importantly, can do the same for my new script. The other thing is getting the pointer addresses, which is much harder in this game because most of them are hardcoded, so it's a process of elimination.

As for changing the pointers, here's what I did. The IPS file format is incredibly simple: the word PATCH in ASCII at the beginning, EOF (end of file) at the end, and in between you get three bytes for the address of a change, two bytes for how many bytes you change, and then the actual bytes. So I took my pointer addresses (making them six digits, of course), put "0001" next to them (cause I'm changing one byte at a time), then put each new value which I got from that script. I also use OpenOffice Calc to add to the pointer addresses, so that I can put the LSB and MSB separately. Finally I paste the whole thing into an .ips file in HxD and hey presto, an IPS I can use to patch my ROM, and all the pointers are changed. :) Sure, it'd be nicer with just a click, but it's a LOT easier than doing it manually (which I'd been doing up to that point). :)