News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Is there anything that cannot be edited via Hexadecimal?  (Read 9758 times)

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Is there anything that cannot be edited via Hexadecimal?
« on: April 17, 2016, 10:01:26 pm »
I am still a very novice hacker, so I don't quite understand how everything works. So if you load a ROM in a hexadecimal editor, is that the code for the entire game? In other words can any aspect of that game being modified by changing only those values? Specifically, there are some changes I would like to make to Final Fantasy III (NES) that I can't find in the ROM map. Here are just a few examples:
1. Changing the "Life" spell from being level 5 White Magic to being level 4 White Magic
2. Editing maps
3. Setting what jobs are obtained from what Crystals
4. Editing text boxes

Here is the ROM Map for reference: http://datacrystal.romhacking.net/wiki/Final_Fantasy_III:ROM_map

toruzz

  • Full Member
  • ***
  • Posts: 184
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #1 on: April 17, 2016, 10:16:31 pm »
Hexadecimal is just a way to represent the content of the ROM, so you can make any modification you want with an Hex editor. That being said, it obviously isn't always the best way to edit things: in example, you may want to use a Tile editor for graphics.

What you want to edit can be stored in several ways, so you need to know some ASM and use a debugger in order to find what values are being read and how is the game using them.

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #2 on: April 17, 2016, 10:24:31 pm »
That sounds interesting. Can you elaborate? It sounds like you are talking about the innerworkings of the game which I would like to understand. Also, there is unfortunately no map editor (or hardly any resources at all).

dougeff

  • Sr. Member
  • ****
  • Posts: 358
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #3 on: April 17, 2016, 10:32:48 pm »
Just looking at hex bytes is not a good approach. Even someone well versed in 6502 asm has a hard time telling 'code' from 'data'.

If you happen to know the exact location of the data you want to change, and the value you want to change to, then yes, it can be done in a hex editor. If you don't know what you're looking for, it's going to be like a needle in a haystack, or perhaps finding a specific needle in a pile of needles.

Alot of things are done by very careful investigation.

Quote
Changing the "Life" spell from being level 5 White Magic to being level 4 White Magic

I would do a trace of the code at the exact moment in the game when your white magic levels up, specifically looking for code that reads the magic code level value and does something with it. (Fceux debugging tools).

Maps is a bit more complicated. I would open fceuxs hex editor, view = rom. Be in a town. Save state. Start the code/data logger. Go from a town to the overworld map. Stop the logger. IIRC, it will highlight data a certain color. Edit some of the data, reload save state, go from town to overworld... Did it change? No? Undo changes, reload save state. Keep trying different bits of data until you locate the overworld map data location. Experiment, try to figure out how it's encoded. Etc.

nesdoug.com -- blog/tutorial on programming for the NES

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #4 on: April 17, 2016, 10:40:20 pm »
Woa, seriously? So the only way to find what values equal what and where they are located is through trial and error?? How is it even possible to make a key of every value if you don't know what each one represents?!

April 17, 2016, 10:45:47 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Also, I'm curious about how hex works in general. This might have to do with the physical hardware, but how does the NES know what each value means? Like how woud it know "A3" is the brown/black color palette for a sprite? How is that defined?
« Last Edit: April 17, 2016, 10:45:47 pm by linkncb16 »

dougeff

  • Sr. Member
  • ****
  • Posts: 358
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #5 on: April 17, 2016, 10:47:00 pm »
In my example, by turning on and off the code/data logger so quickly, you reduce the possibilities from thousands of random data sets, to maybe a few dozen.

Once you locate the map data, you should be able to look at the values and figure out which are valid to test, and should only have to test maybe 20 different values, not all 256. I'm speculating. It may be compressed, it may be some kind of 'bitmap'. Then you'd have to do more investigation. Decifer the decompression code, etc. Or, maybe you will get lucky and figure it out quickly. IDK.


April 17, 2016, 10:52:22 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
.
Quote
I'm curious about how hex works in general.

Magic.

Nintendo hired some wizards to cast a spell on your NES.

Here's some pictures of their handywork.
http://visual6502.org/images/6502/index.html
« Last Edit: April 17, 2016, 10:58:15 pm by dougeff »
nesdoug.com -- blog/tutorial on programming for the NES

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #6 on: April 17, 2016, 10:58:08 pm »
Can you explain what the ROM map guide means when it says a certain mechanic, "points to a table."?


April 17, 2016, 11:02:46 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Jesus what were in those pictures XD

dougeff

  • Sr. Member
  • ****
  • Posts: 358
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #7 on: April 17, 2016, 11:17:02 pm »
Quote
Jesus what were in those pictures XD

Magic.  ;)
- - - - - - - - - -

I couldn't find the exact pointer reference you mentioned. Here's another...
Quote
$58010 to $5894F - pointers to, among other things, shop inventory data

A pointer is an address/location of something. The actual address written on this list, will be relative to it's position when that bank is loaded to the NES (sorry if that's confusing, but MMC3 shifts things around).
 Let's say a shop has a value of 02. The game would look on the list of addesses for the #02 shop, then use that to find the inventory of the shop. You could hack the inventory of a shop, OR you could swap the pointer with another shop, and that would effectively swap their inventories. (#2 would be pointing to the inventory of shop #3, for example).
« Last Edit: April 17, 2016, 11:36:19 pm by dougeff »
nesdoug.com -- blog/tutorial on programming for the NES

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #8 on: April 17, 2016, 11:22:02 pm »
$72010 to $720BF - Base stat data for all classes, 8 bytes per class:
 0x0: Affects CP cost to change to this class
 0x1: Level needed to change to this job
 0x2-0x6: Base stats (Strength-Agility-Vitality-Intelligence-Spirit)
0x8: Base MP bonus. Index to a table.

That is what I was referring to. Is it the same as you previously described?

dougeff

  • Sr. Member
  • ****
  • Posts: 358
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #9 on: April 17, 2016, 11:29:01 pm »
Quote
Is it the same as you previously described?

Nope.

Somewhere is a big list of numbers, = Base MP bonus.

If the index for that class is 5, it will fetch the 5th number off the list. (It may be slightly more complicated).
nesdoug.com -- blog/tutorial on programming for the NES

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #10 on: April 17, 2016, 11:32:37 pm »
Ah, ok. So I guess where that table is located is unknown as I don't see it in the ROM map. Also, I'm reading your blog now actually to see what more I can learn! I had always wondered what code was used on the NES or if Nintendo has to create their own personal coding systems. There is so much I feel like I need to know about this.

dougeff

  • Sr. Member
  • ****
  • Posts: 358
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #11 on: April 17, 2016, 11:40:18 pm »
Quote
There is so much I feel like I need to know about this.

I know the feeling. Maybe you should start with something easier? I got my start by changing the maze of Pac-man (and graphics).
nesdoug.com -- blog/tutorial on programming for the NES

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #12 on: April 18, 2016, 11:09:04 am »
Just looking at hex bytes is not a good approach. Even someone well versed in 6502 asm has a hard time telling 'code' from 'data'.

If you happen to know the exact location of the data you want to change, and the value you want to change to, then yes, it can be done in a hex editor. If you don't know what you're looking for, it's going to be like a needle in a haystack, or perhaps finding a specific needle in a pile of needles.

Alot of things are done by very careful investigation.

I would do a trace of the code at the exact moment in the game when your white magic levels up, specifically looking for code that reads the magic code level value and does something with it. (Fceux debugging tools).
Magic does not level up, rather each spell is assigned to be a specific level (1-8) and when you buy a spell any character who can use that level of magic will be able to use the spell.
Ex. I buy Cure3, a level 5 spell. I give it to a character that can only use up to level 4 magic so even though it is in their spell inventory, they cannot use it. How would I find what value assigns that spell to be level 5 magic even if I were monitoring the game? (Because there is no way to change the level of a spell in game.)

Bregalad

  • Hero Member
  • *****
  • Posts: 2751
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #13 on: April 18, 2016, 11:41:11 am »
Quote
Is there anything that cannot be edited via Hexadecimal?
Technically, no, but there are many things which cannot practically be edited with only a hexadecimal editor. Graphics is the #1 thing that comes to mind.

Quote
Specifically, there are some changes I would like to make to Final Fantasy III (NES) that I can't find in the ROM map.
Unfortunately, FF3j is a difficult game to hack for the very reason you mentioned. This ROM is packed, especially the english translation, there is something like 4 bytes free in the entire ROM if I remember well (don't take my word on that). Since the game is already 512kb MMC3, it is impossible to expand the ROM, or you could, but end up with a largely unsupported MMC3 oversize ROM (the alternate English translation did exactly that).

Don't get me wrong I love FF3j and I'd like to see hacks of it. Good luck !

STARWIN

  • Sr. Member
  • ****
  • Posts: 454
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #14 on: April 18, 2016, 12:02:05 pm »
Also, I'm curious about how hex works in general. This might have to do with the physical hardware, but how does the NES know what each value means? Like how woud it know "A3" is the brown/black color palette for a sprite? How is that defined?

The people who made the game had a certain amount of ROM where they could put the game, the system had a certain amount of RAM for temporary variables for the program running and the system could emit graphics and sound. They worked systematically so that a number in a given location had a given meaning. As long as they sticked to that, the number was only used for that thing and consequently, if you edit that number, it changes that specific thing.

The whole system is a big bunch of numbers and the CPU mostly just moves them from one place to another and sometimes does simple math on them. The CPU does it by having a list of numbers defined as actions (by hardware, moving numbers and simple math, that is). When the system starts the CPU reads the first number in a known place and does the action the number represents. After each action is done it reads the next number, and this continues.

The real magic is that the system has certain places, where when you put a number (or enough numbers) in a certain way, audio or graphics come out. So it is a combination of code being systematic and these input/output locations in memory. Input for reading the controller, of course.

Sometimes you can figure out what numbers in the ROM mean because they are stored in a nice order. Like, if you had a list of prices in some shop, and they happened to be stored in a way you can guess (a list=table in the same order as they appear in the game and each price taking 2 bytes, for example), then you could search for a hex string with a hex editor with a bunch of the prices glued together, and it would then find that exact spot (and it is unlikely any other number sequence would be exactly the same, so this usually works easily if it works).

With computers and hex values longer than 1 byte, you have to always pay a bit of attention to whether the values are stored little-endian (least valuable byte first) or big-endian. The numbers people use in everyday life are neither, because they don't consist of bytes, but big-endian looks very similar otherwise, just with the byte gaps. I have no idea how a given NES game would store 2 byte prices, so I'd try searching for both orders. NES only supports 1 byte math, and anything more complex is written in code in a multiple command sequence, which could eat the numbers any way the coder thought of.

However for many things a debugger emulator like FCEUX is more reliable than guessing, although there you must pay some of your time to get the more reliable answers. It basically lets you read the code, but you need to be able to understand what the code means and you need an "attack plan" on how to find the code (or data, but they are closely related as data is given a meaning only by the code). Breakpoints are the most common tool in most attack plans. You can check the debugger in FCEUX even if you don't expect to be able to use it, because if you get the basic concepts, you pretty much start to understand everything.

One of the very basic problems is that there are a lot of numbers (several magnitudes more than most people would find comfortable) and it is slow to read even small amounts of them, be it reading code or looking at data. But hey, it takes a lot of time to complete the game itself too..

I don't know the perfect attack plans for your problem examples (and I don't know if Maeson or Kea would already know enough to solve them quickly), but I'll try to list something that comes to mind:

1. Changing the "Life" spell from being level 5 White Magic to being level 4 White Magic
 - when you equip LV5 and LV4 magic they end up to their respective lines. this is a difference that touches something much related to the problem (code)
 - when you have them equipped vs not equipped, that difference shows in RAM too (location for a breakpoint)
2. Editing maps
 - encounter background is different on different terrain
 - ships cannot travel over land
 - airship cannot land on mountains
 - no encounters in safe areas
3. Setting what jobs are obtained from what Crystals
 - somewhere during when you gain the jobs, code touches this
 - job selection menu touches jobs
4. Editing text boxes
 - text or boxes? well it is a bit boring for me to think about

@Bregalad

I found, like, 43 sequential unused bytes when I did my little hack. There's plenty of space!! (I hope it is unused)

It isn't so much of a problem for modders though, they usually just change existing bytes. Only adding more text, if that text boxes means that, would apply in the first post here.

VicVergil

  • Hero Member
  • *****
  • Posts: 723
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #15 on: April 18, 2016, 12:03:08 pm »
- Text: Each letter is encoded with a byte, sometimes less sometimes more.
- 2D Graphics: Tiled graphics are encoded in 3 different parts:
palette (a color list)
tile data (the drawings) encoded in different formats depending on the system and number of colors - for example the 1BPP format is a monochrome (black or white) image with 1 bit per each pixel in the image. So a 8x8 tile would be stored on 8 bytes.
tilemap: a reference with the ID for each individual tile from the above, to assemble them like tiny puzzle pieces in a big drawing
There's also bitmap graphics, and then there's 3D models too which are just lists of points and relations between them.
- Sound: whether MIDI (a sequence of bytes picking instruments from a list) or streamed sound (wave sound represented with bytes) and timing info (also in hex).
- Stats: also in hex
- Programming: using all of the above, it's itself written in machine language code in hex and can be "translated" with a disassembler.

Everything can be edited in hex, even compressed data.
However sane people automate the process when it's too tedious and create tools to make said editing easier (like tile editors and disassemblers, among other things)

FAST6191

  • Hero Member
  • *****
  • Posts: 3104
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #16 on: April 18, 2016, 03:35:02 pm »
Technically, no, but there are many things which cannot practically be edited with only a hexadecimal editor. Graphics is the #1 thing that comes to mind.
I would never use one for proper proper editing/making custom sprites but if I am editing a tile to blank something out then many times it has been a hex editor I used to do the deed -- if you know the six pixels are in this location and the colour/alpha is this value then it is almost why not use a hex editor.
Amusingly I did also once see a guide use a tile editor as a hex editor using colours (I think it was for a Golden Sun audio hack, certainly a GBA sappy audio hack). Speaking of colours though I did once have a hex editor colour certain bytes a certain way.

For the truly nightmarish thing to edit with a hex editor though then I am going with LZ compression or anything that references elsewhere the file rather than a lookup table or just the immediate thing it is concerned with in the case of RLE. Editing something there and thinking you might be breaking a reference to something later I do not want to try to comprehend. Maybe if it is was one or two sections and I could test the rest of the window or just run it in an emulator and make sure it works I could see doing it, the graphics thing I mentioned above is just a normal thing as far as I am concerned but not the compression.

Also, I'm curious about how hex works in general. This might have to do with the physical hardware, but how does the NES know what each value means? Like how woud it know "A3" is the brown/black color palette for a sprite? How is that defined?

As others mentioned it is part of the way the program just leaps from place to place hoping the compiler/assembler/programmer was on the ball that day. What you might want to look up though is return oriented programming. It is all about using these values that were never technically intended to be interpreted as instructions in that way being used as instructions. Typically this is to hack things that have protection against arbitrary code being run (so pointless on anything without it as there are easier hacking methods) but people have made quite complex programs using it and there is nothing stopping you from doing something like it on the NES if you wanted.

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #17 on: April 18, 2016, 06:46:12 pm »
Okay, so I'm going to take the suggestion to monitor the game's RAM to pick out any changes I see. If I am using Nestopia how would I go about this step by step?

Disch

  • Hero Member
  • *****
  • Posts: 2814
  • NES Junkie
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #18 on: April 18, 2016, 07:25:27 pm »
Okay, so I'm going to take the suggestion to monitor the game's RAM to pick out any changes I see. If I am using Nestopia how would I go about this step by step?


NEStopia doesn't have a memory viewer... or really any debugging features.

You'll want a debugging emu.  The only game in town for NES debuggers is pretty much FCEUX.

Using FCEUX:
1)  Load your ROM
2)  Go in the Debug menu, select "Hex editor"

You'll see something like this:



Each byte has a unique address.  The column on the far left with the colons tells you the address of that row.  That is, the first row is addresses $0000 through $000F.  Next row is $0010 through $001F, etc.

The numbers in the middle are [roughly] the value contained at that address.  Example, the value at address $0021 is $0E

Different addresses imply different things.  You'll be mostly interested in these addresses:

$0000-07FF = RAM
$6000-7FFF = SRAM
$8000-FFFF = ROM

RAM and SRAM are workspace.  That is where the game keeps track of variables (how much life does this enemy have, where is everything positioned, etc), as well as the game state.

ROM doesn't change, it's the "fixed" data that is contained in the ROM file.  However you may see large sections of the ROM area change all at once when the game bankswaps.

Spindaboy

  • Full Member
  • ***
  • Posts: 161
  • ただいま!
    • View Profile
    • SyndROMe Hacking
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #19 on: April 19, 2016, 11:08:23 am »
Any difference between RAM and SRAM? Also, when the game is saved, is that just a massive overwrite of ROM values? Lastly, what is bank swapping?

April 19, 2016, 11:10:17 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Oh, and what are those weird symbols on the right of the hex editor?