News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Is there anything that cannot be edited via Hexadecimal?  (Read 7038 times)

STARWIN

  • Sr. Member
  • ****
  • Posts: 449
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #20 on: April 19, 2016, 11:17:35 am »
SRAM is the save, but it is also used as ordinary RAM. ROM doesn't change (read only memory).

the CPU cannot see the entire ROM at once, it only sees two average-size slices of it. one of the slices is always the same, the other one can be changed by code when the game runs. the game is also executed at either of these slices! so it typically goes to the static slice and pulls a lever and magically the other slice shows a different part of the whole ROM file.

those symbols show how the hex numbers would look like in a text file. it is useless for us.

dougeff

  • Sr. Member
  • ****
  • Posts: 359
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #21 on: April 19, 2016, 11:32:04 am »
The NES itself has $800 bytes of RAM. Every game can use this. Some cartridges have extra RAM in them, $2000 bytes of it (mapped to addresses $6000-7fff). If you stick a battery next to that, this on-cart RAM will keep its values = saves the game, called SRAM.

The ROM is always mapped between $8000-ffff...but if a game has more ROM than that, there are switches inside the cart which can remap $8000-ffff to a different chunk of ROM. Later mappers could remap smaller and smaller chunks...swapping out only the $8000-9fff addresses, for example.

The same with graphics. Originally, each cart had a special second ROM of $2000 bytes, which held all the graphics. If you want more graphics in a game, you need a mapper to switch that $2000 bytes with another chunk of ROM. Later mappers can swap even smaller chunks, say if you want to keep the hero sprites the same, but load just the enemy sprites for this level. Or if you just want to change the background graphics. Or, you can swap graphics every few frames, and animate the background (waterfall).

The symbols on the right are the ASCII equivalent of the hex values. I think 41 = A, 61= a.

nesdoug.com -- blog/tutorial on programming for the NES

linkncb16

  • Restricted Access
  • Full Member
  • *
  • Posts: 159
  • They actually updated this site. Hallelujah.
    • View Profile
    • Patreon
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #22 on: April 19, 2016, 04:05:55 pm »
I'm trying to use the hex editor in FCEUX and I have no idea what the hell is going on. Whenever I press any input the game lags severely and it seems like 1000s of hex values light up and change. How in the hell am I supposed to find where anything is?! Values $5000-$5FF0  never stop changing even if I am doing nothing. How is this supposed to be an effective method of finding anything.

April 19, 2016, 04:33:51 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
I also have know idea what PPU is. Don't know if that's important.
« Last Edit: April 19, 2016, 04:33:51 pm by linkncb16 »
Final Fantasy Redux is complete! Download

Disch

  • Hero Member
  • *****
  • Posts: 2713
  • NES Junkie
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #23 on: April 19, 2016, 06:05:33 pm »
I'm trying to use the hex editor in FCEUX and I have no idea what the hell is going on. Whenever I press any input the game lags severely and it seems like 1000s of hex values light up and change. How in the hell am I supposed to find where anything is?! [snip] How is this supposed to be an effective method of finding anything.

Pattern recognition.

As with anything else, it takes time to get good at it.  Don't expect to be a superstar right out of the gates.

If this is FF3... From maybe a few minutes of looking at the hex editor I can tell you:

- 00D0-00DF appear to be part of the music driver (see how they change in sync with the music)
- 00F0 is a frame counter (increments every frame no matter what the game is doing
- 0200-02FF is shadow OAM (typical in most NES games -- basically this is all the information for drawing on-screen sprites)
- player names are stored at $6106, $6146, $6186, $61C6.  Player stats are probably stored nearby.
- 0029 looks like the player X position (increments when you move right, decrements when you move left)
- 002A looks like the player Y position (increments when you move down, decrements when you move up)

Quote
Values $5000-$5FF0  never stop changing even if I am doing nothing

$5000-5FF0 are unmapped and are meaningless to you.

Stick to the ranges I outlined earlier:

0000-07FF
6000-7FFF
8000-FFFF

Anything not in that range kind of doesn't matter.

Quote
I also have know idea what PPU is. Don't know if that's important.

The PPU is the graphics processor on the NES.  Basically it's the thing that actually generates the image you see.
« Last Edit: April 20, 2016, 02:19:39 am by Disch »

linkncb16

  • Restricted Access
  • Full Member
  • *
  • Posts: 159
  • They actually updated this site. Hallelujah.
    • View Profile
    • Patreon
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #24 on: April 21, 2016, 03:14:45 pm »
Okay so I'm trying to dump the RAM before and after I make a change, but when I do the file is in .bin format and thus unreadable. How do I read the dump as if it were still in hex? And after I do that how will I quickly be able to spot changes between the two?
Final Fantasy Redux is complete! Download

STARWIN

  • Sr. Member
  • ****
  • Posts: 449
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #25 on: April 21, 2016, 03:31:02 pm »
huh? it is a binary file so you can read it with.. a hex editor. :P

i have used vbindiff (http://www.cjmweb.net/vbindiff/) myself when i have had the need to compare RAM dumps. there might be other methods but this worked well enough for me.

Disch

  • Hero Member
  • *****
  • Posts: 2713
  • NES Junkie
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #26 on: April 21, 2016, 08:03:48 pm »
Yeah get a hex editor to examine the bin file.

I strongly recommend HxD.  It's the only free hex editor I've used that doesn't suck:  https://mh-nexus.de/en/hxd/

If you want to compare two files ... I use Beyond Compare... which is a great program but isn't free.

dougeff

  • Sr. Member
  • ****
  • Posts: 359
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #27 on: April 21, 2016, 10:05:58 pm »
You won't be able to spot the change in a dump, unless you know exactly what you're looking for.

The advantage of using the hex editor tool in FCEUX is that you can watch the RAM while the game is running. Or, while you make changes. Using save-states, you can rewind the tape and try something else. Using 'cheats' you can set and freeze RAM values, and see what happens.

I can't imagine a use for a RAM dump, unless I was actually programming the game and had the source code...and knew exactly what each RAM address was programmed to do.
nesdoug.com -- blog/tutorial on programming for the NES

Disch

  • Hero Member
  • *****
  • Posts: 2713
  • NES Junkie
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #28 on: April 21, 2016, 10:42:08 pm »
I think the idea was comparing two different RAM dumps side-by-side

But yeah I agree it's probably easier to just load state repeatedly while keeping an eye on RAM.  At least for most things.

UltimateUrinater

  • Jr. Member
  • **
  • Posts: 19
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #29 on: May 12, 2016, 09:10:19 am »
Any difference between RAM and SRAM? Also, when the game is saved, is that just a massive overwrite of ROM values? Lastly, what is bank swapping?

April 19, 2016, 11:10:17 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Oh, and what are those weird symbols on the right of the hex editor?
When the game is saved...there is usually a seperate place within the cartridge that the saved game data goes to. For example, in the gameboy, there is a place for the ROM(read only) and then there is a seperate place for the RAM. 
Bank switching is done with the ROM. For example...lets say you can only access 32kb of the ROM at one moment.....then you switch to the next bank to access the next 32kb of the ROM. Bank switching is pretty much used to increase the amount of data you can access.

May 12, 2016, 09:16:04 am - (Auto Merged - Double Posts are not allowed before 7 days.)
OHHHHHH .....snap....there's two different pages....didnt know that.
 :laugh: :laugh: :laugh: :laugh: :laugh: :laugh:
« Last Edit: May 12, 2016, 09:16:04 am by UltimateUrinater »

DougRPG

  • Full Member
  • ***
  • Posts: 145
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #30 on: May 18, 2016, 01:59:49 am »
Quote
So if you load a ROM in a hexadecimal editor, is that the code for the entire game? In other words can any aspect of that game being modified by changing only those values?

Seems that you do not even know what hexadecimal is. You need to start from the very beginning.

Translating a game has a lot of parts, like translating the texts, editing the graphics, changing the game code, etc. If you want to change the game's code then you need to know programming and how to debug assembly and stuff like that. It's not novice stuff. Maybe you should focus on translating text, so go find someone who needs a translator.
If you want to work with code, and assuming you even know what hexadecimal is, then start studying programming from the very beginning and return to romhacking after some time (may take years).

omega_rugal

  • Jr. Member
  • **
  • Posts: 69
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #31 on: May 19, 2016, 11:09:22 am »
wow, you have a lot of question, maybe going full hex wasn`t the best way to go for a noob hacker...
Done your packing?, your life journey is over...

linkncb16

  • Restricted Access
  • Full Member
  • *
  • Posts: 159
  • They actually updated this site. Hallelujah.
    • View Profile
    • Patreon
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #32 on: May 23, 2016, 11:07:40 am »
wow, you have a lot of question, maybe going full hex wasn`t the best way to go for a noob hacker...
Well it's a bit difficult to learn when I have pretentious cunts like you refusing to provide useful information and instead insulting someone with less experience. Maybe posting here wasn't the best way to go for a noob English speaker?
Final Fantasy Redux is complete! Download

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #33 on: May 29, 2016, 12:04:04 am »
LOL. just wanted to chime in and say i agree with that these type of replies are the cuntiest, most pretentious and ignorant possible replies that this person's questions about hex editing could possibly merit.

1. to the person who is starting out with hex editing: ignore everything said here and google subjects like character control codes in hexadecimal editing; what are character codifications, what is shift-jiss, etc. start there and you'll naturally continue exploring these matters.

2. the short of it is that hexadecimal values are a mid-way representation of machine-readable and human-readable code. yes, if you open a snes rom with a hex editor you are indeed seeing the game's entire code but not in the way you're thinking, you're looking at a hexadecimal INTERPRETATION that nevertheless is very much code.

3. if you want to see much more detailed interpretations of a game's code download the program IDA Assembly software and... have fun :)

4. starting out i would say the easiest way to start translating game text is something like the following scenario:

- like you, i knew absolutely nothing about rom hacking and i also did not have a single clue as to what hex editing was. i have never coded or programmed anything before this point so i was as new as new can get. I had in my hands a japanese windows PC game called wizardry empire 1: the ancient princess, and i was desperate to play it but the game was in japanese.

- looking inside the game's folders i found that inside a folder there were some files that were named things like: ITEM.DAT, SPELL.DAT, MONDATA.DAT, etc.

- a friend adviced me to simply open those files with a hex editor and start carefully scrolling through the entire .DAT file(s) until I spotted anything I recognized, like a spell name or _anything_.

- I opened first the SPELL.DAT file, and immediately realized i was looking at all of the game's spells in japanese. and they were perfectly organized. I copied (by hand lol cos windhex32 doesn't support copy/paste) the japanese characters and then downloaded a japanese dictionary software and inputted the characters and then copy/pasted that into a google search and sure enough: it was indeed a spell and i knew immediately then that the one following after it in the file would be the level 2 spell, so on and so forth.

- armed with that knowledge I began doing what I now know is termed "corruption", but i didn't know that at the time: basically i just overwrote the japanese characters with english letters and then launched the game to see if the spells had changed. And sure enough, they had indeed!

so that was how i got into rom hacking. there is no better way to start than via pure hex :)

btw, i'm almost done translating that wizardry game. just waiting for some remaining NPC conversations to get translated by someone who is fluent and then hex-edit that in and done.

also, you're going to hear a lot about pointers: the best way to solve the pointer problem is learning to use IDA, hehe. IDA just tells you where shit is, it's great. But focus first on finding a game file that is clearly labeled, like the spell.dat one in my example, opening that in a hex editor (I RECOMMEND DOWNLOADING CRYSTALTILE2) and first do corruption, see if shit changes in-game, then go back and start replacing japanese characters with english words.

it's easier with PC games as console games always require a lot of hoops to jump through in order to extract the relevant files from out of the game.

oh, one pro tip i learned: go to google translate site and set it to ENGLISH -> JAPANESE, and then write in english the word you want to find inside the game. for example, i was recently busy working on wizardry empire 3 for PSP and the very first thing i did to quickly find where the list of weapons/items were was:

- go to google translate, set it to eng->jap, write "Sword", copy the japanese characters that google spit out.

- open the game using a PSP-specific tool for extracting the files out of the PSP game iso's (UGMD or however it's called) and extracted every file and then I went through every single file and did a CONTROL + F with the japanese characters for Sword that I'd copied earlier.

- sure enough i hit the jackpot in less than 10 minutes and found the japanese characters for Longsword very quickly in a .BIN file titled UMDDATA.BIN. using that as a starting point everything else fell into place.

if you need to find where the chunks of japanese dialog is i have actually had a lot of success by utilizing the game's japanese wiki/faq fan-site. set google search to language->jap, and region->jap, and search for any sort of wiki/faq fan site for the game you want to translate. if you're lucky you'll find one! and once you find one these type of sites are great because they have lists of items and sometimes a lot of DIALOG QUOTES.

copy the japanese dialog quotes and then control + F through the game files in your hex editor (crystaltile2) until you find it! this has worked for me very, very well. of course it depends entirely on the game having these type of wiki-faq fansites ^_^

<('   '<)

oh! btw, regarding the thing about changing the FF3 life spell's spell level? one easy way to do it is to move around the CONTROL CHARACTERS that are normally invisible and are hidden inside ordinary looking text.

i don't really have the energy right now to explain this step by step but basically, and i'll just use another wizardry example for quickness:

- in all 3 of the wizardry empire games, AND also in the elminage games, (both series are made by the same dev, STARFISH) you will always find a section in the hex data where in the TEXT FIELD to the right you will see the Wizardry game's spell schools abbreviated and in double-width letters (i.e. letters that are wider than "regular" ones, which are single-width)

- so in this example the alchemy spell school double-width abbreviated word they used was of course, A L C, but the kicker is that if utilize CrystalTile2's "CODE CONVERSION" option it flips shit and you can see the invisible control characters. What you THOUGHT was a "C" is in reality a lower-case "h", and this lower-case "h" is a control character.

- if you then go to M A G, which is of course next/near to A L C since you're in the "magic section" of the file, you'll see that the "G" is actually a lower-case "f". switch those two around and now your mage learns alchemy spells and your alchemist learns mage spells. weee!

- these control characters are directly related to the topic of pointer tables and to pointers themselves! they won't help you FIND the pointer, as you need to do either math utilizing addresses and whatnot or use IDA or an ASM program that is similar to IDA. I also want to stress that a lot of people talk endless about pointers but in my opinion they are not that important to keep track of in terms of a game translation... why?

- if the text don't fit, ain't no pointers gonna help you there. not REALLY. only way is to hack the game's font (i.e. the game functions that order the game code to call the font, to then have the font function begin "drawing" letter-glyphs from the "physical" font-bitmap file and then eventually placing the glyphs into character codification supported by the game, which is shift-jiss usually if it is a japper game.

- so basically don't worry too much about pointers and instead i recommend that after you are comfortable hex editing shit or whatever to start learning how to really use the debuggers in the emulators. they are invaluable. Google terms like "how do i backtrace a function", "what is a stack", hehe. if the emulator debugger is not good enough download CHEAT ENGINE it's pretty good especially for listing DLLs, functions and strings.

basically what i'm trying to say to you is that yes, you can ABSOLUTELY do what you want to do in your OP WITHOUT SPENDING 10 YEARS IN A HYPERBOLIC CHAMBER RELIGIOUSLY STUDYING PROGRAMMING LANGUAGES THAT WILL NOT HELP YOU IN ANY WAY BECAUSE:

SPOILER ALERT! HEX IS NOT A PROGRAMMING LANGUAGE!

you can probably do the life spell switchero by moving around the control characters for the spell and spell levels. obviously this requires spending time familirizing yourself with the FF3 game rom and how it ticks. it just pisses me off when people treat other people like dirt and dismiss their search for improvement by insulting them and trying to get them to lose hart and stop pursuing something.

like what has happened in this thread. I recommend that everyone who posted that this person needs to just unplug his laptop ASAP and never again use a computer until he has spent 100 years training in C++ to go fuck themselves and to not post at all if they are not going to at least provide a helpful reply.
« Last Edit: May 29, 2016, 12:43:34 am by aweigh »

FAST6191

  • Hero Member
  • *****
  • Posts: 2540
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #34 on: May 29, 2016, 05:34:35 am »
Those are quite harsh words so I went back over the thread. Questions were asked, they were then answered if they were sensible questions and if not the reasons why they might not have been the right path either in general or because a lack of experience would make it pointlessly hard were given. There were no insults and aside from a side still fairly relevant side conversations it was all pretty much on point.

I guess this is one of those "if you can't see the problem then you are it" situations. Though I am  quite content to call this a good thread and a good example of the forum doing what it does, if that is bad then meh I will stick around as long as it continues.

Anyway to the person who is starting out with hex editing, stop as it is not a skill or at least not a great way to frame things. You can learn to use a hex editor well and know the features of one, much like you can learn to use a word processor to do text formatting and layout, but if you lack a foundation in other aspects, or language if we are continuing the word processor analogy, then you are at best going to be able to hex edit at the direction of someone else, dictation being the word processor equivalent.
By all means learn what hex is (it is a numbering system, as computers tend to work in binary it is just 4 binary digits stuck together and as that means 16 combinations A through F get stuck on the end to represent 10 through 15 decimal) but there is a reason everything else is framed by what you might want to do (edit text, edit levels, edit graphics, edit music...).

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #35 on: May 29, 2016, 06:40:48 pm »








Please don't say such an ignorant thing as to tell someone who wishes to learn about game hacking that he should literally STOP using a hex editor. That is probably the most offensive and willfully detrimental piece of advice I've ever seen thrown to someone. How dare you tell someone to NOT LEARN?

I would be less offended if at least there had been then been many other examples of all of the other things he should be googling/studying/learning instead of hex editing but nope, absolutely nothing... Telling someone to stop doing something without a valid reason as to why they should stop and then not providing alternatives is worse than giving bad advice it's basically sabotaging the person.

Of course he can change a spell's spell level using a hex editor. In fact it's probably, _probably_, the simplest and fastest way to make such tiny changes in an RPG game.

---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------

My point of view on this "subject" is that hex editing is the single most essential tool in a game translation project. It is not the BEST tool for any single area, as there are better ways to translate text than by editing the strings in a hex editor; and it is not the BEST tool for editing anything at all that has to do with images/textures as you would need to be a bonafide hex-editing genius overlord to accomplish a texture edit that you could do in 2 minutes by simply extracting the game images and editing them with software and then re-inserting them; and it is certainly not the BEST tool for changing around game code as the majority of things needed in a translation project such as changing font width or hacking a new pointer table so you can make your own pointers and give yourself longer text strings to work with and of course for the implementation of variable width-- none of these things are done "better" with a hex editor".

HOWEVER beginning with a FIRM grasp of hexadecimal values can do and how the game itself utilizes them is extremely helpful for: DEBUGGING, REVERSE-ENGINEERING AND CODE MODIFICATION (ASM/ETC); and last but not least hex-knowledge is probably the single most important tool in the translation box when translating actual game text. Knowing the nitty-gritty of how what a HEADER is and what it does and knowing what CONTROL CHARACTERS for CARRIAGE-RETURN/LINE-FEED/ETC are of unbelievable importance.

Knowing what double-width is and why it exists and how it used and knowing what single-width is and why it exists and how it used and knowing the history and makeup of CHARACTER SET CODIFICATION, all the way back to the 1960's, will provide anyone with a better understanding of game translation due to the simple fact that they can immediately know how and why an NPC dialog box stops where it stops, and/or why TEXT WRAPPING issues happen.

These are very simplified examples, I know. I am not saying that you need to translate an entire game using a hex editor (although I have, and I enjoy editing hex more than I do implementing ASM OPCODES or editing textures for translation in Photoshop or dealing with anything else really).

I realize that you are speaking about this subject, i.e. game hacking, in a COMPLETE context. As you said yourself, level editing, etc.

I'm coming from a purely translation-oriented point of view as I have never been interested in changing a game's levels or in doing things like changing Cloud's 3D MODEL with SCOOBY-DOO sprites. Whatever floats your boat, I say, and indeed for those type of things you will NEVER, EVER need to use a hex editor!

However, some of the things one can do by editing hex values which a lot of people would say "should not be done in a hex editor" and instead done with "tools":

- I can open one the file containing the entire list of item data in any of Starfish studios' games (My goal is to eventually translate into english every single Starfish game, from 1999 to 2016); for example the ITEM.DAT file from Wizardry Empire 1, go to the item HOLY WATER which when used in battle produces the spell TZALIK which is useful versus undead enemies and I can LEARN the following things:

Each item has a hex value that assigns its DMG/Defense/Number of hits/special effect when used/special property it gives you passively or actively/what specific spell the item triggers when used that way/from what spell school that spell comes from/the amount of times that item can be used/etc. You can know instantly ALL OF THOSE THINGS simply from looking at the item's entry in a hex editor.

In the Holy Water item example I learned that it uses the spell Tzalik which the game assigns the hex value of '44', and guess what? when converted into decimal that relates directly to the placement of the spell in the game interface AND also relates directly to the spell LEVEL, i.e. 4th level in that spell school.

I also learned that it uses the value '68' to denote that this item is considered part of the PRIEST CLASS spell book. Items that trigger a MAGICIAN CLASS spell utilize the value '64' _in this particular game_.

Now guess what? I went back to the game's executable where all of the character class information and all of the spell information is there for our pleasure and I LEARNED that these values are no coincidence whatsoever!

Starfish utilized the hex value '64' as an invisible control character inside the double-width abbreviation of the Magician spell book, in this case what you see is M A G, and THAT is why items that utilize Magician spells are assigned the 64 value.

This isn't necessarily helpful at all for actually translating TEXT, _but_ it IS a completely LEGITIMATE example of how hex editing can be utilized to do things that people always tell you cannot be done or simply should not be done via hex. Armed with this game-specific knowledge I can play around and mod all of the items, all of the classes, all of the spells and switch whatever I wish around and effectively make a gameplay mod... all in less than 10 minutes and using a hex editor without ever using any other tool.

Now, the thing is the original poster was asking about two things: mainly about editing text, (naturally), and about how to change a spell's workings. I wanted to try to communicate that both things can be easily done with a hex editor if you study, REALLY study, how hexadecimal values relate to in-game factors.

Obviously if his questions were about how to extract a complete game script from a compressed game archive and then build a text-editing tool that will re-insert the text back into the archive and modify the appropriate headers then of course no one in their right mind would say "brah use windhex32" or whatever. Same with if he wanted to modify the game's music/sounds, or the game's textures/models.

If he HAD asked about that I would now be writing an extremely long post about how to use, STEP-BY-FUCKING-STEP, the myriad CONSOLE-SPECIFIC tools that are currently available so he can go and rip the textures/models/music/etc and then what programs he needs to download so he edit them, how to edit them to achieve what he wants, and then finally how he can go about re-inserting them.

I'm not saying It would be a guarantee of what I write working for him, since every game is completely different (a TRUTH I have had to learn the hard way...) but it would at least HELP HIM GET STARTED.

And, btw... STILL no need for any programming knowledge as of yet in ANY of these examples.

Where programming knowledge "comes in handy" is when reverse-engineering, which is what I'm teaching myself right now. (I need to implement a variable-width font hack into Wizardry Empire 3: Return of the Emperor for the PSP console). And fuck, I'm being generous here as reverse-engineering and implementing a few opcodes and/or expanding a "rom" are not REAL PROGRAMMING.

Some of the people in these communities, mainly here but also in GBATemp (they're much nicer and much more helpful to people than the people here on RH) have a somewhat high opinion of themselves in terms of how they are master coders and how game hacking is all about being a l33t programmer. Sure, in many, many examples, but not in others; such as someone asking how to translate into english a text string for example.

I had excellent success modifying the Wizardry 1 font utilize ADA/Cheat Engine but then I hit the hard truth that the second game's font I need to hack, Empire 3 on PSP, is not x86 architecture. I DID NOT EVEN KNOW WHAT x86 IS BEFORE THIS. However I googled and taught myself and of course asked for help from people (not from gbatemp/RH, but from an RPG website I post in, where one person actually went and step-by-step showed me how to use cheat engine to locate strings and eventually locate and modify the _HEX VALUES_ that determined the font's width, height, pitch, precision, output, quality, pixel size and its weight.

Once again, he probably only did it because it is an x86 game and thus easy as fuck to hack since absolutely EVERYTHING is spelled out for you inside IDA or even Cheat Engine. A toddler could figure most things out. The problems are always in console game hacking as they are... tedious to work with... to say the least.

I choose to look at it positively: if it hadn't been for my deciding to begin translating Empire 3 on PSP I would never have run into the font problems and I would never be on the path I am right now studying websites and forum posts and googling obsessively on my free time and learning more and more about reverse-engineering.

Dabbling with Elminage 2, also on PSP, I was forced to learn to extract and edit textures and translate the japanese characters on the textures into english and then re-insert them into the game. I had NEVER BEFORE EVER edited a single image file in my life before this. Once again an obstacle became a great experience for learning. Note that I have not mentioned that a hex editor would be needed in this example!

Anyway, fingers are tired from typing lol

May 29, 2016, 09:30:09 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
« Last Edit: May 29, 2016, 10:04:12 pm by aweigh »

tomaitheous

  • Hero Member
  • *****
  • Posts: 543
    • View Profile
    • PC Engine Dev
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #36 on: June 05, 2016, 05:24:38 pm »
Those are quite harsh words so I went back over the thread. Questions were asked, they were then answered if they were sensible questions and if not the reasons why they might not have been the right path either in general or because a lack of experience would make it pointlessly hard were given. There were no insults and aside from a side still fairly relevant side conversations it was all pretty much on point.

I guess this is one of those "if you can't see the problem then you are it" situations. Though I am  quite content to call this a good thread and a good example of the forum doing what it does, if that is bad then meh I will stick around as long as it continues.

Anyway to the person who is starting out with hex editing, stop as it is not a skill or at least not a great way to frame things. You can learn to use a hex editor well and know the features of one, much like you can learn to use a word processor to do text formatting and layout, but if you lack a foundation in other aspects, or language if we are continuing the word processor analogy, then you are at best going to be able to hex edit at the direction of someone else, dictation being the word processor equivalent.
By all means learn what hex is (it is a numbering system, as computers tend to work in binary it is just 4 binary digits stuck together and as that means 16 combinations A through F get stuck on the end to represent 10 through 15 decimal) but there is a reason everything else is framed by what you might want to do (edit text, edit levels, edit graphics, edit music...).

 I kind of agree with this view point. As some point, a hex editor without knowledge of the system's processor or debugger (as in how to properly step through code and understand what it's doing), is going to be beyond inefficient. It amazes me to the lengths a lot of hackers go to in avoiding using something with soo much more efficiency like an emulator debugger, but instead limit themselves to dabbling around in hex editors. The fact that hackers also tend to avoid using an assembler, is also right up there. Then again, most hackers here aren't programmers. I'm not baggin on anyone in particular, it's just that hacking takes such dedication and figuring stuff out (self learning), and a whole range of intellectual skill and effort, that it makes me cringe when I see more than not - "backyard" approaches every where; bad habits learned early on.

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #37 on: June 06, 2016, 05:11:42 pm »
...ok i've had a lot of time to calm down, heh.

i understand what you mean about the "backyard practices" and learning "bad habits". and i agree wholeheartedly that game-hacking (and/or "japanese-to-english translation", which is what 95% of the romhacking.net and gbatemp.net userbase want to achieve) is indeed incredibly difficult.

it is reverse-engineering, plain and simple. i've mentioned in my posts in this thread, and in another thread that is also titled something about hexadecimal editors, that i keep failing to see how a beginning game hacker (i.e. japanese-to-english translator) is doing himself or herself "wrong" by starting out "full hex" and learning from hexadecimal --> ida debugging and emulator debugging and cheat engine usage --> actual programming study.

now, see, here's the detail that i finally figured out; the thing that makes these hexadecimal editor usage threads make complete sense to everyone reading right now:

- the majority of the people posting on rh.net and gbatemp.net are not programmers and do not come from a programming background. myself included.

- to someone with zero programming knowledge and a game which they want to translate this theoretical rh.net/gbatemp.net user will, obviously, opt to open the game's files first in a hex editor because that is probably what one of his "buddies" will recommend they do first.

- this person is going to reach the following: "go ahead and continue translating all of this japanese text that i'm finding inside these files using this hex editor and continue making actual in-game progress", OR... "stop completely what i'm doing and take a step back and decide to put on hold this game's translation until i spend the necessary amount of time studying all of the programming knowledge i will need".

which do you think these users will choose? I know which one I chose! and because I chose the first option, I have already translated 1 game into english and have another half done.

HOWEVER... after exploring as much as could be explored via hex editor I obviously saw all of the limitations that I faced with each and every game I wanted to translate. this person will see that the japanese games all have monospaced fixed-width double-width fonts. I cannot do much about this via hex editor, so I must explore other avenues. And now this person will google and learn about the things he or she needs or wants to do which can be only be done by writing custom tools that will:

- extract text into a seperate tool for translation (and all of the baggage that comes with a tool like this).
- expand and patch all of the necessary shit for variable-width.
- (almost definite) probability of having to create or modify a tool for extracting graphical images.
- editing images and then re-inserting them into the game.
- translation-patch creation and distribution for the game.
- extraction tools for full-motion videos, audio and subtitles and other stuff like that.
- editing those things and re-inserting them.
- (probable) creation of new textures and new graphics for the game to accomodate the translation.

by the time this person is finished doing this game's translation the last time they will have edited the game files using only a simple hex editor will probably have been months, if not LITERALLY 1 or 2 years ago.

(the average time required for a translation group/person to go from zero programming knowledge to implementing variable-width and graphical hacking, etc, of a game is 1-2 years. This time-frame is taken from the anecdotal progress-blogs of translation projects for games such as Mother 3 and many, many others).

now we come back to the beginning: a person with zero programming knowledge can translate a game's text, AND even modify its font if they get creative with the emulator debugger + IDA + Cheat Engine and find the hexadecimal values for the font "weight", "width" and "height", and/or the hexadecimal values for the amount of pixels in-between each drawn game-letter.

And if they're lucky and the game does not require ANY font modification then they can probably translate the game's text as long as it is not compressed.

This is of course assuming it is a small-scale game, obviously, as 1 single person translating via hex editor a game that is massive such as a final fantasy or whatever will probably never finish, even assuming the font does not require modification. (yeah right!).

so, once again: a rh.net/gbatemp.net user on the majority is not a programmer and does not come from a programming background; they are people that usually want to translate 1 specific game into another language AND/OR they wish to "mod" that 1 specific game in odd ways that is basically only for their own private use... such as the OP asking how to change FF3's life spell's spell level from 5 to 4, and also asking how he could go about editing the ff3 "maps".

obviously a hex editor isn't going to help him much, if at all, in the map thing... but here's the thing: instead of immediately telling this person that he has to completely stop what he's doing with FF3 and instead go learn "programming knowledge" because he "does not know what he is doing"; the friendly neighborhood rh.net/gbatemp.net poster could reply instead with:

- the tools he'll need to extract the maps, the tools he'll need to edit them, and how to do this step-by-step, or at least with google links to relevant forum posts.

- explanations about what a hex editor can do for him and what he can do with it, and in this particular case the rh.net/gbatemp.net poster could download the ff3 rom (or plug his cart into his PC) and do the LIFE spell's spell level change and then demonstrate step-by-step how to replicate it using a hex editor, AND/OR provide relevant forum post links of similar things.

when the person, i.e. the OP in this scenario, begins asking about things that are beyond this scope then a helpful reply instructing them in the stuff they need to learn accompanied by hopefully many, many google links!

if this type of reply is anathema to you and instead you can only think of replying with all of the things he is "lacking" and without anything positive/concrete to ADD then your reply is of 0 (zero) use to the poster. and when in doubt... just don't reply!

an empty thread can probably serve better than a 2 or 3 pages of programmers arguing with complete beginners who do not know a programming language from the other about the best way to achieve something as simple as changing ff3's life spell's spell level.

and that's really the crux: the majority of the people posting for help on things like this OP are concerned with translation and a translation can be done via a hex editor. many have been done. especially for simple, small games like nes/snes roms that are well documented.

if a hex editor-only translation of a game offends you because it is of "poor quality" then remember that the person who did it... did not do it for you!

they did it so that the game could be enjoyed by people who really, really, REALLY want to play it! regardless of whether the translation is "top notch! master stroke!" or "dude did it using only a hex editor and atlas".

THE ONLY THING THAT MATTERS IS THAT A GAME CAN BE PLAYED IN YOUR LANGUAGE. NOTHING ELSE. HOW IT WAS ACHIEVED DOES NOT MATTER.

 

FAST6191

  • Hero Member
  • *****
  • Posts: 2540
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #38 on: June 06, 2016, 07:24:07 pm »
"THE ONLY THING THAT MATTERS IS THAT A GAME CAN BE PLAYED IN YOUR LANGUAGE. NOTHING ELSE. HOW IT WAS ACHIEVED DOES NOT MATTER."
So if it compiles it ships? There is plenty of room for elegance. Also if the end result can be just as refined in a sensible amount of time with just a hex editor then more power to you, I don't think I could do it. Avoiding unnecessary work is pretty much the definition of gaining a trade and I can't see that as a bad thing.

"- the majority of the people posting on rh.net and gbatemp.net are not programmers and do not come from a programming background. myself included."
Those looking to be hackers can no more avoid gaining some programming knowledge than those looking to be readers can avoid learning words and grammar. Equally it is not so much classical programming knowledge (text encoding for programmers usually runs use ascii if you are a cowboy (or it is 1990 when you are learning this) or if you are really good then unicode ( http://www.joelonsoftware.com/articles/Unicode.html ) and there is this freaky old setup called EBCDIC but you will hopefully never encounter that) but general concepts of data representation which dicking around in a hex editor will not get you.

Going a bit further what you place under the umbrella of hex editing is probably not what the rest of us was classify it as. I would still say you are a bit overly enthusiastic about the concept. If it works for you then great, I absolutely can not get behind teaching others to go the same path though.

"an empty thread can probably serve better than a 2 or 3 pages of programmers arguing with complete beginners who do not know a programming language from the other about the best way to achieve something as simple as changing ff3's life spell's spell level."
There invariably comes a time when too much off topic is a thing, indeed it would probably be the nicer aspects of being a mod, but so much juicy stuff comes out of the side discussions that to try to curtail it would do a disservice to almost everybody. Failing that is give a man a fish not in play here?

There are times when endless repetition of basic tasks gets you somewhere where you are not a liability any more, this is not one of them. There is plenty of boring grunt work you can get in on though if you want that too -- fill out my tables with the punctuation if you want, proof reading...

"- the tools he'll need to extract the maps, the tools he'll need to edit them, and how to do this step-by-step, or at least with google links to relevant forum posts."
That maybe exists for pokemon, some aspects of mario games and not a lot else. The vast majority of games will not have been looked at properly, and maybe not at all outside of automated searches.
Equally what you might be reading as cunty, pretentious and ignorant could also be the 5 minutes before I have to do something, but that 5 minutes can probably help another already in the know be able to direct it better.

" obviously, opt to open the game's files first in a hex editor because that is probably what one of his "buddies" will recommend they do first."
I usually find that if it is a file system based console then it will be opened with a tool for handling that and then known names/extensions/magic stamps will be used to direct more automated tools. If not that then premade tools will be used and if not that then comes tile editors, emulators, cheats and only after that comes a hex editor, assuming they have not given up before then.

Also
- extract text into a seperate tool for translation (and all of the baggage that comes with a tool like this).
- expand and patch all of the necessary shit for variable-width.
- (almost definite) probability of having to create or modify a tool for extracting graphical images.
- editing images and then re-inserting them into the game.
- translation-patch creation and distribution for the game.
- extraction tools for full-motion videos, audio and subtitles and other stuff like that.

If not having to deal with pointers in my head and all the other things that text hacking tools seek to avoid is baggage then so be it.
That really depends upon the system.
Tile editors exist and work for most here, they also have extraction and insertion options.
See above. It is not ideal but plenty doable.
Pretty trivial, though I suppose uploading it to major hack sites or hoping it gets picked up and spread by others can be tedious.
Done in almost no cases.

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
Re: Is there anything that cannot be edited via Hexadecimal?
« Reply #39 on: June 07, 2016, 10:19:43 pm »
THAt was a very good post. i can't say i disagree with anything you said.

my main point has basically become: "for a person with zero programming knowledge they can open a japanese computer RPG game's main .executable in any hex editor and scroll down until they find japanese text which they can then translate by hand".

for most of "these type" of users, and i include myself among this groupd, the goal is not to gain knowledge but to complete a game translation. if completing this game's translation requires a months-long immersion in assembly in order to hack the game's font then great, however it would be preferable if not because that's a lot of time "wasted" on ASM stuff; remember that for the majority of these would-be translators they are not doing it to release the translated game to the "scene", in fact the "scene" would be repulsed by their efforts and most probably badmouth their efforts and send the would-be translator into a confused depression as to what exactly he or she is supposed to do.

no, this theoretical group of would-be translators i am painting in this example, and i use myself as the basis, began to become interested in hacking game text because quite simply they ran across a japanese video game (or other language) that they've been wanting to play for years, and YEARS, and for whatever x or y amount of reasons it is NOW that the necessary steps are taken into doing something about it. the goal is not to release but to translate the game so you can PLAY IT YOURSELF.

from that point and as the person grows and learns more about everything related to hacking game text via hexadecimal editors they will absolutely start branching out and learning about other "methods". always remember that none of these "other methods" were on their radar, remember that this person is a gamer and has (probably) never taken a single comp. sci. course.

to these persons there is simply nothing simpler than opening a game's .exe or the game files in a hex editor and doing _immediate game-changing translation work_ on the game they want to play.

this will VERY quickly cause them to slam headfirst into all sorts of problems for which they don't have any answers and the ONLY answer they have is that the closest they've ever come to programming has been through hex editing. temping to call this scenario unique to myself but trust me you would be very, very surprised at just how many people all of what i've written describes to the TEE.

it is completely up to them whether or not they will be writing text extraction software that handles all of the game's pointers 1 or 2 years down the road from the day they first opened a game rom or file in a hex editor....

because, in the end, if for example they can translate all of the text using only crystaltile2, i.e. they are very lucky and their game of choice features no compression or other nasty things AND features a workable font that natively supports latin-ascii characters and all of that good stuff, then quite probably the only time this person will venture away from the text editor will be when they reach the moment when they realize they also have to translate the japanese text in the game's textures.

of course, as i mentioned previously, all of this mainly applies to would-be translators fixated on a SMALL-SCALE game which, having no knowledge of how to manipulate data before this, they will dive into using a hex editor. gigantic games that feature thousands of lines of text strings are OF COOOUURSE out of the scope of being worked on with 1 single hex editor. that goes without saying!

however the "small-scale game" caveat also applies to "small-scale hacks" like... you guessed it, someone who wants to change FF3's life spell's spell level! 1 or 2 days' worth of googling chracter control codes and shit and they will be able to do this.

any other kind of reply, such as "stop trying to do that and instead first learn all of this stuff which will not immediately help you change that spell's spell level" tends to rub people the wrong way.