News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Making hacking easier  (Read 12820 times)

zonk47

  • Sr. Member
  • ****
  • Posts: 343
    • View Profile
Making hacking easier
« on: April 08, 2016, 02:11:16 pm »
What can we do to make the process of hacking easier and faster?
A good slave does not realize he is one; the best slave will not accept that he has become one.

Chronosplit

  • Hero Member
  • *****
  • Posts: 1395
    • View Profile
Re: Making hacking easier
« Reply #1 on: April 08, 2016, 03:11:56 pm »
Not much IMO.  Research ends up in better tools for automating the simpler tasks in certain games, but other than that eventually you're always going to end up doing the same thing either way.

zonk47

  • Sr. Member
  • ****
  • Posts: 343
    • View Profile
Re: Making hacking easier
« Reply #2 on: April 08, 2016, 04:00:17 pm »
I think better emulator debugging functionality is key.

Let me restate/extrapolate on some ideas I had about the FF Legend/SaGa disassembly, and its challenges, that I conducted last night:
- there should be a debugger view option which displays the last values of the registers (only those changed by the instruction) at the moment of execution alongside the code. Also, for the bit manipulations such as shifting and rolling, it would help if the actual decimal arithmetic equivalents were also displayed.
- as someone mentioned in the "How to increase interest in hacking" thread, there is need for the ability to make the main routine auto-trace its loops. I can think of a few methods by which to implement this functionality, each with their own purposes.
- the code view window should only display bytes already executed as ASM, and mark areas read into RAM appropriately, so as to "map out" the ROM as the game is played. While there are limited cases of self-modifying code/RAM execution, I think work-arounds could be made for these. The former stipulation is vital, as the status quo is confusing as hell to everyone and makes hacking a pain generally.

Also I think there needs to be a "trace on input" function, in that this would help the hacker to invoke the debugger when triggering circumstances (such as map enters/exits and NPC triggers) that require decompression of data. We could crack compression easier this way, and find addresses for things like HP/shop data/exp with less trouble.

And I think we need to make these changes ourselves, because I don't think we can necessarily count on emulator authors to do them for us. If we do do them, however, hacking suddenly becomes loads easier and takes exponentially less time to pull off. We do need to stop relying on tools like IDA Pro though, because only debuggers can take the mental work burden off our brains (nevermind the fact that IDA Pro is designed with circumstances that have nothing to do with game ROM hacking in mind, nor will it ever be).
« Last Edit: April 08, 2016, 04:11:50 pm by zonk47 »
A good slave does not realize he is one; the best slave will not accept that he has become one.

FAST6191

  • Hero Member
  • *****
  • Posts: 2610
    • View Profile
Re: Making hacking easier
« Reply #3 on: April 08, 2016, 05:37:47 pm »
Most debuggers that allow you to step through code will allow you to view the registers as much as you like.

I am not sure how useful a decimal readout would be though, maybe for cheats (which tend to have such things already) but if you have made it to the point in life where you are doing something useful with a debugger you probably know hexadecimal well enough. That said it would probably be but a few lines of code and not take up much UI space so eh.

A logging emulator that gives me a good idea of the standard game loops, vblank loops and such could be helpful at times. I can not see it being of all that much use outside of the people reverse engineering an entire engine though. My just needing to hack in a variable width font or something is less likely to benefit from this. Possibly more on that in a bit.

An actually executed instructions function to help narrow down data or nothing could be of some use too. I will avoid saying too much more as I deal mainly in later systems there "ram execution" is arguably the norm or at least nothing to note and share with the board as an oddity. Self modifying code we tend not to see (programmers largely agreed it was not a great plan) but emulation on systems (and I see everything from partial emulation to game specific emulation to could easily inject any old game you like in there) and interpreted languages would be troubled too I reckon and those are popular.

Compression seems to be less of an issue than it once was for the newer consoles which tend to use standard types (we call it things like GBA BIOS compression even). The compression is seldom the hard part anyway, it seems to be more the writing of a decompression tool or a compression tool (even a compression dodging tool that just sets uncompressed flags in the case of something like LZ) is the bigger ball ache. That said I think my approaches are strange here anyway and the main use for an emulator for me when testing compression related things is giving me a decompressed/decrypted file to look at as the result or testing it once I tried to put it back in. Again though I am not faced with too many truly custom compressions on the newer consoles and for the most part it is just a difference in flag type ("ooh this one uses 3 bits for the copy value and 5 for the read back distance compared to the normal 4,4 split" being the extent of the excitement there).

"in that this would help the hacker to invoke the debugger when triggering circumstances (such as map enters/exits and NPC triggers)"
Since we left single screen games and got scrolling then map exits are not really a hardware concept. If you then still have to debug the game to figure out what to look at for said concepts then a standard breakpoint would do, maybe one you tie to a certain value/range of OAM if you prefer to reference it to something on the screen, or better yet if you have to press a button to trigger the exit in the game then a standard button press breakpoint could do things here.
That said I forget what it is called but fceux offers a kind of new executed code so you start the session, do everything but what you want to look at, note it and then do the thing where the emulator then says "this is new" and you have your code to look at.

justin3009

  • Hero Member
  • *****
  • Posts: 1615
  • Welp
    • View Profile
Re: Making hacking easier
« Reply #4 on: April 08, 2016, 05:53:22 pm »
I'm not sure as far as debugging on other systems but the SNES has it really nice with Geiger's SNES9X (Even though it's not 100% perfect) with the Trace Logging system it has.  That is seriously a god send when it comes to debugging.  More debuggers should incorporate things like that in my opinion, make it a lot easier to figure the code out.
'We have to find some way to incorporate the general civilians in the plot.'

'We'll kill off children in the Juuban district with an infection where they cough up blood and are found hanging themselves from cherry blossom trees.'

SleepyFist

  • Hero Member
  • *****
  • Posts: 842
    • View Profile
Re: Making hacking easier
« Reply #5 on: April 08, 2016, 05:59:11 pm »
Stable/Updated/Better Utilities would be great, many are in a state of decaying compatability.
Sleepy's tune of the week|| 2 Mello - after midnight: unaired broadcasts || https://youtu.be/FKqLCJ4mStI

RyanfaeScotland

  • Sr. Member
  • ****
  • Posts: 361
    • View Profile
    • My Brill Game Site
Re: Making hacking easier
« Reply #6 on: April 08, 2016, 06:10:29 pm »
Learn more. The more you learn the easier and faster it will become.

zonk47

  • Sr. Member
  • ****
  • Posts: 343
    • View Profile
Re: Making hacking easier
« Reply #7 on: April 08, 2016, 06:15:09 pm »
Quote from: FAST6191
Compression seems to be less of an issue than it once was for the newer consoles which tend to use standard types (we call it things like GBA BIOS compression even). The compression is seldom the hard part anyway, it seems to be more the writing of a decompression tool or a compression tool (even a compression dodging tool that just sets uncompressed flags in the case of something like LZ) is the bigger ball ache.

This is inconsistent with everything everyone but you has said about the topic. When people stop their hacking, it's usually because they can't deal with the compression.

Quote from: FAST6191
Since we left single screen games and got scrolling then map exits are not really a hardware concept.

I didn't say they were. Map exits happen when the sprite moves beyond the logical boundaries of the map, or onto a door tile. This happens, usually, in response to the player's motion of the player to move into that space. Once the player moves into that space, the game takes over until the next map is loaded. This about always involves decompression, and is moreover the only time except at the intro/ending sequences when it actually happens. So if say, you had a control panel in which to check a button for break-on-press, you could mosey your character over to the map's edge, or next to an NPC, then pause the emulator; check the button; and press the button. Then you'd be able to trace all the code dealing with the transition between maps.

This is actually completely necessary, because the code for input handling is generally interrupt based, particularly on handhelds where it is expedient to relieve pressure on the battery by suspending execution until an interrupt forces it to resume.

Learn more. The more you learn the easier and faster it will become.

Not likely. There's too much to be learned and not enough people willing to become specialized experts.

I do think we have to do this stuff ourselves, particularly because ROM hacking divides the community. In the beginning emulator authors had to work hand in hand with hackers... this is no longer the case. One of the most notable events that I can recall in recent emulator history was the departure of one of the lead programmers of PCSX2 several years ago. The departure was abrupt and without qualm: the dev said he had been offered a job that would receive his complete commitment from that day forward. Considering how passionate people in this community tend to be about their work, it must have been quite a sum. Over the past decade emulator authoring has moved from pariah status to sought-after specialization: emulator authors, for modern systems at least, are thought of as being experts in the modeling of highly abstract systems and self-starters besides, strong qualities for any technical project lead. At least a hundred (probably more) theses have been presented around emulator design. Association with pirates and/or people who work with (presumably) pirated material is a professional liability. Even those who are not pursued for their skill (or else dismissive of the pursuit) seem to be doing fine on Google Play, except when an association with piracy is made. Emulator authors have little financial incentive to make our job easier, so we will have to make the improvements ourselves. (disclaimer: I'd find it awesome if some of these ideas were integrated into working emulators by their original authors... I just don't think it likely).

April 09, 2016, 12:09:18 am - (Auto Merged - Double Posts are not allowed before 7 days.)
I'm not sure as far as debugging on other systems but the SNES has it really nice with Geiger's SNES9X (Even though it's not 100% perfect) with the Trace Logging system it has.  That is seriously a god send when it comes to debugging.  More debuggers should incorporate things like that in my opinion, make it a lot easier to figure the code out.

Geiger's emulator is a start. Needs work, but a start nonetheless.

@FAST6191: it seems to me that the following approach would make your work easier:
- you could run the decomp code in a CPU core to conduct the decompression of the material to RAM, then HDD automatically. This way, you'd only need one program for all decompression needs. As for the recompression, you could replace the decomp routine with a generic (such as LZW or Huffman) and compress it with that.
« Last Edit: April 09, 2016, 12:09:18 am by zonk47 »
A good slave does not realize he is one; the best slave will not accept that he has become one.

Rotwang

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Making hacking easier
« Reply #8 on: April 09, 2016, 03:17:49 am »
We do need to stop relying on tools like IDA Pro though, because only debuggers can take the mental work burden off our brains (nevermind the fact that IDA Pro is designed with circumstances that have nothing to do with game ROM hacking in mind, nor will it ever be).

IDA Pro is tailor made for ROM hacking.  It has support specifically for Gameboy and SNES ROMs and will map out all the registers for you, address the ROM as though it were loaded on hardware and has this kickass tree view that makes ASM tweaks child's play. What the fuck is wrong with your brain that you created a thread called "Making hacking easier" and one of your suggestions is that people STOP using IDA Pro?


Not likely. There's too much to be learned and not enough people willing to become specialized experts.


And now you say that there's no point in learning how to hack because there is too much to learn... You're only kidding yourself at this point. If that's really your goal you might as well have called this thread "Making hacking braindead" which is completely unrealistic. You can't turn this hobby into something that it isn't. You get out of it what you put into it, and that's true of any hobby. If not learning anything is your strategy then all you can really do is sit around twiddling your thumbs for someone to make your magic bullet decompressor tool for you, which I guess is technically easier than learning to do your own hacks. The problem is much more complicated than you are making it seem, and I get the idea that you really don't know what you're talking about. You can't really expect the entire hobby to work like it's some sort of commercial make-your-own-game toolkit.
« Last Edit: April 09, 2016, 03:46:14 am by Rotwang »

Turambar

  • Full Member
  • ***
  • Posts: 145
    • View Profile
Re: Making hacking easier
« Reply #9 on: April 09, 2016, 03:50:36 am »
A feature I've always wanted is the ability to see what the program counter was on the previous instruction or even the last handful of instructions. Sometimes a breakpoint happens right after a jump and it would be really useful to see where it jumped from.

tryphon

  • Hero Member
  • *****
  • Posts: 722
    • View Profile
Re: Making hacking easier
« Reply #10 on: April 09, 2016, 03:59:10 am »
A feature I've always wanted is the ability to see what the program counter was on the previous instruction or even the last handful of instructions. Sometimes a breakpoint happens right after a jump and it would be really useful to see where it jumped from.

quoted for truth :)

Rotwang

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Making hacking easier
« Reply #11 on: April 09, 2016, 04:01:27 am »
A feature I've always wanted is the ability to see what the program counter was on the previous instruction or even the last handful of instructions. Sometimes a breakpoint happens right after a jump and it would be really useful to see where it jumped from.

Logs are really useful for shit like this.

Turambar

  • Full Member
  • ***
  • Posts: 145
    • View Profile
Re: Making hacking easier
« Reply #12 on: April 09, 2016, 04:11:24 am »
You could get it in a log, but then you'd have to properly time and make the log file and then dig through it. I want to see it immediately in the debugger, that is much more convenient.

STARWIN

  • Sr. Member
  • ****
  • Posts: 449
    • View Profile
Re: Making hacking easier
« Reply #13 on: April 09, 2016, 10:31:56 am »
- there should be a debugger view option which displays the last values of the registers (only those changed by the instruction) at the moment of execution alongside the code.

trace log? or trace loggy additions for the disasm screen?

Also, for the bit manipulations such as shifting and rolling, it would help if the actual decimal arithmetic equivalents were also displayed.

what? like multiplying by two or dividing by two? for bit arrays definitely useless. rolling sounds like algo wizardy, it tends to be equally incomprehensible regardless.

- as someone mentioned in the "How to increase interest in hacking" thread, there is need for the ability to make the main routine auto-trace its loops.

how is this any different from setting breakpoint in the loop and using a trace log? "auto"? do you even know what is part of main routine and what not?

- the code view window should only display bytes already executed as ASM, and mark areas read into RAM appropriately, so as to "map out" the ROM as the game is played. While there are limited cases of self-modifying code/RAM execution, I think work-arounds could be made for these. The former stipulation is vital, as the status quo is confusing as hell to everyone and makes hacking a pain generally.

what are the benefits of mapping out a ROM?

Also I think there needs to be a "trace on input" function, in that this would help the hacker to invoke the debugger when triggering circumstances (such as map enters/exits and NPC triggers) that require decompression of data. We could crack compression easier this way, and find addresses for things like HP/shop data/exp with less trouble.

is this the same as setting a breakpoint at input routine that breaks when a button is pressed and then turning logging on manually?

And I think we need to make these changes ourselves, because I don't think we can necessarily count on emulator authors to do them for us.

good luck, but won't you turn into an emulator author if you do that?

A feature I've always wanted is the ability to see what the program counter was on the previous instruction or even the last handful of instructions. Sometimes a breakpoint happens right after a jump and it would be really useful to see where it jumped from.

I suppose "last PC + last n branches taken" could save a bit of time. When within a spaghetti routine at least. Between routines is easier because of step out.

zonk47

  • Sr. Member
  • ****
  • Posts: 343
    • View Profile
Re: Making hacking easier
« Reply #14 on: April 09, 2016, 01:20:53 pm »
I must admit... I'm quite taken aback... it looks like nobody here means to help me achieve these goals, so in self-respect to myself I am frankly hesitant to try advancing them any further. Is your pride really so great, that you are unable to make these small sacrifices for the good of the community?
« Last Edit: April 09, 2016, 02:05:25 pm by zonk47 »
A good slave does not realize he is one; the best slave will not accept that he has become one.

Isao Kronos

  • Hero Member
  • *****
  • Posts: 1249
    • View Profile
Re: Making hacking easier
« Reply #15 on: April 09, 2016, 02:12:58 pm »
you talk about "the good of the community" but some things I've seen you say don't match up with that

Rotwang

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Making hacking easier
« Reply #16 on: April 09, 2016, 02:13:13 pm »
I must admit... I'm quite taken aback... it looks like nobody here means to help me achieve these goals, so in self-respect to myself I am frankly hesitant to try advancing them any further. Is your pride really so great, that you are unable to make these small sacrifices for the good of the community?

Okay, I will stop using IDA Pro and stop learning things. For the good of the community. You have convinced me.

What sacrifices are you even talking about? The time spent on making a magical tool that hacks all the ROMs for you and can abstract things like "map exits" on its own? Good luck with that. The fact that you want a program to tell you what math is really being accomplished by bit shifting betrays your inexperience. That's something that's covered quite early on in ASM documentation.

And what's with this "your pride is so great" bullshit? What pride? Stop trying to cover up the fact that this is a utility request thread in disguise as some kind of community altruism. Or is it a utility request thread? With some of your other ideas I don't know what goal you are even trying to accomplish here, I guess for someone or something to do all your thinking for you?


April 09, 2016, 02:29:59 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
You could get it in a log, but then you'd have to properly time and make the log file and then dig through it. I want to see it immediately in the debugger, that is much more convenient.

It's not like you only get one shot at it. Set a breakpoint to your instruction, record a log, and then the only "digging through" you have to do is looking at the end of your log.
« Last Edit: April 09, 2016, 02:39:19 pm by Rotwang »

zonk47

  • Sr. Member
  • ****
  • Posts: 343
    • View Profile
Re: Making hacking easier
« Reply #17 on: April 09, 2016, 03:13:47 pm »
Rotwang you are totally misunderstanding me. I talked about using an AI to assist in language translation, but I don't believe it would be the right road for hacking. Such a pipe has two ends, as they say, and the tools we make to help us could also be used against us.

I would just like to have greater clarity as to what's going on in the machine. I've got a couple VB8-based emulator projects in my IDE right now, but I could only do a really intuitive job of it by putting in custom code for each of the 256 (or more) operands in these cores.

As for the program counter history that's a five minute fix, for the VB-based emus anyway. This is not a utility request thread, it is a -collaboration- request thread.

Quote from: STARWIN
what are the benefits of mapping out a ROM?

It helps you locate obscure code.
« Last Edit: April 09, 2016, 03:24:05 pm by zonk47 »
A good slave does not realize he is one; the best slave will not accept that he has become one.

tryphon

  • Hero Member
  • *****
  • Posts: 722
    • View Profile
Re: Making hacking easier
« Reply #18 on: April 09, 2016, 03:45:31 pm »
One thing I'd like to get would be a FREE IDA, a little less complex and rom-hacking oriented.

BTW, do anyone know which algorithm IDA uses to draw its tree view ?

STARWIN

  • Sr. Member
  • ****
  • Posts: 449
    • View Profile
Re: Making hacking easier
« Reply #19 on: April 09, 2016, 04:09:33 pm »
Well the IDA version 5 seems to be freeware at their site. No idea what functionality it has as I haven't tried it.

What is this tree view exactly and what makes it so useful?