ROM Hacking Tools - Room for Improvement?

[thelynchieconjecture]

Hi Guys,

I am a ROM hacker of old, which is to say when I was younger and had more time on my hands I used to bang my head against ROMs in an effort to learn how to hack them.

As I got older I feel out of it, as work took over my life as it does in the late-20s, but now I am looking to get back in to it as a way to relax and have some fun.

The good news is that I am a much more experienced and better programmer now, the bad news is that as far as I can tell, ROM hacking tools don't seem to have really changed in the last 15 years.

I was thinking about writing a NES emulator in C# but with a particular focus on ROM hacking. I don't want it to be the best NES emulator (I am not that cocky - another advantage of age), but I want it to have some cool debugging tools that I would have died for 15 years ago, and am still wanting today.

I thought I would join this community, as it seems to be one of the more active old-school ROM hacking communities both to ask for ideas for my project, and to ask whether I am wasting my time (e.g. are there tools out there that I have simply missed).

Some tools I was thinking of adding:

• Jump/Branch Tracer - Hit a button to toggle the debugger recording the addresses of any jumps or branches that get triggered. Should make it easier to figure out where code / sub-routines relating to actions are.
• Write Tracer - Basically a memory map / dump, but with any addresses/values that were written during the tracing period clearly highlighted.
• Temporary/Debug Patches - Write a patch and store it in "dynamic memory" (fake memory) that you can jump to in place of existing code. Allowing you to test ideas on the fly.

Of course in addition to these there would be more standard (though still surprisingly absent in many debuggers) features like break points on reader/write/PC and on the fly machine code-ASM translation.

[tryphon]

There seems to be a lot of good emulator for NES, with advanced hacking features. Why don't you aim to a less popular console ? For example, Megadrive (although some good emulators exist). I don't know about the PC Engine.

Later consoles such as Saturn or Dreamcast sure need some work too, but I'm not sure it responds to the criteria of 'a way to relax and have some fun'

[thelynchieconjecture]

Definitely thought about other consoles; though NES is what I know best (and love), so I figured it would be a good starting point. I figure if I can built a good set of tools, they will largely be transferable - will just need to replace the underlying core

[FAST6191]

You are supposed to become more career focused as time goes on? I think I am doing it wrong.

In case you missed it fceux is considered the premier NES debugger, and generally the gold standard* in hacker grade/focused emulators as far as debugging tools go. http://www.fceux.com/web/help/fceux.html for more on what it has.

*I am not sure if I want to say reference standard but I would probably uses its features as a reference.

Jump/branch logging. Can't say I have really considered that one before, though a distribution of jump locations/ranges could be useful.

Write logging. I did actually see a modified version of desmume (DS emulator) do something like this -- it was aimed at being a free space finder (DS carts are not memory mapped at all so free system memory is very useful to know about). Equally I do not imagine it would be terribly hard to fiddle with the cheat search/RAM watch functions to do something like this.

Mind you fceux does feature full logging ( http://www.fceux.com/web/help/fceux.html?TraceLogger.html ), no doubt you could then take the output from that and have some fun with grep, awk or regular expressions to do much of what you want.

I am not really sure what I would term that as, though I am not sure if you are asking more for scratch space/virtual memory or the ability to dump any fiddlings you made in the binary/memory to a file to load back up next session a la what many do with IDA. However I reckon between Lua support (fceux and desmume out of the box, many others if you take a wander through speedrun forums) and a bit of end user thought you can probably already do that. Of course that does risk being something like http://xkcd.com/763/ . In case you missed lua support and an example of what it can do then you might wish to have a snout through http://www.romhacking.net/forum/index.php/topic,18717.msg266807.html#msg266807

There is always room for improvement and anybody cansurprise us all, and it happens all the time, but I would have probably picked basically any other system, save perhaps the PC, to improve the debugging tools of, many of which are actually crying out for a decent debugger, especially if you are going to open source it. Though if the NES has the games you want to hack, the assembly you know (or is filtering down from dim and distant memory) and the like then by all means welcome back to the fold.

[thelynchieconjecture]

I thought I had seen FCEUX, though clicking through to the provided link made me realize I had downloaded an old version.

I will have to check out the new version to see if it can do what I was thinking.

Mostly my thoughts were centered around discovery, rather than debugging or modifying a well mapped ROM.

I remember way back when, I wanted to trigger some code when Mario mounted Yoshi in SMW for SNES. It was a fairly well mapped game, but no one had identified (or at least identified and shared) the addresses of code that executed when this event happened, so I had to figure it out.

Basically I used a tracer to trace everything that happened as Mario mounted Yoshi. I then poured through the trace to identify all the jumps/branches, and added a break point on each. Painstakingly removing them one-by-one until I found the one I needed.

I always thought it would be cool if you could just hit a toggle (like you do with a tracer), but instead of tracing everything, it just recorded the addresses of jumps and branches. Even cooler if you could then toggle it to a different mode to remove any re-encountered addresses.

So:

Toggle to Record Mode: Jump on Yoshi.
Toggle to Trim Mode: Idle, run around, jump, do a bunch of common stuff but don't mount Yoshi.

By the end of Record Mode you would have a ton of potential break points. But by the end of trim mode, you would have a much smaller list, because all the common jumps encountered during idle and just the act of jumping would have been trimmed.

While I am mostly doing this as an exercise for myself (and potentially as an educational tool for others), I wouldn't mind drumming up some level of interest in the project to get new ideas and what not; so perhaps I should consider doing a different console if NES is well and truly covered.

[Nightcrawler]

Toggle to Record Mode: Jump on Yoshi.
Toggle to Trim Mode: Idle, run around, jump, do a bunch of common stuff but don't mount Yoshi.

You can already do this with tracing in Geiger's SNES9x debugger, except the opposite of what you said. You'd start tracing and let it run while you do everything BUT jump on Yoshi. Then hit a button to trace again, and this time you will only get newly executed code ideally related only to jumping on Yoshi. I would imagine NES debuggers can also do this, but I haven't done NES hacking in a long time to say which.

I think things are a bit farther along than you think they are.

[thelynchieconjecture]

You can already do this with tracing in Geiger's SNES9x debugger, except the opposite of what you said. You'd start tracing and let it run while you do everything BUT jump on Yoshi. Then hit a button to trace again, and this time you will only get newly executed code ideally related only to jumping on Yoshi. I would imagine NES debuggers can also do this, but I haven't done NES hacking in a long time to say which.

I think things are a bit farther along than you think they are.

I was playing around with the latest version of FCEUX (turns out the version I had was quite old indeed - from 2007), and their trace logger is a lot better now, and includes an 'executed for first time option' which can be used to do what you described.

So I suspect you are correct, and that things are a lot more advanced now, with my primary problem being out of dated resources (for example I downloaded FCE from a website I used to frequent 12-14 years ago, but it is obviously a dormant website now).

For example: I set my self the goal of finding the entry point for when a character jumps in Super Mario Brother's 2 (without looking it up, though I am sure in a game that popular it is already mapped), and using the trace logger in FCE I was able to find it in about 20 seconds (including the learning curve of using the tool).

FYI in SMB2 USA Rev.A the entry point is $8C99.

Anyway, thanks for your help all! Going to keep playing around tomorrow (yay for holidays, as brief as they may be).

[STARWIN]

I was thinking about writing a NES emulator in C#

But.. but that is an interpreted language, right? Will it scare people away if they need mysterious runtime components?

Jump/Branch Tracer - Hit a button to toggle the debugger recording the addresses of any jumps or branches that get triggered. Should make it easier to figure out where code / sub-routines relating to actions are.

FCEUX's trace log with part of instructions trimmed away.

A slightly similar idea would be having logging points in the code, that log that particular instruction and state each time it gets executed.. or a logging point in memory that logs all reads and/or writes to/from that location (including memory mapped I/O, ROM, RAM).

Write Tracer - Basically a memory map / dump, but with any addresses/values that were written during the tracing period clearly highlighted.

FCEUX should be able to log code/data/mixed as far as ROM goes.. that leaves us RAM and memory mapped I/O. I think I haven't noticed this kind of logging. So, I can interpret your suggestion in two different ways: Either a memory map that highlights written (or read!) RAM locations, or a log of all instructions that accessed memory (the point is that one location can be accessed multiple times in different ways within the tracing period.. so if we want to reveal this information, logging the instructions might be useful extra). Getting closer to the earlier point, but from a different approach.

Temporary/Debug Patches - Write a patch and store it in "dynamic memory" (fake memory) that you can jump to in place of existing code. Allowing you to test ideas on the fly.

I believe various emulators allow you to modify RAM/ROM temporarily manually. Testing larger patches can be done traditionally via multiple files.

Definitely thought about other consoles; though NES is what I know best (and love), so I figured it would be a good starting point. I figure if I can built a good set of tools, they will largely be transferable - will just need to replace the underlying core

This kind of modularity would be interesting.. not sure what kinds of issues there will be. Of course your point can be reversed - if things are modular, you can start with something else than NES and later expand to it. Or just have two cores from the start.

Even cooler if you could then toggle it to a different mode to remove any re-encountered addresses.

I agree that a pattern of automatic collection and then automatic trimming seems very attractive.. probably in various cases. When I look for an attack vector into a game engine, I often start by identifying a relevant RAM byte. I have done it manually by multiple RAM dumps (stateNdumpM) and then comparing byte equivalence/non-equivalence via vbindiff .. based on some of FAST6191's posts, it seems that I should be looking for "cheat search" or something similar - just haven't noticed such a button yet. The point is that automatic collection+trimming would be/is a rather convenient way to find an attack vector like this.

I think the following list would include the "best" debuggers for a given system, but others can of course correct me:

NES - FCEUX
SNES - Geiger's snes9x (though at least the emulation part in .. bsnes v065d is less flawed - bsnes/higan codebase/clone might be ideal for a SNES debugger)
PS1 - NO$PSX

If you go over their functionality, that should give a good idea of what current debuggers do or don't do.

[thelynchieconjecture]

FCE for NES definately seems to tick most of my boxes now I am using version 2.2.2 instead of 0.98

I wouldn't say it is the most elegant of tools, but is definitely powerful. I have been able to find entry points for nearly every common event in SMB2 without having to look at an existing ROM/RAM map once.

I was thinking about writing a NES emulator in C#

But.. but that is an interpreted language, right? Will it scare people away if they need mysterious runtime components?

If I did push forward with writing an emulator, a large goal would be for the final result to be an educational resource.

Perhaps projects exist now already (once again, today is my first day getting back in to the community), but historically there haven't been many good clean educational emulator projects. They tend to be written by hackers, for hackers (if for anyone at all), from a code point of view at least.

I feel C# is good language to use for education projects, both because it is wide spread (commercially speaking), powerfully enough to do the job, and with a free to use and clean IDE (VS Express).

Javascript + HTML5 would also be a good option, but I am not sure the performance is quite there (not for an emulator with debugging at least).

Java would be another potential, but Java and C# are pretty much different sides of the same coin, and I would also pick C# given the choice, just because I feel the IDE is better (personal preference I guess).

C/C++ I would rule out simply because it is no longer a beginner language (nor is it a "hip" one).

[Rhys]

C# isn't really interpreted, it's dynamically recompiled - same as Java. It tends to be pretty performant these days, nothing compared to C++/C but it's a lot easier to work with

I think it's a wise choice if this is more of an educational project

[tryphon]

Mqtter of taste. I'd choose Python.

[BlackDog61]

I think the following list would include the "best" debuggers for a given system, but others can of course correct me:

NES - FCEUX
SNES - Geiger's snes9x (though at least the emulation part in .. bsnes v065d is less flawed - bsnes/higan codebase/clone might be ideal for a SNES debugger)
PS1 - NO$PSX

I think ppsspp deserves a good mention here. 'been using it for some time and it's very comfortable debugger-wise (it has memory breakpoints ). It might be improved as far as traces go, but that's all I can think of. Oh, and it's open source, regularly updated (I proposed a patch which got accepted & merged in less than an hour), so I'm sure you'll be met with open arms if you decide to contribute to its tracing capabilities.

[Gideon Zhi]

Geiger's SNES9X isn't... well, very good. It has a lot of problems - breakpoints not triggering when savestates are loaded, poor keyboard support in its interface, poor support for debugging VRAM, no visual memory viewers or tilemap debugging support, no conditional breakpoints, "squelch" removes useful information from the tracelog (stack pointer) while having it disabled includes much less useful stuff like VC and FL (wtf is FL?), the debug console shrinks by a few pixels horizontally and vertically every time you restart the emulator... If FCEUX's debugger is practical, useful, and surgical, Geiger's is like trying to hack with a stone axe. It gets the job done, barely, but it doesn't make it easy.

I'd also like to see a decent replacement for TLP, especially with support for non-standard tile layouts. Tile Molester uses Swing for its interface (oh god why) and CrystalTile2 is... I don't even know, its UI fails basic usability guidelines and I've never actually managed to do anything with it.

[FAST6191]

Crystaltile2 is one of those "kind to its friends, but is awfully selective about those it makes friends with" type programs, however it does not even have the benefit of being something as truly potent as other examples of that concept (Blender for instance).

If you are after a generic but highly configurable viewer at least then I highly suggest http://code.google.com/p/tiledggd/
The lack of any real editing abilities will probably drag it down for you though.

Also if we are being crazy then all those javascript, webgl and the like emulators might be good to see some basic debuggers attempted.