I was thinking about writing a NES emulator in C#
But.. but that is an interpreted language, right? Will it scare people away if they need mysterious runtime components?
Jump/Branch Tracer - Hit a button to toggle the debugger recording the addresses of any jumps or branches that get triggered. Should make it easier to figure out where code / sub-routines relating to actions are.
FCEUX's trace log with part of instructions trimmed away.
A slightly similar idea would be having logging points in the code, that log that particular instruction and state each time it gets executed.. or a logging point in memory that logs all reads and/or writes to/from that location (including memory mapped I/O, ROM, RAM).
Write Tracer - Basically a memory map / dump, but with any addresses/values that were written during the tracing period clearly highlighted.
FCEUX should be able to log code/data/mixed as far as ROM goes.. that leaves us RAM and memory mapped I/O. I think I haven't noticed this kind of logging. So, I can interpret your suggestion in two different ways: Either a memory map that highlights written (or read!) RAM locations, or a log of all instructions that accessed memory (the point is that one location can be accessed multiple times in different ways within the tracing period.. so if we want to reveal this information, logging the instructions might be useful extra). Getting closer to the earlier point, but from a different approach.
Temporary/Debug Patches - Write a patch and store it in "dynamic memory" (fake memory) that you can jump to in place of existing code. Allowing you to test ideas on the fly.
I believe various emulators allow you to modify RAM/ROM temporarily manually. Testing larger patches can be done traditionally via multiple files.
Definitely thought about other consoles; though NES is what I know best (and love), so I figured it would be a good starting point. I figure if I can built a good set of tools, they will largely be transferable - will just need to replace the underlying core
This kind of modularity would be interesting.. not sure what kinds of issues there will be. Of course your point can be reversed - if things are modular, you can start with something else than NES and later expand to it. Or just have two cores from the start.
Even cooler if you could then toggle it to a different mode to remove any re-encountered addresses.
I agree that a pattern of automatic collection and then automatic trimming seems very attractive.. probably in various cases. When I look for an attack vector into a game engine, I often start by identifying a relevant RAM byte. I have done it manually by multiple RAM dumps (stateNdumpM) and then comparing byte equivalence/non-equivalence via vbindiff .. based on some of FAST6191's posts, it seems that I should be looking for "cheat search" or something similar - just haven't noticed such a button yet. The point is that automatic collection+trimming would be/is a rather convenient way to find an attack vector like this.
I think the following list would include the "best" debuggers for a given system, but others can of course correct me:
NES - FCEUX
SNES - Geiger's snes9x (though at least the emulation part in .. bsnes v065d is less flawed - bsnes/higan codebase/clone might be ideal for a SNES debugger)
PS1 - NO$PSX
If you go over their functionality, that should give a good idea of what current debuggers do or don't do.