[Bregalad]

What can we do to make the process of hacking easier and faster?

Hacking games that use generic format for whathever they use is easier. For instance, many GBA shares a generic music engine, and as such hacking music or sound in GBA games is very easy - or in other words after having gotten the principle, doing it on other games is exactly the same.

Hacking is easier when whathever you're working on has already been documented. It's also easier if you know which RAM and ROM locations is used and which is left unused for your own usage.

Finally, hacking is a lot easier when games leaves lots of ROM space free available. Changing things without expending is hard and very constraining.

I hope this helped.

[KC]

A PC history is not practical for many systems due to the massive performance costs. For recompiled code it could cut the speed into a fraction of the original speed, even if it's encoded directly into the code.

And I'm probably biased, but I feel PPSSPP's and PCSX2's debuggers do a pretty goob job improving the user experience.

[zonk47]

If I do automatic trans, I'll probably do it on my own. I have a lot of specialized expertise for that kind of thing that I don't expect you to have. Right now I'm just trying to improve feedback in emulator debuggers, to make it easier to follow what's going on.

[Bahamut ZERO]

I imagine if YYCHR could load more types of savestates (like GBA, GBC, NES, Genesis), then a fair number of people would find at least the graphical aspects of hacking easier. 

[Klarth]

A PC history is not practical for many systems due to the massive performance costs. For recompiled code it could cut the speed into a fraction of the original speed, even if it's encoded directly into the code.

To expand on this for those interested. Systems with higher clock speeds typically use dynamic recompilation to improve performance. Tracing debuggers (like PCSXTrace) will force you to use the static interpreter which reduces performance. For code tracers on the PSX with a 33MHz clock speed, this can be done by formatting the instruction, register values, etc via a sprintf call 33 million times a second. This is actually viable if you only need a few frames worth of data as your FPS will be < 5. You can achieve realtime operation by allowing the user to specify narrow code ranges to trace or by logging the execution of each instruction only once.

When you move up to PS2 at ~300MHz, these restrictions become even more important. An unfiltered, full trace 300 million times a second will now take many seconds to process a frame of execution. Thus techniques to limit the amount of code being traced become even more important. If you need instructions to be logged multiple times but don't know the range, tracing in two steps is required. The first "silent" trace will mark common instructions to not be logged. A second trace will then log the code of interest. This technique relies on the 90/10 rule where 90% of the time, only 10% of the code is running. Performance-oriented logging measures also become more important. Instead of text-formatting every instruction, execution can be faster by maintaining binary data structures that are text-formatted as a post-processing step.

The big takeaway for those who do not program utilities is that tracing becomes less useful for code isolation in higher clock rate systems due to performance and the massive logs created (expect many GB as compared to several MB for a SNES trace). Debugging via breakpoints becomes a necessary skill to isolate the code of interest. After the relevant code is located, understanding the code can be done through the debugger, a restricted trace log, or a disassembly via IDA Pro.

[zonk47]

I'd think that slowdown during a log session/disassembly is actually desirable: gives you more ability to get a finer cut of the code.

[elmer]

I must admit... I'm quite taken aback... it looks like nobody here means to help me achieve these goals, so in self-respect to myself I am frankly hesitant to try advancing them any further. Is your pride really so great, that you are unable to make these small sacrifices for the good of the community?

WTF???

I've got to say ... the things that you're saying, and the way that you're saying them aren't exactly likely to endear you to any of the folks who might be in a position to help you.

And I think we need to make these changes ourselves, because I don't think we can necessarily count on emulator authors to do them for us.

If I do automatic trans, I'll probably do it on my own.

At the end of the day ... that's exactly how things get done in the real world.

If you see a need for something specific, then go and implement it yourself.

People are more likely to help out if you can show that you've already put in a lot of effort on your own part to improve things, and that means more than just talking up your own capabilities.

A bunch of emulators are open-source ... so just pick one that you like and start "fixing" things. If you do a good job, then perhaps your "improvements" will get rolled back into the mainstream distribution and everyone will win.

That's what I've done with Mednafen ... although Rypheca has stubbornly refused to take the changes (yet). 

In the meantime ... those changes have dramatically improved my experience in hacking the translations that I've been working on, and have been well-received by the folks that hang-out on the forum where I've posted the binaries (and source code).

May I suggest to you that following a path like that could get you further than just asking folks to gather around the campfire and pledge to contribute their time to creating your own personal utopian ideal of automatic translation tools.

[VicVergil]

I'd think that slowdown during a log session/disassembly is actually desirable: gives you more ability to get a finer cut of the code.

There's something called frame advance and trace logging.
Look it up. Or better yet, try it out. I recommend FCEUX.
I hope you won't just go "no this is unacceptably complicated" like in the topic about copy pasting a font's graphics in a tile editor that prompted you to make this thread (or maybe stuff about how FCEUX is coded in C rather than BASIC and hence it obstructs people from using it somehow) and try to be humble enough to learn how to do something new.
It'd be much more constructive and do you more good to be the changing force rather than waiting out the rules to bend for your convenience.

I imagine if YYCHR could load more types of savestates (like GBA, GBC, NES, Genesis), then a fair number of people would find at least the graphical aspects of hacking easier. 

Since the savestates are a glorified RAM dump plus other register/CPU data and emulator fluff, it's a matter of making something recognizing where the RAM data begins in each savestate for each emulator, and where the palette in said ROM is stored.
I don't get why there's so many palette formats used (act, pal, yy-chr, etc..) but maybe starting with support for all these formats should be a good starting point.

[Spooniest]

It would make it a lot easier if people weren't dysfunctionally rude, but there's little that can be done about that.

I'm not kidding or digging at anyone; this is simply a truism that I think kind of resounds through any creative endeavor.

[zonk47]

Mednafen... isn't that that commandline-only multi-emulator? I tried it briefly then quickly shunted it for Bizhawk. Total pain in the ass. Maybe I'd like your improvements though. One issue I have with a lot of emulators is retrace tearing in window mode. V-sync doesn't help, and neither does triple buffering. Probably has something to do with the 75hz refresh I've set for my monitor, but I'm not about to give that up.

[RyanfaeScotland]

I'd think that slowdown during a log session/disassembly is actually desirable: gives you more ability to get a finer cut of the code.

I had this discussion with Kaneda a few weeks back but interesting to mention here too. Is frame rate that important when tracing? Unless you are doing some time critical code then I'd suspect you'd be stepping through as mentioned anyway so the 2 (trace vs framerate) can be considered mutually exclusive.

[mrrichard999]

Support emulator development and encourage devs to include powerful debugging tools in their emulators. With great tools comes great hacks.

[RyanfaeScotland]

Support emulator development and encourage devs to include powerful debugging tools in their emulators. With great tools comes great hacks.

and great responsibility?

[elmer]

Mednafen... isn't that that commandline-only multi-emulator? I tried it briefly then quickly shunted it for Bizhawk. Total pain in the ass.

Everyone has their own "comfort zone".

I just tried BizHawk for the 1st time, and it's not for me ... too many windows, and the debugger doesn't show all the information that I want. But it does have some nice features, and I can see it being improved into something nicer.

Mednafen's problem (IMHO) is mainly in the debugging display layout and unreadable-fonts used, i.e. basic UI-design issues ... so that was what I "fixed".

Command-line-only doesn't bother me, nor does using hotkeys instead of menus (although it does involve a learning-curve).

Probably has something to do with the 75hz refresh I've set for my monitor, but I'm not about to give that up.

Does it really surprise you when you're displaying emulated-console screens that refreshes at 60Hz with a 75Hz monitor refresh???

If you're using an LCD display, then I just don't understand why you'd particularly want to set a 75Hz monitor refresh ... I guess that I'm missing something.

[zonk47]

It's not LCD... it's CRT I think (later model, small back). At 75hz the image is considerably more crisp. Not all the emulators have this issue so it's definitely a technical issue for the ones that do.

[zhade]

One of the things I don't like about romhacking, as someone who is still kind of new to assembly / hex / binary operations and who probably isnt aware of the best tools out there and all.. is how much I have to repeat the same operations over and over again.. For example, sure I can find the binary value of "A5" and the the result of AND'ing it with "43" all in my head.. but honestly, I think it is quicker for me to use the calculator instead and more importantly, I really don't trust myself enough at doing it and prefer to use the calculator to be sure I don't end up pulling my hair off later if something don't make sense and waste time searching at the wrong places for what went wrong just to find out I made a simple calculation error when I could have just take 10 seconds to make the calculation using the calculator.

There is also the need to use multiple tools which are not "compatible" directly with one-another like for example I could be using geiger's debugger, have it break when some address is read, then when it breaks, since it only shows the current line of code and lets say I need to know what comes before that like of code, I'll have to go look in the disassembly, which means switching to notepad++, open the right bank file, copy the address from geiger's debug console (by double clicking the address from the output box and pressing CTRL+C) then press CTRL+F in notepad to search, paste the address from the debugger, change it a bit so it has the right address format for the disassembly (which could be say, from "$01/C01A"  to "01C01A:"). Then let's say the code makes me think the address read could be some graphics, I could switch to tile layer pro, copy the address I had set as a breakpoint from the debugger's breakpoint window, then use "goto address" (or whatever its called) in tlp, paste it, modify it from D31234 to 131234, find out I was wrong and it seems to be some data that is not graphics after all, switch to an hex editor to use a table to see if it might be text which again requires modifying a bit of the address..

Sure, none of this is "hard" and this thread's title is "Making hacking easier" but I think zonk47 was thinking of stuff like that when he asked "What can we do to make the process of hacking easier and faster?" and since I was actually in the process of making exactly that, things to automatize repetitive tasks, and thought I am surely not the first to have thought of that, I was eager to see what some you guys came up with.. but sadly zonk47's bad attitude towards beign wrong / corrected really seems to have wasted a big part of what could have been a really interesting read and a source of useful infos about tools and tricks to make the whole process less tedious which less experienced romhackers (or anyone into really) could have benefited from.

So, hoping the discussion can continue on the right track, here is what I have to propose as a way to save time, avoid errors and cope with multiple programs with picky address formats and the like:

I came across that beautiful thing called "Auto Hot Key", it has been around for like.. a decade maybe ? But I just never heard about it anywhere and since nobody talked about it here I just had to spread the word in case some of you guys didnt know about it.
Its an app that runs ".ahk" scripts which can do many useful things but what I use it for mostly is to make repetitive tasks automatic. With it you can assign "hotkeys" that will basically change the behavior of some keyboard/mouse/joystick inputs to something else.
For example, the first script I made was for visual studio (you can make hotkeys work only with some programs) I didnt like to have to right click on function calls and select "go to declaration" to navigate to the function's declaration, because even tho it takes only 2-3 seconds, Its 2-3 seconds wasted every time and I use this all the time.. Sure I could just go into visual studio's settings and set some key shortcut to make it faster like CTRL+ALT+Q or whatever, which is what I did at first, but since I was used to do it with the mouse, it felt wrong to use the keyboard shortcut and I had to left-click on the function first to put the carret on it for it to work anyway so I still used the right-click menu, and there is no way to set hotkeys to the mouse in visual studio..

Then I found out about autoHotKey, and right away, made a small script that basically just makes it so when I press the middle mouse button, instead it does as if I pressed the left mouse button followed by CTRL+ALT+Q. so now its done in a single click and im a happy bunny.

I got so used to it that I caught myself middle-mouse-clicking on addresses following jumps/branches when looking at disassembled snes asm.. so I thought.. why not make it actually work ? Easy, make the middle mouse button act as "LEFTCLICK / LEFTCLICK / CTRL+F / END / : / ENTER / ESCAPE" which selects the word under the mouse with the 2mouse clicks, search for it, add ":" to the end of the search string (because each disassembled line starts with the address as such C0/0000: ) then start the search with ENTER and close the search box with ESC. and voila, all it takes is a single click now.

That alone isnt very impressive (or so much useful for romhacking) but what makes me talk about this program is the way it makes it very easy to control the behaviour of applications and the way it is possible to enhance any debugger/tool with it. You can easily retreive the text from a chosen control from most applications as well as set it so for example I made a script that gets whats in geiger's debugger's console, takes the address of the last line, open the coresponding bank file in notepad++, then jump to it and highlight it, which makes it so I can follow the code execution using step into/over directly from the disassembly.

The best thing about it all is that it wouldn't be hard to adapt the script to make it work with bsnes+ as well and make the disassembly open into another text editor than notepad++ or make a middle-clicked address open in tile layer pro at the right offset.

it shouldnt be too hard to make a script that would detect whats under the cursor and if it is an hex value, show a tooltip with the decimal and/or binary value of it..

So even if a debugger is not open-source and its development stopped, there is still some things that can be done to make it work like you would like it to and then you can even reuse that feature you added to another debugger later so in a way its even better than having access to the source code !

oh and the scripts can be compiled to .exe so you can share it with the world afterwards... ok.. enough.. Sorry for the huge post I could talk about this all day XD

[STARWIN]

Geiger's does have a disassembly button that you can use to look at the surrounding code. It can also trace. I suppose having a disassembly open and searching it is roughly equally simple though. I'm not seeing any optimizations specific for that.

Overall going backwards in time has so far been easy for me. Having a save state slightly before the moment of interest, and then setting breakpoints earlier in the routine or doing a step out and setting a break before the call is basically it. The only difficult case would be spaghetti where the execution goes all over the place in a large routine (or lack of routine structure..), but haven't seen such yet.

[zonk47]

My motive for editing vb8086 is to offer a capable debugging emulator for the PC-98 in English. Which is why I'm rather surprised that the only help being offered is advice (which really isn't helpful at all). What I need is division of the work. I think we can all agree that most debugging emulators do not offer enough information/feedback as to the immediate processes (or when they do, they only offer this information in terms of a single instruction at a time). That is what's really needed to make the process of hacking faster and smoother. I think the real reason you guys don't want to help is because you don't want it to be easy, because you enjoy the challenge of pushing your ability and wrestling with the experience of your own limits (and, I assume, it makes you feel good to see "lesser" minds fail). Coming from a cog sci background, I can assure you that's all gravy: if you compete better with another in terms of managing figures in a running formula/algorithm, its because your brain simply has more "buffer space" and therefore, more short term data retention (you might say, your CPU has more registers . But one brain design doesn't make one person better or superior to another.

@zhade: there is a difference between circumstantial coping and actual evolution. An evolution of tools would make the art of hacking more appealing and accessible... coping techniques like hot key settings really just make the process more obtuse to the outsider, who has to be briefed on yet more cryptic, arcane terminology and method (and as a result, is more likely to want someone else to teach them. However they will be unwilling/unable to pay for this tuition (if it is even offered) and no one will pay it for them, so they will never get the education). Nonetheless I think you did drive home the point that making hacking "easier" means making it faster to do. But again, I don't think the community really wants that because it would make hacking seem more like work (remember these are professional programmers who are used to having everything spelled out for them and bored of it) than a game.

[Bahamut ZERO]

I'm rather surprised that the only help being offered is advice (which really isn't helpful at all). What I need is division of the work.

I'm surprised you're still half expecting people to do half the work for you while shooting down the advice those same people offer in one fell swoop.

You oughta be glad they're even giving you advice with the way you  keep responding like a dildo.

Also, I think Zhade had some good, valid advice on making things a bit less tedious through outside means of the debuggers themselves. Saying that people are going to need paid tutors or higher education (or whatever the hell it was you were getting at) to set up some hotkeys is retarded. You essentially told him "fuck your advice, no one wants to use hotkeys WHY'S NO ONE BUILT HALF A DEBUGGER FOR ME" but with a paragraph instead of a sentence.

[Klarth]

My motive for editing vb8086 is to offer a capable debugging emulator for the PC-98 in English.

Use currently existing debuggers to see what features you like. Then code it. Nobody is going to do it for you. Imagine if your neighbor asked you to help him paint a fence and then sits down while you paint 90% of it. You're wanting to be the lazy neighbor.

I think the real reason you guys don't want to help is because you don't want it to be easy, because you enjoy the challenge of pushing your ability and wrestling with the experience of your own limits (and, I assume, it makes you feel good to see "lesser" minds fail). Coming from a cog sci background, I can assure you that's all gravy: if you compete better with another in terms of managing figures in a running formula/algorithm, its because your brain simply has more "buffer space" and therefore, more short term data retention (you might say, your CPU has more registers .

No, this is because the work is long and difficult, especially if you are a working in an area with poor documentation. Your lack of experience and drive to learn/experiment shows in this series of posts and not many people would want to work "with" you. Many knowledgeable people, myself included, have not done a good job in improving documentation to reduce the learning curve. At the end of the day, it is a hobby and writing documentation is not a goal for most people. I have a half-dozen other non-romhacking related hobby projects I could be working on instead of writing documentation.

What I need is division of the work.

Divide the work among yourself and then do it. Then let yourself or the community evaluate whether it's worthwhile.

But again, I don't think the community really wants that because it would make hacking seem more like work (remember these are professional programmers who are used to having everything spelled out for them and bored of it) than a game.

You have a poor concept of what professional programming is about. Most of us are either hobby-level programmers or were involved with romhacking well before getting a professional programming job.