Romhacking.net

Romhacking => Newcomer's Board => Topic started by: AnderHershey on September 01, 2015, 06:01:49 pm

Title: Branching in Disassembled Code
Post by: AnderHershey on September 01, 2015, 06:01:49 pm
I have looking into this type of thing since middle school, but it always seemed so far away.
Yesterday I decided to just go for it, and I eventually got everything I needed to begin working.
I started reverse-engineering some of the code, and eventually I got to a weird situation:
Basically, they use branches, e.g. BNE $03 or alike.
So I take the hex location of BNE.
e.g. 1:000Bh
I add 03h to it:
1:000Bh + 03h = 1:000E
And then I look at that location as where the branch will continue if taken.
So if at 1:000E is A5, and then at 1:000F is 00 I know that's LDA $00.
So my question is this, assuming that I'm correctly adding these indexes, why would it be that a branch leads into the middle of another operation?
e.g. suppose you have C4 F3 D4 or some arbitrary jump instruction.
The branch goes to F3 as opposed to C4 or the location right after D4.
That is to say, the branch leads right into the middle of another operation, not before nor after.
I don't know how the system would have the intelligence to know it's in the middle of an operation, and thus skip to right after D4, but that has been my guess as what these branches would do.
The game runs so this must work somehow (or maybe it's wrong and has some implication I don't know about).

I'd be grateful for any information on whether the way I'm adding the branch offsets is wrong, or how the snes would handle a branch into the middle of a different operation.

(If it were to handle it as just an operation, this will do a whole different set of ASM since all parts of every instruction will be off by 1 Even if the change was limited to 1 operation if a TYA corrects the offset, I still don't
think one would be allowed to do this when actually creating the ASM, i.e. when coding it one wouldn't be able to code it such that it branches into an argument of a different operation, and since I doubt the game was coded in pure hex, I don't think the intent was to run a different set of operations based on jumping to  the middle of another).

As an addition, even after I decode everything and have a vague idea as to how the code is broken up, is the way I figure out the intent of each block by testing? Or is there a way to know that "Ok, this is to load something from this location, and I know at this location is a character sprite, so this loads the character sprite." Basically, can everything be done without having to do a lot of testing and just parsing code?

Lastly, BRA $00 does nothing right?
Title: Re: Branching in Disassembled Code
Post by: KingMike on September 01, 2015, 06:13:00 pm
At least in 6502, count the first byte of the NEXT instruction as a +0 branch. Then do the math from there. The instruction is limited to one byte, so it can only branch a maximum of +127 (0x7F bytes forward) and -128 (0x80 bytes backwards)
On 65816 there is a BRL instruction that extends the range to 2 bytes but otherwise the same (+32767 to -32768)

I'm sure the CPU is not smart enough to know if it's branching into the middle of an instruction. If it does branch there, it will just take whatever data is there and turn that into an instruction.

Yes, any branch instruction (not just BRA) with a distance of 0 will effectively do nothing, because 0 is the relative offset of the next instruction.
Title: Re: Branching in Disassembled Code
Post by: Dr. Floppy on September 01, 2015, 08:30:47 pm
I'm sure the CPU is not smart enough to know if it's branching into the middle of an instruction. If it does branch there, it will just take whatever data is there and turn that into an instruction.
Bingo.

Moreover, I've seen instances of such "dirty" coding in published NES games. Probably the most common variation is branching to the nearest #$60 byte, thus double-casting an operand as an RTS command. (And when some rookie hacker decides that Captain Awesome should have more than 96 bullets, start out somewhere other than slightly left-of-center, etc., the game crashes and needless discouragement ensues.)
Title: Re: Branching in Disassembled Code
Post by: STARWIN on September 01, 2015, 08:54:24 pm
I think the processor adds the size of the just executed instruction to the program counter after each instruction. In case of a branch instruction, the instruction itself also adds/substracts from the PC, independently. The first byte in each instruction is the opcode, which tells the processor how many bytes long it will be.

On decoding the intent.. yes, to stay sane, a chain of causality is necessary IMO, and you need some anchoring points like that. The only other tip I can think of is to keep in mind that functions (subroutines) often do a certain task, and a good pace for understanding is one function at a time. You can also check if stepping over (executing) a certain JSR call in a debugger does something that you can understand, instead of trying to read all the code. They can change game state, send output to screen or take in arguments in registers/memory locations and return a calculation result in others.
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 01, 2015, 09:02:52 pm
@King Mike (and I guess also @STARWIN who just commented):
So, if 65816 follows the same convention as 6502, then for:
2:9B0C (which is D0) and 2:9B0D (which is 26)
when I calculate BNE $26 I would start my counting at 26?
i.e. 2:9B0D + 26 as opposed to 2:9B0C + 26?
This being because the $26 is automatically added to the PC once it reads that it's a branch instruction?

On a side note to that, I've seen things where the only possible interpretation is AND #$0004, and thus this takes up 3 bytes of memory. Upon reading the 65816 reference (superfamicom.org) I saw it state that that operation is only 2 bytes. Is this part of the disassembly up to the reverse-engineer in figuring out the intent of the code?

@Dr. Floppy ("seen instances") and @KingMike(sure...CPU not smart...):
Since you claim you've seen such  code and the statement about it not being intelligent, then I guess I'm supposed to assume that if:
2:9B0C is D0 and 2:9B0D is 26. i.e.  BNE $26.
then 2:9B32 which is 57 (between 4C and 9B, i.e. 4C 57 9B).
will be interpreted as EOR [$9B], Y (even though had the branch not been taken this would wind up as JMP $9B57)? [For this example question assume I had offset counted correctly until I get clarification for the first question where I ask about my offset counting].


As a last addendum, @STARWIN:
I try to do the stepping through code stuff, but it gets annoying since there is a wait state I guess which gives me (especially in a trace log) a ton of BIT/BNE operations. Thus my desire is just to stick with the code itself which is constant in size and only look to the game when I have to (e.g. I notice in the code something is done with a RAM address E7B020 so I look at that and figure out what it's for).
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 01, 2015, 09:37:25 pm
@King Mike (and I guess also @STARWIN who just commented):
So, if 65816 follows the same convention as 6502, then for:
2:9B0C (which is D0) and 2:9B0D (which is 26)
when I calculate BNE $26 I would start my counting at 26?
i.e. 2:9B0D + 26 as opposed to 2:9B0C + 26?
This being because the $26 is automatically added to the PC once it reads that it's a branch instruction?

No, you'd start counting at 2:9B0E, the start of the next instruction after the branch instruction.

Quote
On a side note to that, I've seen things where the only possible interpretation is AND #$0004, and thus this takes up 3 bytes of memory. Upon reading the 65816 reference (superfamicom.org) I saw it state that that operation is only 2 bytes. Is this part of the disassembly up to the reverse-engineer in figuring out the intent of the code?

This is something specific to the 65816 and very annoying for disassemblers (and programmers) to deal with: the length of immediate operands depends on the state of the CPU when it executes that instruction. If the M flag is set, immediate operands are 1 byte (the entire instruction is 2 bytes). If the M flag is clear, immediate operands are 2 bytes (the entire instruction is 3 bytes). Except for instructions that affect the index registers (LDX, LDY, CPX, CPY), where it's the X flag that matters instead of the M flag.

The M and X flags are changed by the REP and SEP instructions:

REP #$10 clears X to 0
REP #$20 clears M to 0
REP #$30 clears X and M to 0
SEP #$10 sets X to 1
SEP #$20 sets M to 1
SEP #$30 sets X and M to 1

Instructions that pull the flags register off the stack (PLP, RTI) naturally also change the M and X flags.

The upshot is that you can't disassemble 65816 code without at least partially simulating execution of that code, to keep track of changes to the flags. Most SNES software leaves the X flag as 0 more or less all the time, but the M flag flips back and forth constantly because the software has to work with both byte-wide and word-wide data. Usually routines that change the flags will restore them to a common state before returning via RTS, but not always.

The way 65816 disassemblers such as Neill Corlett's hax65816 deal with this mess is you tell the disassembler what state the flags should be in at the start of each routine. The disassembler simulates flag changes caused by REP and SEP instructions, and every time it disassembles a RTS, RTL or RTI instruction it resets the flags to the specified defaults. It's not error-proof but works fairly well for most software that isn't deliberately obfuscated.

I strongly recommend that you obtain and use a disassembler such as hax65816 rather than trying to disassemble by hand (though you should certainly be aware of the issues that disassemblers have to deal with, so that you understand what's gone wrong when the disassembler's output doesn't make sense)
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 01, 2015, 09:56:30 pm
The feeling when you write out a long response, and your login time times out.

Redo:
I have two issues with using a disasssembler, one is that it may misinterpret some of it, two is that, since it's ASM and everything is branched everywhere (and there could be overlapping code), I would get confused quickly.

My present method is to find something via the debugger, and then go from there in the 010 Editor.

edit addition: To note this method results in a nice hierarchy as well as a very good order in regards to branches, and makes everything into a "method".

I'd also like to ask about the Absolute Index Indirect operation, in that how on earth is one supposed to figure out where that is in memory? From bisqwit's page it states it's equvilant to:
ram[ramlong[db:$addr: x]] but then this is not only dependent on register X, but also on what is stored at this location.
Is the only way to figure out where this is is by running the game?


Lastly:
So If I start counting at 2:9B0E then considering:
2:9B0C (which is D0) and 2:9B0D (which is 26)
We take the start of the next location (as stated 2:9B0E)
and then do:
2:9B0E + 26 = 29B34

So the place I start doing operations at is 29B34? (If this is true this fixes the ugly code error where there may have been overlap).
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 01, 2015, 10:24:52 pm
The feeling when you write out a long response, and your login time times out.

Redo:
I have two issues with using a disasssembler, one is that it may misinterpret some of it, two is that, since it's ASM and everything is branched everywhere (and there could be overlapping code), I would get confused quickly.

You don't use a disassembler to disassemble an entire ROM at once. You can disassemble a few dozen instructions at a time, ones that you already know are valid instructions and not data, and paste them into a text file to add comments and notes.

Quote
I'd also like to ask about the Absolute Index Indirect operation, in that how on earth is one supposed to figure out where that is in memory? From bisqwit's page it states it's equvilant to:
ram[ramlong[db:$addr: x]] but then this is not only dependent on register X, but also on what is stored at this location.
Is the only way to figure out where this is is by running the game?

Yes, you're going to have to keep track of data addresses in order to understand any but the tiniest and most trivial snippets of assembly code. In C terms, think of an indirect operand as dereferencing a pointer variable (if you don't understand pointers in C, learn about them or you won't have a hope of understanding assembly language).

Quote
Lastly:
So If I start counting at 2:9B0E then considering:
2:9B0C (which is D0) and 2:9B0D (which is 26)
We take the start of the next location (as stated 2:9B0E)
and then do:
2:9B0E + 26 = 29B34

So the place I start doing operations at is 29B34? (If this is true this fixes the ugly code error where there may have been overlap).

Yes, that is correct.
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 02, 2015, 01:51:01 am
Sorry if this is being rather "hand-holding" of a request, but:
While going through corrections for my branches, I had to change:
82 4E 82, i.e. BRL $824E to go to a different location.
This resulted in going, instead of to an instruction I know, to one I don't.
i.e., previously I had things going:
C5 02 B0 09 85 02 i.e. CMP $02, BCS $09, STA $02.
But now, I instead am heading for 02.
This is the COP instruction which I do not know much about besides its definition. I also am unsure if it can act like the AND instruction (i.e. the constant doesn't necessarily have to be 1 byte).
I know the only way to maintain the structure, if I'm going to land on 02, then COP has to be COP #09B0 as a whole.
If that is allowed, then I'm forced to ask, where this would lead me. It's a co-processor interrupt, so am I to assume this would result in the co-processor doing something, eventually reaching a RTI, and then continuing at 85 02, i.e. STA $02 (where we left off)?
(If COP can't handle a 2 byte constant like AND can, I am at a loss).

(As a note, I have learned ASM before, but I've never tackled anything beyond those superficial Uni. projects. Hopefully as I continue in Comp. Eng. things get more interesting but that seems to lead only into architecture, and CS only leads into more high level language).
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 02, 2015, 02:20:09 am
Sorry if this is being rather "hand-holding" of a request, but:
While going through corrections for my branches, I had to change:
82 4E 82, i.e. BRL $824E to go to a different location.
This resulted in going, instead of to an instruction I know, to one I don't.
i.e., previously I had things going:
C5 02 B0 09 85 02 i.e. CMP $02, BCS $09, STA $02.
But now, I instead am heading for 02.
This is the COP instruction which I do not know much about besides its definition. I also am unsure if it can act like the AND instruction (i.e. the constant doesn't necessarily have to be 1 byte).
I know the only way to maintain the structure, if I'm going to land on 02, then COP has to be COP #09B0 as a whole.
If that is allowed, then I'm forced to ask, where this would lead me. It's a co-processor interrupt, so am I to assume this would result in the co-processor doing something, eventually reaching a RTI, and then continuing at 85 02, i.e. STA $02 (where we left off)?
(If COP can't handle a 2 byte constant like AND can, I am at a loss).

(As a note, I have learned ASM before, but I've never tackled anything beyond those superficial Uni. projects. Hopefully as I continue in Comp. Eng. things get more interesting but that seems to lead only into architecture, and CS only leads into more high level language).

The COP instruction is never, ever used on the SNES. You've miscalculated the branch target or made a mistake somewhere else (possibly with the M/X flags). Remember that both 8-bit and 16-bit branches are signed offsets. If the offset is bigger than $7F (8-bit) or $7FFF (16-bit) then subtract $100 or $10000, respectively, to get the correct target address.

Please, please, just use a disassembler and save yourself a ton of pain...
Title: Re: Branching in Disassembled Code
Post by: Nightcrawler on September 02, 2015, 06:16:06 pm
Please, please, just use a disassembler and save yourself a ton of pain...

Right. Or better yet, use a debugger and step through the specific code in question so you can start to learn exactly what each instruction is doing.
Title: Re: Branching in Disassembled Code
Post by: dougeff on September 02, 2015, 06:30:03 pm
Quote
use a debugger

Agreed.
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 02, 2015, 08:44:58 pm
@AWJ I wont be willing to use a disassembler until I go through it like this at least once. It's a good learning experience. Like a trial  by fire.

@NightCrawler/@dougeff:
I would like to use a debugger to double check things and I considered it for this one instruction, but it's a bit of a circular issues:
Without knowing the code I can't know where it occurs (unless that had been how I discovered it).
And without knowing what it is, I can't know what to trigger to reach that place in the code and get into the loop that would trigger my breakpoint.

Also, I continue to make headway, and, again, I've corrected my branches in regards to the sign issue which (@AWJ) I probably wouldn't have thought about unless I had done this in such an archaic manner, but this itself gave me trouble.
Just by how the code lays out, I know I have a STA, LDA, ASL, TAX, JMP, then since I know the CPU just follows commands and doesn't know better, I know once it returns from the JMP (via an RTS) I will hit E3 80, i.e. SBC 80, then 88, i.e. DEY, 81 1C, i.e. STA_1C_X, and then the issue, I hit 82 4E 82, i.e. BRL 824E.
Thanks to AWJ I know about the sign now, so I take this, 2's complement, and then take the location of the first byte of the next operation in this case it'd be an F position) and subtract.
This results with a direct branch to a COP instruction, which I now know doesn't work.
Consequently I am left with only one thought, and that is that, in that JMP, it doesn't return. But that would seem weird. Why enter a routine if you never return from it? (Since I didn't hit a breakpoint at the start of the code, [i.e. assuming the hex labels of my editor match those I'd input into a breakpoint viewer], I know it isn't some main loop which is hit once and then never goes back).

I also would like to say thanks to all who've commented, especially AWJ for consistent responses.

Upon reflection of this post, I think I'll follow that JMP and see where it leads. If there is an RTS, then my issue remains, if there doesn't, then I ignore it until everything else is worked out.
Also, it may appear I am encountering what Dr. Floppy referred to (although not really since they aren't branches) but rather jumps that just jump to a location which stores 60h, i.e. RTS. (Although I don't see the point in that. Why JMP to a RTS if there aren't even instructions to change the stack pointer. I remember a thread on here about compilers being bad back in the day, but I doubt this is compiler based).

[As I edited AWJ commented so I may be able to remove this part about the RTS. Since JMP doesn't push anything, the RTS will undo the Jump Subroutine before it. I.e. if A calls B, and B jumps to RTS, then RTS doesn't return to B, but rather to A).


Last edit of this  post before I make a new one:
But what if it's a JSR calling an RTS? JSR is jump sub routine, hence RTS should return back to its spot.
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 02, 2015, 09:48:29 pm
@AWJ I wont be willing to use a disassembler until I go through it like this at least once. It's a good learning experience. Like a trial  by fire.

@NightCrawler/@dougeff:
I would like to use a debugger to double check things and I considered it for this one instruction, but it's a bit of a circular issues:
Without knowing the code I can't know where it occurs (unless that had been how I discovered it).
And without knowing what it is, I can't know what to trigger to reach that place in the code and get into the loop that would trigger my breakpoint.

Also, I continue to make headway, and, again, I've corrected my branches in regards to the sign issue which (@AWJ) I probably wouldn't have thought about unless I had done this in such an archaic manner, but this itself gave me trouble.
Just by how the code lays out, I know I have a STA, LDA, ASL, TAX, JMP, then since I know the CPU just follows commands and doesn't know better, I know once it returns from the JMP (via an RTS) I will hit E3 80, i.e. SBC 80, then 88, i.e. DEY, 81 1C, i.e. STA_1C_X, and then the issue, I hit 82 4E 82, i.e. BRL 824E.
Thanks to AWJ I know about the sign now, so I take this, 2's complement, and then take the location of the first byte of the next operation in this case it'd be an F position) and subtract.
This results with a direct break to a COP instruction, which I now know doesn't work.
Consequently I am left with only one thought, and that is that, in that JMP, it doesn't return. But that would seem weird. Why enter a routine if you never return from it? (Since I didn't hit a breakpoint at the start of the code, [i.e. assuming the hex labels of my editor match those I'd input into a breakpoint viewer], I know it isn't some main loop which is hit once and then never goes back).

I also would like to say thanks to all who've commented, especially AWJ for consistent responses.

Upon reflection of this post, I think I'll follow that JMP and see where it leads. If there is an RTS, then my issue remains, if there doesn't, then I ignore it until everything else is worked out.

JMP is not a subroutine call. It's like a goto. It doesn't push a return address onto the stack, so there's no way for the CPU to "return" to the instruction after it. The bytes you're disassembling after the JMP are probably data and not instructions, which is why they doesn't make any sense when disassembled.

83 80 88 81 1C 82 4E 82 looks like a jump table to me. Especially since it's coming after an ASL, TAX, JMP. Is the JMP instruction a JMP (addr, X) by any chance?
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 02, 2015, 09:58:00 pm
(Please see the last edit I made to the previous post).

In regards to your question:
It is absolute index indirect.
I do not know much about the jump (as I explained in a previous response, I don't know how to deal with it and have been told by you to use a disassembler or a debugger).
I do know though if I "guess-te-mate" the BRL to 1 location before or 1 location after I reach a series of instructions which eventually reach a part which is just STA $dp, X for quite a few lines. Like a BRL into a "storing" part of the code before ending that "process".

Also, when I used a debugger I got instructions whose value, when I searched as a whole, could find one occurrence of, but it was off by 3 banks from that in the debugger.
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 02, 2015, 10:25:46 pm
(Please see the last edit I made to the previous post).

In regards to your question:
It is absolute index indirect.
I do not know much about the jump (as I explained in a previous response, I don't know how to deal with it and have been told by you to use a disassembler or a debugger).
I do know though if I "guess-te-mate" the BRL to 1 location before or 1 location after I reach a series of instructions which eventually reach a part which is just STA $dp, X for quite a few lines. Like a BRL into a "storing" part of the code before ending that "process".

Forget the BRL completely. It's data, it never gets executed as an instruction. What you're looking at is the assembly language equivalent of a C switch statement:

Code: [Select]
LDA some_variable
ASL
TAX
JMP (table, X)

table:
routine0
routine1
routine2
...

is equivalent to the C code:

Code: [Select]
switch(some_variable) {
  case 0:
    routine0(); break;
  case 1:
    routine1(); break;
  case 2:
    routine2(); break;
...

The bytes after the JMP are not instructions, they're the addresses of different routines, one of which will be jumped to depending on the value of some_variable. Take each two bytes in the table, flip them, and that's the address of one possible destination for the JMP. So 83 80 88 81 1C 82 4E 82 means

8083
8188
821C
824E

If the memory loaded by the LDA contains a 0, the JMP will jump to $8083. If it contains a 1, it will jump to $8188. Et cetera. (Do you see why the ASL is necessary before the TAX?)
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 02, 2015, 10:54:52 pm
If I assume A is 16 bits wide.
We have:
if it stores an address, it'll be something like $1234, with the 2 bank bytes being relative to the position.
If I shift it left I get: 2340.
This gives room for X to have this vary over some range of values in a table.
But is this is where foresight as a programmer has to come in so that when (the actual programmer) does something like this they know they don't care about the first byte?
i.e. we stored the value $1234 at the location
from which we load A, but we knew we didn't
care about the first byte (the 1). [Or they could just use 00. The programmer here has 00s dispersed in various locations. I guess they're "flags" for a lot of things as well].

Oh, I see, the reason one could claim it has to be 00 is because if one is going to use 80D7 while in line 80D7, of course one would be trying to access the could right there via some sort of process.
What baffles me is why they don't use this process more.
I've encountered a lot of BNE, BEQ, (giant if-else trees basically) and according to the nes dev wiki, this way can be slower (the BNE BEQ way).
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 02, 2015, 11:11:53 pm
If I assume A is 16 bits wide.
We have:
if it stores an address, it'll be something like $1234, with the 2 bank bytes being relative to the position.
If I shift it left I get: 2340.
This gives room for X to have this vary over some range of values in a table.
But is this is where foresight as a programmer has to come in so that when (the actual programmer) does something like this they know they don't care about the first byte?
i.e. we stored the value $1234 at the location
from which we load A, but we knew we didn't
care about the first byte (the 1). [Or they could just use 00. The programmer here has 00s dispersed in various locations. I guess they're "flags" for a lot of things as well].

Oh, I see, the reason one could claim it has to be 00 is because if one is going to use 80D7 while in line 80D7, of course one would be trying to access the could right there via some sort of process.
What baffles me is why they don't use this process more.
I've encountered a lot of BNE, BEQ, (giant if-else trees basically) and according to the nes dev wiki, this way can be slower (the BNE BEQ way).

ASL shifts left by one bit, not one hex digit. If you left shift $1234 you get $2468. It's equivalent to multiplying by two. You do an ASL before indexing into a jump table because each address in the table is two bytes long.
Title: Re: Branching in Disassembled Code
Post by: Zoinkity on September 03, 2015, 04:30:19 pm
Quote
I've encountered a lot of BNE, BEQ, (giant if-else trees basically) and according to the nes dev wiki, this way can be slower (the BNE BEQ way).
Jump tables are faster when there's a large number of conditions, but for a small number of tests it's far more practical to use a branch or two.

In large part this was the compiler's preference.  It was an exception to program in pure ASM.
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 03, 2015, 05:13:12 pm
Jump tables are faster when there's a large number of conditions, but for a small number of tests it's far more practical to use a branch or two.

In large part this was the compiler's preference.  It was an exception to program in pure ASM.

In my experience disassembling SNES games by several different developers, compiler-generated code was the exception, hand-written ASM was the rule. Not that compilers weren't used at all--I've found code in Romancing SaGa and Seiken Densetsu 3 that was almost certainly compiler-generated.
Title: Re: Branching in Disassembled Code
Post by: tryphon on September 03, 2015, 05:44:43 pm
I'd bet Street Fighter 2 was also written in some kind of C (at least MD version, but won't be surprised if all versions shared more or less the same code).
Title: Re: Branching in Disassembled Code
Post by: AnderHershey on September 04, 2015, 02:40:31 am
Quite a few long nights and a lot of lines transcribed, I now have a few more questions. Forgive me if I'm asking too much, personally I wish there was a way to ask quick questions without making something as big as a "post" on a forum.

So, in a routine I found, it jumps to a location.
This location does have stuff, and I wrote it all out.
While using the debugger I set the zero flag such that I'd
land on the JSR that would send me to the location in order
to check a few things.
Upon doing so though I got a completely different set of instructions.
I then picked one of the instructions, looked it up in the
hex editor, and found only one instance of it.
I was like "Ok this has everything the same except for the first byte".
meaning, of XXXX the one on the far left is different.
I then thought, ok, maybe something jumps here.
So I just look up the address to where it'd jump, leaving
the first part wild card since I don't know what type of jump
they might use to get here.
Even then, nothing appeared except for things where it wasn't a jump command.
At this point I was hoping I had written something wrong, but I hadn't.
So I don't know what to do.
The jump instruct in the hex is JSR B767.
In test code it's
0b767 (where the next instruction starts)
this next instruction, the one it starts at after the JSR,
occurs only in one location in the rom though, and that is at 3767.

Also, as an aside, PLB, from the manual on the langauge,
pulls the top 8 byte of the stack and has it replace B register
which stores the bank one is currently in.
I could see this holding purpose if one needed to get back to
a location without a JMP (since maybe the location is variable),
but then they proceed to RTL, which should return them to where
they want regardless of the bank they are in.
(I am assuming that since JSL can go "anywhere", RTL can return "anywhere").
I guess if the return location is on the stack and was stuck under
the bank data register 8 bits I can see that being a reason to PLB
before RTL, but beyond that, I don't see why.

update on the last part:
So, if when one JSL's it pushes the address lets say the current address location is 018040 the when it's put on the stack it's:
40 placed, 80 placed, and lastly 01 placed.
In this idea, PLB will take the 01, and then RTL takes the 8040 to return. (this is if RTL doesn't incorporate the bank for some reason, even though JSL does).

And as a final addition:
Also, one of my JMPs, JMP 8D70, which in test code goes to there (in the debugger), but that location would be 0D70.
Since the JMP is evident in the hex, I feel like that should head to 8D70, but the test code shows otherwise. So I don't know why what I debug and step through gives different from logical progression from finding the breakpoints, finding them in hex, finding corresponding jumps, and following manually.

edit: Upon looking at:
ASM Hacking for Dummies
I noticed 5.1 so I think that might have something to do with these issues.
Title: Re: Branching in Disassembled Code
Post by: AWJ on September 04, 2015, 06:00:20 am
Quite a few long nights and a lot of lines transcribed, I now have a few more questions. Forgive me if I'm asking too much, personally I wish there was a way to ask quick questions without making something as big as a "post" on a forum.

So, in a routine I found, it jumps to a location.
This location does have stuff, and I wrote it all out.
While using the debugger I set the zero flag such that I'd
land on the JSR that would send me to the location in order
to check a few things.
Upon doing so though I got a completely different set of instructions.
I then picked one of the instructions, looked it up in the
hex editor, and found only one instance of it.
I was like "Ok this has everything the same except for the first byte".
meaning, of XXXX the one on the far left is different.
I then thought, ok, maybe something jumps here.
So I just look up the address to where it'd jump, leaving
the first part wild card since I don't know what type of jump
they might use to get here.
Even then, nothing appeared except for things where it wasn't a jump command.
At this point I was hoping I had written something wrong, but I hadn't.
So I don't know what to do.
The jump instruct in the hex is JSR B767.
In test code it's
0b767 (where the next instruction starts)
this next instruction, the one it starts at after the JSR,
occurs only in one location in the rom though, and that is at 3767.

CPU addresses aren't the same as offsets in the ROM. You need to understand the SNES memory map. Within each bank:

$0000-1FFF is RAM
$2000-7FFF is memory-mapped I/O (registers controlling video, sound, DMA, etc.) and cartridge-specific stuff (e.g. battery RAM)
$8000-FFFF is ROM

The exceptions are banks $7E and $7F which are entirely RAM, and banks $40-7D and $C0-FF which are entirely ROM.

The first $8000 bytes of ROM are mapped at $8000-FFFF in bank 0, the next $8000 bytes are mapped at $8000-FFFF in bank 1, the next $8000 bytes in bank 2, et cetera.

To convert a CPU address to a ROM position: Take the non-bank part of the address, subtract $8000 from it, then multiply the bank number by $8000 and add it. If the non-bank part of the address is less than $8000 then the address isn't in the ROM at all, it's in RAM or it's a memory-mapped register.

So a JSR $B767 in program bank 0 will indeed jump to offset $3767 in the ROM.

To convert a ROM position to a CPU address do the reverse: the bank number is the ROM offset divided by $8000, and the address within the bank is $8000 plus the offset modulo $8000.

(All of this applies to Mode 20 aka LoROM, the most common type of SNES cartridge. There is another type called Mode 21 or HiROM that is mapped differently. Don't worry about it for now. I can tell from your description that the game you're disassembling is LoROM and not HiROM)

Quote
Also, as an aside, PLB, from the manual on the langauge,
pulls the top 8 byte of the stack and has it replace B register
which stores the bank one is currently in.
I could see this holding purpose if one needed to get back to
a location without a JMP (since maybe the location is variable),
but then they proceed to RTL, which should return them to where
they want regardless of the bank they are in.
(I am assuming that since JSL can go "anywhere", RTL can return "anywhere").
I guess if the return location is on the stack and was stuck under
the bank data register 8 bits I can see that being a reason to PLB
before RTL, but beyond that, I don't see why.

The program bank and the data bank are two different registers. The program bank is where code is executing from--think of it as an extra 8 bits on top of the program counter. The data bank determines which bank the data is accessed from when you do LDA, STA, etc. If the CPU does LDA $1234 and the data bank is $7E, the full address it loads the data from is $7E1234. PHB and PLB affect the data bank, not the program bank.