Possible to change the tile palette attribute in original Gameboy Games ??

Started by amaturehr, January 28, 2023, 02:52:42 PM

Previous topic - Next topic

amaturehr

Hello,

I was wondering if this situation is possible:

When booting an original GB game on a GBC, the GBC bootstrap rom can apply 3 color palettes, one to BG, one to OBP0 and one to OBP1.

Sometimes, the colorization with a builtin palette looks really good minus a few colors here and there (i.e. "Yoshi" gameboy game on GBC looks great out of the box, so to speak).

Would it be possible to modify the game rom in such a way that the gameboy color would apply the OBJ0 palette to a BG tile (for example), if it thought the tile attribute was OBP0 instead of BG?

If this is possible, what would be the general approach to possibly testing this?

I am very new to everything, but in general, if you load up the VRAM viewer in BGB Emulator, it shows you things such as:

- OAM address
- Tile Addres
- Palette
- Attribute
- Tile Number

Would it be possible, using the above information, to walk this backward and find it in the raw ROM file via Hex Editor and changing a bit or byte some place to have the GB think the palette attribute is different than what it was originally??

Thanks in advance to any responses.

xenophile

Quote from: amaturehr on January 28, 2023, 02:52:42 PMWould it be possible to modify the game rom in such a way that the gameboy color would apply the OBJ0 palette to a BG tile (for example), if it thought the tile attribute was OBP0 instead of BG?

I find the GBC/GBA's built in colorization really interesting. You can see some examples of what I've done with it mixed into my Legend of the Mana Sword thread.

It does have a lot of limitations, though. While you can switch sprites between the two sprite palettes (though not without potential issues to consider), the background tiles will only ever use their one palette.

QuoteWould it be possible, using the above information, to walk this backward and find it in the raw ROM file via Hex Editor and changing a bit or byte some place to have the GB think the palette attribute is different than what it was originally??

Yes, for sprites. Backgrounds do not have attributes unless your game identifies as a real GBC game, and if you do that you typically need to write a lot of code just to match what you had before with the auto-colorization.

I like how you are thinking, though. FYI, games published by Nintendo are the ones with custom palettes, but it's pretty simple to "hack" a game that doesn't have a built in palette (or if you want to change it to a different built in palette) just by changing the game's internal name.

SegaMan89

It has to do with which sprites are in which layer.  There was a device called a "Monster Brain" that was a more advanced version of the "Brain Boy" that allowed you to set which palettes you want and save them for different games.  It was AWESOME!

amaturehr

Thanks for the responses.  I have been tinkering with the Yoshi GB version to try and get all the characters to look as "correct" as possible with the built in GBC Palette that is used for this game:






This has proved pretty challenging because I decided to try a dither pattern, to make up for situations where there were not enough colors for a given palette.  I even try to use the invisible "color" to line up with a background color to give more color options for a sprite.

Because I don't know anything about ASM, this is the best I can do, BUT, if it IS possible to assign a attribute to a OBP0 vs. a OBP1 tile, that would help alot, for example here, how could I go into a HEX Editor to tell the GameBoy that this tile should get the Palette for OBP1 instead of OBP0:



I am likely using the wrong kind of terminology here, but are any of those HEX numbers something that indicates a OBP0 sprite vs. OBP1 sprite?  I understand even if this is possible you could run into sprite limits, etc, but I am interested in testing the concept.

xenophile

Quote from: amaturehr on February 01, 2023, 11:42:26 AMBecause I don't know anything about ASM, this is the best I can do, BUT, if it IS possible to assign a attribute to a OBP0 vs. a OBP1 tile, that would help alot, for example here, how could I go into a HEX Editor to tell the GameBoy that this tile should get the Palette for OBP1 instead of OBP0:



I am likely using the wrong kind of terminology here, but are any of those HEX numbers something that indicates a OBP0 sprite vs. OBP1 sprite?

Yes, and yes you can.

The bottom number in each column is the sprite's attribute(s). Looks like you're using bgb, so if you mouse over the sprite it will show it on the right with the attribute number labeled, but not decoded. Here is information on understanding that value. Short answer, if you change that from 00 to 10, or 01 to 11 it should change those sprites from OBP0 to OBP1. Caveat: it's doing something strange because the flipped sprites have bit 0 set instead of bit 5, which won't do anything on DMG.

Next, finding the source of those values. If you use Emulicious it has tools for "tracing" the source of a value that might work automagically. If you use bgb your best bet is to set a break on the VRAM transfer function at $ff80, use that to figure out where "shadow OAM" is, then set a watchpoint where the sprite's attribute will be written, and when it breaks look just above to see where it's reading from. Your goal is to trace backwards to find a read from ROM that gets used for that attribute. Hopefully Emulicious makes it easy for you.

Good luck! Looks like a cool project.

amaturehr

My god @xenophile it WORKED!



Thank you so much for your help!  I had never heard of Emulicious before, but you were right, it really helped me narrow down the location in ROM for the sprite, and I did just what you said!  Much appreciated.  Just for a comparison so far, here is how the game "Yoshi" looks with the default GBC palette vs. my attempt at edits:

-------ORIGINAL----------------CHANGES---------



With this new found power, I know I can make better changes.

xenophile

Those look much improved (IMHO). Good job taking my vague instructions and running with them.

Different emulators are good at different things. Emulicious is really good for tracking down the source of tile attributes.

amaturehr

So last-ish question, as I am bumping up to the extent of my knowledge on all this, but @xenophile, with your help, I have been able to modify almost all the graphics to my liking, considering the GBC color palette limitations (since, I have no way to do a actual real GB to GBC mod w/ my knowledge). 

The last real sticking point I am at is a set of sprites that are basically "re-used" to simulate Mario turning. 

It seems like the sprite is basically flipped and has the opposite OBP* palette applied to it, causing a situation where it looks how I want it to when Mario flips one way, and NOT how I want it to when he flips the other.  Granted its only a few frames of animation, BUT it would be super cool if I could fix.

It is a little hard for me explain, but I tried to provide a screen cap that shows what I mean.  I have modified the ROM to apply the palette to Mario how I like in the first screen shot.

Using Emulitious, I flip Mario in the game, pause, and find that sprites' palette attribute in the ROM, 2nd and 3rd screen shots show the sprites, ROM address, and how it looks in game.  Basically Mario has a darker red palette applied, instead of the light red that I am going for:



...trying to change this in the ROM causes huge graphical corruption with the game, as seen in the last screen shot.



Is it possible I am "close" to figuring out where to tell the GB to not change the palette when the sprite is flipped, or something??


xenophile

One of two things is happening to flip it: either the sprite attributes have bit 5 set (0x20), or it uses tile graphics that are already flipped.

You should be able to figure out which. Do the attributes have bit 5 set? Or do the tiles show up in the VRAM tileset viewer already flipped? One and only one of these things should happen.

If bit 5 is getting set then it is quite likely that the game is only storing one hard coded attribute value and changing it for the flipped sprites.

On the other hand, if you see pre-flipped tiles in the VRAM tileset viewer then it might be that they are being flipped at runtime and something is wrong with the flipping.

The third option is that they store both the flipped and unflipped tiles in the ROM and you just need to edit a second set, but that seems unlikely.

Looking back to one of my previous replies, "it's doing something strange because the flipped sprites have bit 0 set instead of bit 5, which won't do anything on DMG."

amaturehr

I appreciate your patience with me answering my questions, I did recall your comment about something weird with the bits but honestly I still don't even know what that means.  I tried to take a few more screen caps with more information.

So here is the sprite with the palette I WANT applied (which is the result of my edit):



Here is how it looks when it gets flipped, and has the darker palette (which is the default).



I make the edit to the ROM, which can be seen in the screenshot (01 to 11) , but...It causes a issue where now the graphics are messed up, only showing his arm:



So, it seems like where I am editing the value, is not actually correct, the tile attributes don't change and it seems to just point to the wrong place $00??

xenophile

Thanks for the screenshots.

Okay, so the fact that the value in ROM at 01:44d9 is not equal to the attributes is a clue that there is more going on. At this point it will require at least looking at the assembly. Thankfully, Emulicious tells you right where to start looking when it says "see ROM 01:44D8", but there's another big hint right there that assembly is involved: the address of the instruction that wrote the attributes is one byte before the supposed attributes value.

A glance at the hex and a quick visit to an opcode table, and unless I'm misreading it, the code is:

ld hl, $c200
ld [hl], $01
inc hl
inc hl
ld [hl], $00
inc hl
inc hl

And then it cuts off on the bottom of the window.

That's a little confusing and it would be good to get a real disassembly of that function.

Even then, what you probably need to do is watch for any writes to c203 (or c207) to see what is really going on.

Sorry if this is a bit confusing. The difficulty on this is definitely a bit higher than what you've encountered so far.

amaturehr

By god, you have the ASM right, based on the disassembly Emulicious shows, amazing! 

Obviously I still don't really know what it all means, but here is the full output on that function:

    _LABEL_44D0_:
   ld a, $01
   ld [_RAM_C6E0_], a
   ld hl, _RAM_C200_
   ld [hl], $01
   inc hl
   inc hl
   ld [hl], $00
   inc hl
   inc hl
   ld [hl], $80
   inc hl
   inc hl
   ld [hl], $20
   ret

This seems like you are teaching someone who can't fly a plane how to land it :)

Would...something like a GameGenie code be do-able ?  I suppose there could be some unintended consequences if you have a gamegenie code that compares a RAM value of X and changes it to Y if its not super isolated, but maybe that would be easier than trying to edit the rom and see what happens?

xenophile

Quote from: amaturehr on February 06, 2023, 11:22:11 PMBy god, you have the ASM right, based on the disassembly Emulicious shows, amazing!

Thanks! But being able to read it and being able to make sense of it are two different things. 

QuoteObviously I still don't really know what it all means, but here is the full output on that function:

    _LABEL_44D0_:
   ld a, $01
   ld [_RAM_C6E0_], a
   ld hl, _RAM_C200_
   ld [hl], $01
   inc hl
   inc hl
   ld [hl], $00
   inc hl
   inc hl
   ld [hl], $80
   inc hl
   inc hl
   ld [hl], $20
   ret

Okay, what I expected to see was a label between ld hl, _RAM_C200_ and the next line, defining a loop. It doesn't look like that's what is happening here.

I expected that $c200 in RAM would be "shadow OAM" where everything about sprites is written before being transferred to OAM at $fe00. My guess is it's not, but I'm unsure.

QuoteThis seems like you are teaching someone who can't fly a plane how to land it :)

Anything you can walk away from is a landing?

QuoteWould...something like a GameGenie code be do-able ?  I suppose there could be some unintended consequences if you have a gamegenie code that compares a RAM value of X and changes it to Y if its not super isolated, but maybe that would be easier than trying to edit the rom and see what happens?

Think of GameGenie as a really weird hex editor for ROMs. What it does is intercept the CPU's request to read a byte from ROM and replace the real byte with something different. From what I've seen, there's nothing the GameGenie can do that you haven't already been able to do.

Open questions:

Q: Where is shadow OAM?
A: Set a break point at $ff80, look at the value in the A register when it is hit, and report it here.

Q: Where are the attributes really coming from?
A: Once you get the location of shadow OAM (the value of the A register above, followed by two zeros), set a watch point on that address plus 7, and see what writes $22 to it.

FAST6191

Quote from: xenophile on February 07, 2023, 04:54:38 PMThink of GameGenie as a really weird hex editor for ROMs. What it does is intercept the CPU's request to read a byte from ROM and replace the real byte with something different. From what I've seen, there's nothing the GameGenie can do that you haven't already been able to do.

Certainly game genie = ROM modifier and especially in this case
https://doc.kodewerx.org/hacking_gb.html#gg
Some later ones, even before the name was sold on for save editors in somewhat later generations, did have minimal RAM abilities.
I should also note that apparently it is limited to 3 codes at a time and even on 8 bit that is going to be a serious limitation, unless you are hardpatching with CCGP, game genie guy or have an emulator/flash cart/similar that is all fill yer boots.

amaturehr

Well @xenophile I really did give it the ol college try, but unfortunately I just wasn't able to fully understand how to do what you recommended. I tried setting a break point at FF80 but it seems like it was just constantly breaking, so I was prob. not understanding how to isolate to the sprites with the palette I want adjusted. 

But I have put a good amount of work into this and wanted to share how it looks so far.  I would say aside from this one last issue you have been helping me with everything is mostly done.  The real bummer is in a two player game, playing as Lugi this weird color issue is way more noticeable.



This is my first attempt at a rom hack, but I tried to create a xdelta patch, I haven't had time to test it yet, but its here if anyone cares to look:

https://github.com/amaturehr/yoshi_gbc_hack/blob/main/yoshipatch.xdelta

xenophile

Quote from: amaturehr on February 10, 2023, 10:45:01 PMWell @xenophile I really did give it the ol college try, but unfortunately I just wasn't able to fully understand how to do what you recommended. I tried setting a break point at FF80 but it seems like it was just constantly breaking, so I was prob. not understanding how to isolate to the sprites with the palette I want adjusted.

FF80 gets called once a frame, so it will constantly break. There's one piece of information we need from it: The value of register A when it is called. This should be the same every time it's called (so only need to break there once). That will tell us where Yoshi keeps its real sprite RAM. (It's not actually at FE00 because that's not how GB games actually work.)

QuoteBut I have put a good amount of work into this and wanted to share how it looks so far.  I would say aside from this one last issue you have been helping me with everything is mostly done.  The real bummer is in a two player game, playing as Lugi this weird color issue is way more noticeable.

I'm not really familiar with the game, but your before an after images look like great improvements! I'm really impressed with what you've done.

amaturehr

Thanks for not giving up on me!

I *think* the value is "00" but, here is the full screen shot:


xenophile

Quote from: amaturehr on February 11, 2023, 01:53:34 PMThanks for not giving up on me!

Thanks for putting in the work!

QuoteI *think* the value is "00"

Technically, you're right, but the first line of that function loads $c4 into A.

So what does that mean? In short, it means that Yoshi is (barring anything really strange) going to make all of its sprite changes in $c400 to $c49f. If you want to see what writes the attributes of sprite 1 (for example), you should watch for changes at $c407. Whatever the final state of $c407 (I think it might be written more than once per frame?) is will be copied by the $ff80 routine into $fe07.

xenophile

Looking at this again, what it looks like is that the palette its self is being changed. A little background: Each of the three palettes--the background and the two for objects--have four colors, but sprites can only display three colors. Which three of the four colors can be changed on the fly, but it usually affects all the sprites currently using that palette.

That said, I took a quick look at it and I haven't been able to reproduce what you showed, but I haven't tried your patch yet (it's a 404 for me, and also ips would be easier to deal with). Does it happen with an unmodified rom? (MD5: a8804c8514619cc918960c2008ed65d1)

amaturehr

I was spending all this time to draft up a reply w/ screen shots and in doing so, believe or not @xenophile, you helped me again! I figured it out! 

So, when you said you didn't see the issue in the original game, I thought...yeah, why did I change all this stuff again??  I have mucked w/ the ROM so much I kinda forgot what I did, but there is a good reason it doesn't happen in the original game.

The Mario Sprites rely on the palette for OBP0, and uses the transparency color for "White".  But, I had changed the white background to a white and green dither, to help give other sprites more "accurate" color.  Thus, making Mario now have a green and white checkerboard face.

No problem, I thought, I will just swap the "transparent" color with the other lightest color available in the palette, and it created sun-burned Mario, which I thought, is too red, I didn't like it.

So, then I said, heck lets tell the game to give Mario the OBP1 palette, its got pure white, light red and black, which is all I need.

THAT is when the problem started, I got Mario looking just how I wanted, except for this rotation issue where the game mirrors the sprites and it still having OBP0 for those very few frames.

So, retracing all my steps to what I had done, I came across a situation where I had actually just randomly guessed what to do.

Most of the time, I did how you show, where I find the tile attribute location in ROM and change like a 0 or 1 to vice versa.

However, there were a few sprites who's tile attribute instead of being "00" or "10", was "20".  And I was like...uh, I dunno, change it to 30???  And...it worked, so I thought "I am genius" and continued on, if I found something with a 20, I changed it to a 30.

But, as I retraced all my steps, I noticed that when the sprite was flipped, the tile attribute would point to address that had "01", changing this would corrupt the game.

But, and again I don't even know why I thought to try this or why it works, I went to the "not flipped" attribute location, looked a few bits over for a "20" or "21" and just change it to "30" or "31"

Now he looks "correct" on either side: