NES - Trying to change $0D black in Mystery World Dizzy

Started by willy_poodle, February 11, 2022, 05:53:28 PM

Previous topic - Next topic

willy_poodle

Hello everyone.
I'm trying to replace the infamous $0D black from the entire game with $0F.
I do it as some show in their tutorials:
1. find the set of colours in FCEUX PPU Viewer or VirtuaNES Palette Edit
2. find where they are in hex editor and change
Of course I save it in the new file and load it in emulator or a flash cart.
It works with the sprites but no matter which 0D I edit nothing changes the black in the backgrounds.

As you know this game has been released not so long ago, when everyone already knew about the issues with this colour so I have no idea why they haven't changed it straight away. On the TV it looks horrible with this grey instead of black  >:(

KingMike

Have you verified the changes were actually made by opening the patched ROM and checking the colors in FCEUX?
"My watch says 30 chickens" Google, 2018

willy_poodle

Yes, after the changes I was saving to a new file and loading the patched rom. Nothing changes it in the backgrounds, only the sprites.


Just like in The Fantastic Adventures Of Dizzy (version 1 and 2), where nothing changes the background under the scroll at the beginning.

Quick Curly

Hi. I was interested by your post and goal. I played through the game and tried my best to track down all of the 0D black values to change. It's always possible that something could have been missed though. I didn't bother with the egg item to find out if that would have possibly led to another different area that would use additional palette data.
I believe your issue(s) might have been caused by some additional code that writes 0D black values to CPU memory, and then those values are applied to the palette in PPU memory. So the 0D value(s) that you were looking for was/were actually code, and not data in the form of a full palette.
All of the following ROM offsets need to be modified.

0x1D708: 0D - Change to 0F
0x1D70C: 0D - Change to 0F
0x1D710: 0D - Change to 0F
0x1D714: 0D - Change to 0F
0x1D718: 0D - Change to 0F
0x1D71B: 0D - Change to 0F
0x1D71C: 0D - Change to 0F
0x1D720: 0D - Change to 0F
0x1D724: 0D - Change to 0F
0x3AB77: 0D - Change to 0F
0x3AB9D: 0D - Change to 0F
0x3AB9E: 0D - Change to 0F
0x3ABA1: 0D - Change to 0F
0x3ABB0: 0D - Change to 0F
0x3ABC5: 0D - Change to 0F
0x3ABD4: 0D - Change to 0F
0x3ABF8: 0D - Change to 0F
0x3AC04: 0D - Change to 0F
0x3AC0A: 0D - Change to 0F
0x3AC1C: 0D - Change to 0F
0x3AC2E: 0D - Change to 0F
0x3AC31: 0D - Change to 0F
0x3AC40: 0D - Change to 0F
0x3AC50: 0D - Change to 0F
0x3AC62: 0D - Change to 0F
0x3AC74: 0D - Change to 0F
0x3AC86: 0D - Change to 0F
0x3AC9A: 0D - Change to 0F
0x3AC9D: 0D - Change to 0F
0x3ACAC: 0D - Change to 0F
0x3ACAF: 0D - Change to 0F
0x3ACBE: 0D - Change to 0F
0x3ACD0: 0D - Change to 0F
0x3ACE2: 0D - Change to 0F
0x3ACF4: 0D - Change to 0F
0x3AD06: 0D - Change to 0F
0x3AD18: 0D - Change to 0F
0x3AD2B: 0D - Change to 0F
0x3AD2D: 0D - Change to 0F
0x3C90C: 0D - Change to 0F

Unfortunately, code that applies fade-in and fade-out effects will take values like 1D and 2D and result in 0D still getting applied to the palette. However, this is only during the actual transitions, while the screen is black. During my searching and testing, the backdrop colour was never overwritten to a 0D black. Therefore, it might not be something that needs to be worried about. However, I didn't test any modified ROMs on real hardware - only through FCEUX - so you might need to check for any potential issues, and report back any additional findings.

Unrelated to the 0D black issues, depending on how "refined" you would like the game to be, it seems like this game has a few spelling errors, and maybe other errors.

For example:

"DESPARATELY" should be "DESPERATELY".

0x2F81A: 41 - Change to 45

This modification would be simple enough, since only one value needs to be changed, and it's not necessary to expand the text string.

"YOU TIE TO BUCKET TO THE END OF THE DANGLING ROPE."
Should be: "YOU TIE *THE* BUCKET TO THE END OF THE DANGLING ROPE."

0x038B1-0x38B2 (0x2): TO

0x01B27-0x1B73 (0x4D):
THE LIGHTENING BOLT WAS DEFECTED BY THE SHIELD. IT BOUNCED BACK AND HIT ZAKS.

"LIGHTENING" SHOULD BE "LIGHTNING".
"DEFECTED" SHOULD BE "DEFLECTED".

In the cases of some spelling errors, like for the bucket situation, there isn't enough space to apply a fix without cutting into other text. So, the appropriate text pointer(s) would have to be located, so that the text could be moved to a different area where there is enough free, unused space.

The above examples most likely don't cover or account for everything, but they touch upon some other factors to consider for the game overall.

This is a neat game though, and the backstory behind the game finally becoming available for everyone to enjoy is fantastic. :woot!:

Best of luck with your project and goals.

Bavi_H

Quote from: willy_poodle on February 11, 2022, 05:53:28 PMI'm trying to replace the infamous $0D black from the entire game with $0F. [...] no matter which 0D I edit nothing changes the black in the backgrounds.

willy_poodle, if you want to continue investigating yourself, the next step I would take is to set a conditional breakpoint to catch any write to a palette location that writes a value of hex 0D. (I made pictures that show how to set this conditional breakpoint in FCEUX or in Mesen.) Each time the breakpoint is activated, continue investigating to find where the 0D value originated from in the ROM.


Quote from: Quick Curly on February 15, 2022, 01:00:01 PMI played through the game and tried my best to track down all of the 0D black values to change. [...] All of the following ROM offsets need to be modified.

Quick Curly, please post a hash of the NES file you used. When I was also investigating this, I discovered there were two(?) versions of the NES file.

Details: I had downloaded a copy of the game when it was originally released and still had that copy on my computer, so I started looking at that copy of the file. A little later, I found the official website again to make sure the file was still publicly available. I noticed the file currently available on the site was different than the one I had originally downloaded! Using a well-known archive that captures versions of websites, I discovered there are probably two versions. (I believe the forum rules say I can't link to the files, but, as you know, the files were officially released for free, so it's perfectly legal to get them.)

Original release(?), archived 2017-04-08, PRG+CHR CRC32 E0413A95
Revised release(?), archived 2017-04-15 and later, PRG+CHR CRC32 2EAFD5A9

I like to use the PRG+CHR CRC32 as a hash. You can use emulators FCEUX or Mesen to check the PRG+CHR CRC32 of an NES file as follows...

To get the PRG+CHR CRC32 in FCEUX:

1. Open FCEUX and open the game.
2. Go to the Help menu and choose Message Log.
3. Look for the line ROM CRC32.

To get the PRG+CHR CRC32 in Mesen:

1. Open Mesen and open the game.
2. Go to the Tools menu and choose Log Window.
3. Look for the line PRG+CHR CRC32.


What's different? I did a shallow comparison and believe there are translation differences and code differences, but I didn't dig into it in detail.

The first change you may notice: If you open both of these NES files in a hex editor and look at the beginning of the file, you can see ASCII text and see the Portuguese translation is a little bit different. The first line that is different is also visible if you play the game in Portuguese and talk to the troll on the first screen:

In English, Dizzy says "PLEASE LET ME OUT."

In the original release's Portuguese, Dizzy says "ME DEIXE SAIR!" ["Let me out!"], and the text overflows the 8 × 2 text box.

In the revised release's Portuguese, Dizzy says "ME DEIXE IR!" ["Let me go!"], and the text fits in the now 10 × 2 text box.


For most purposes, my guess is it'd be better to use the revised release -- it looks like the original release was only available for a short time. Perhaps someone may eventually want to investigate the differences in detail to see if there are any benefits to the original release, or just to document the differences.



February 15, 2022, 09:27:06 PM - (Auto Merged - Double Posts are not allowed before 7 days.)

Quote from: Bavi_H on February 15, 2022, 06:15:17 PMQuick Curly, please post a hash of the NES file you used. When I was also investigating this, I discovered there were two(?) versions of the NES file.

Well... it's always a good idea to post the hash of the NES file you are using, but maybe the versions aren't so different to cause a problem?:

I checked the long list of offsets containing 0D values, and both versions have 0D in all of those locations -- except 3AD2B (contains hex 24 in both versions) looks like it is a typo that is supposed to be 3AD2A (contains hex 0D in both versions).

Some of the offsets for the text are shifted a few bytes between the two versions, so it looks like Quick Curly was using the revised version (current version on the site, and archived 2017-04-15 and later, PRG+CHR CRC32 2EAFD5A9). Because it's ASCII-based, anyone attempting to edit the text can easily see it in a hex editor and confirm they are editing the correct bytes.

Quick Curly

Good call, Bavi_H. I normally would have provided ROM checksums and information, but I guess I've been out of it for a while. It's interesting that a revised version was released very early on. I'm curious what they actually changed.

The original, unmodified ROM of Mystery World Dizzy that I was using appears to be the revised version:

ROM CRC32: 0x2eafd5a9

Another good call about the 0x3AD2B offset. Indeed, it was supposed to be 0x3AD2A. I must have referred to the hex editor window to type that offset out after I had already modified it to a 0F value, so the cursor would have moved and highlighted the next offset. My setup is not great, though it works for me... well, most of the time. My bad. Sorry for any inconvenience. :-[

To elaborate on the process I used to document the offsets for anyone else reading in the future, these were the conditional breakpoints that I used. Again, I used FCEUX.

$3F00-$3F1F - Write - PPU - Condition: A==#0D
$0478-$0497 - Write - CPU - Condition: A==#0D
$0103-$0122 - Write - CPU - Condition: A==#0D

$3F00-$3F1F is for finding the writes to the palette.
$0103-$0122 and $0478-$0497 seem to be buffer areas for those writes.

What's funny is that the title screen already uses the good 0F black for the palette values. The first hit is after the game is started, at $C8FD (0x3C90D in the ROM file), writing to $2007. 0x3C90C is 0D, but outside of palette data in the form that you would normally be used to seeing or expecting. In the debugger window, there are "Run" and "Step Into" buttons. By clicking "Run", the same breakpoint is hit again, and it is after a few more times. So, to proceed from this point, in the hex editor window, I modify 0x3C90C from 0D to 0F. Through my process, I didn't save until the end, just in case I had to perform an emergency evacuation. Unsaved changes light up in red, and can take effect for your active game, but aren't permanent until actually selecting the "Save Rom" option in the hex editor.

Now, after clicking "Run" again, this breakpoint is still getting hit because, despite that value now being 0F, the value in A is still 0D. To save time and spare the reader details, I'll recommend to just reset (since the game only recently loaded up anyway) or reload a recent save state that you could have made; though you could try to use a combination of the stack to find the most recent jump location in code, and execute breakpoints to locate certain critical hits for tracking down lines of code and events that you could be looking for. Possible actions depend on the details of the current situation and how deep and dirty you want to dig.

After restarting, the next breakpoint that gets hit is $AB68 (0x3AB78 in the ROM file), writing to a range starting at $0478. The previous line, $AB66, A9 0D, is a LDA #$0D instruction. Therefore, the next value to change is 0x3AB77. Take note that whenever you make a modification in the hex editor, and then you switch activity from the hex editor to the debugger window, the debugger window will update. So, the 0D value has updated to 0F. However, as far as the game's logic is concerned, A has already been set to a value of 0D. By using the "Step Into" button, you can navigate through one line of code at a time, checking to see what is happening at a more readable pace. After stepping through multiple lines of code, as X is now 04, the $AB66 line is executed again, this time with the value of 0F. However, $0478 in CPU memory has already been set to 0D from the previous instance of this code running, and that's not what we want for the process of "fixing" this game overall.

So, from here, you can either reload a previous save state again from before this code would have been executed (easily controlled by setting the save before you pressed start from a main menu, or entered a new area, etc.), or if you want to get really crafty, you can "fix" the memory address yourself. In the hex editor window, under the view menu, switch from "ROM File" to "NES Memory". Navigate to $0478. The value here is currently 0D. So, I manually set it to 0F. See, that's what would have happened if the line of code that was already executed the one previous time had set A to our intended value of 0F. Therefore, this is a sneaky way of "fixing" a memory location that was set before planned modifications, so that you don't have to return to an earlier point of execution.

Though, yes, you do have to remember that direct changes that you make to RAM or the PPU are not permanent, nor are they saved if you reset or close the current ROM or program. Only the changes made to the ROM file that are saved will be permanent.

I'll also take this moment to apologize for not having any images to provide friendly visuals to follow. Normally, I would try to include them as well, but really, there would just be too much to show from many individual steps. So, sorry for a large wall of text that isn't broken up by pretty pictures.

After clicking "Run" again, the next breakpoint hit is at $AB73 (0x3AB83).

0E:AB70:BD 7A AB  LDA $AB7A,X @ $AB8D = #$0D
>0E:AB73:9D 78 04  STA $0478,X @ $048B = #$00


A is 0D, so the conditional breakpoint is hit, and the previous line of code shows us where the LDA ("LoaD Accumulator") obtained the value from: $AB8D (0x3AB9D). At this point, there are a lot of visible 0D values within this area of the ROM file. As it turned out, all of them needed to be changed. (Well, actually, I didn't change the 0D at 0x3AB9A, because I never experienced it actually getting used while I played the game. This doesn't actually confirm that it's never used, though, but some games do have bytes for padding that are left unused.) However, I would go through the process of having the game obtain each unique value to confirm that the game uses them as it seems they would be, because in reality, you never know until after the fact! Every game is different, and there are numerous ways one can program their game. Evidently, when you consider using 0D for black instead of using 0F. :D

The procedure of manually fixing 0D writes to the buffer areas in memory would be put into play as I played through the rest of the game to confirm all of the 0D values.
When the debugger window "snaps" as a breakpoint is hit each time, normally whenever Dizzy enters a new area, you can track down the specific ROM address, and modify the value in the actual ROM file through the hex editor.
Next, in the debugger window, you can click the "Step Into" button once to execute the line of code for writing the 0D value into CPU memory (i.e. the STA instruction line, though depending on the game, one or all of STA, STX, and STY could be used, similar to LD[?] opcodes: LDA, LDX, and LDY), and then in the hex editor window, you navigate to the appropriate address in CPU memory, and fix the 0D value to 0F.
Between that and making use of convenient save states for useful checkpoints in case the 0D black situation ever gets out of control, you can track down any and all offsets relevant to your situation, whether it's data or code.

Do note, there are also "false positive" hits to account for, even after modifying a lot of the 0D values that definitely are palette values. For instance, the relevant stack buffer is reused for other purposes. When entering areas with different music or other unique features (like a gate, dragon, etc.), the $0103-$0122 conditional breakpoint is hit at $E0DB (0x3E0EB) because A is equal to a 0D value, obtained from $800D (0x3401D), while storing that value to $010D, within the $0103-$0122 range. However, this is an occurrence that is unrelated to the palette. Therefore, one has to be cautious and take care in not modifying *every* single value hit, by recognizing what the code is actually doing, and how those values are being used. That's part of the reason why, even though it might feel redundant... superfluous... excessive... repetitive... I took the time to confirm each and every 0D byte that was loaded and stored to have been specifically used as a palette related value.

What was that about me not including images? Well, here's one, I guess.



This is an area of interest regarding those pesky fades/transitions. Even if you modify all of the 0D values, colours like 1D and 2D are still set to 0D during fades. So, I would like to think that, outside of having to change those colours as well, an additional check/compare could be added following the "AND $00D4" line, so if A was ever 0D, set A to a value of 0F instead, and then write *that* value to that stack buffer. However, like with many issues that could be resolved by modified code, there isn't enough free space there, so a jump to an area with unused space - within a bank accessible within memory at this relevant time, mind you - would be necessary.

Finally, the 0D values within the 0x1D708-0x1D727 (0x20) range are used in the palette on the ending screen.

Out of curiosity, Bavi_H, are you currently working on any projects of your own? Looking through your posts, and considering that you've been around for a few years, you definitely have skills and have been very helpful for other projects. I would be curious and interested to see what you would enjoy working on and coming up with of your own, if creating was something that you would enjoy doing. :)

Bavi_H

Quote from: Quick Curly on February 16, 2022, 12:21:12 PMthese were the conditional breakpoints that I used. [...] I used FCEUX. [...] Condition: A==#0D

Tip: In FCEUX, when you set a conditional breakpoint, you can use W to represent the value that was written. Or when catching reads, you can use R to represent the value that was read. Reference: In the online FCEUX documentation, go to the Debugger page and scroll down to "Conditional Breakpoints". This same help documentation is available from the FCEUX Help menu as a Windows help file.

(This feature exists in the most recent version of FCEUX, but it might have been missing in some earlier versions? For example, in the past, I remember writing a FCEUX lua script that was trying to catch a value like this and having to look at the value of A in the lua script because there was no other way to do it.)


Quote from: Quick Curly on February 16, 2022, 12:21:12 PMWhat's funny is that the title screen already uses the good 0F black for the palette values.

Discussion about Dizzy games using the 0D black previously happened in this 2015 thread on NESdev forums: Televisions don't like Camerica games

In that thread, one user pointed out that the recently released Wonderland Dizzy used a safe 0E black, so it looked like the NES Dizzy games had switched away from using the bad 0D black at some point. I mentioned that as part of the release of Wonderland Dizzy, the programmer that got the re-discovered source code to compile, Łukasz Kur, got a blessing from the Oliver Twins to add other languages and a casual infinite lives mode. I wondered if he also might have changed the black color used as well. But Łukasz replied that he didn't change it, the original code he was provided for Wonderland Dizzy already had the 0E black.

For Mystery World Dizzy, I assume a similar process happened: The menu at the beginning that has the casual mode and language options was presumably developed in modern times, and the modern programmer used a safe 0F black. But when the original game code starts up right after the menu it uses the bad 0D black, I guess because the modern programmer didn't think of looking through the original code to make sure it was using a safe black.


Quote from: Quick Curly on February 16, 2022, 12:21:12 PMBavi_H, are you currently working on any projects of your own? [...] you've been around for a few years

I replied in Introduction Topic.

Cyneprepou4uk

Quoteit might have been missing in some earlier versions?
2.2.3 was still missing it.

Quotebecause there was no other way to do it
A couple of alternatives, though not as perfect as when using W.

1: A==#0D | X==#0D | Y==#0D

2: set execute breakpoint to 0000-FFFF range with a condition like $0123==#0D