Romhacking.net

Romhacking => ROM Hacking Discussion => Topic started by: Chicken Knife on September 19, 2018, 08:25:57 am

Title: General NES Hacking Questions
Post by: Chicken Knife on September 19, 2018, 08:25:57 am
So I've mostly finished my script revisions to Dragon Warrior 1 and I'm now looking at taking on the sequel. There isn't a dedicated table file posted, but when I plug in the table data from the first game, I'm able to read the text for spells, items and monster names in my hex editor. The curious thing is that the bulk of in game dialogue text does not show up in a readable form.

Glancing over information for later Dragon Warrior games, it looks like text compression exists for the dialogue in the 4th game. Am I most likely facing a compression issue in DW2?

If so, are there any good universal tools that would help me with the process of extracting, decoding, and reinserting the text?
Title: Re: Handling possible text compression
Post by: FCandChill on September 19, 2018, 05:43:03 pm
Chances are, this game uses DTE compression. It's very common in NES games. Can you find data that sort of ressembles dialogue like "S.e.h prince." ("Save the princess" compressed)?

Also, you might want to have a dedicated thread for your Dragon Quest questions ... just a suggestion.

September 19, 2018, 05:43:36 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
As for tools, check here https://www.romhacking.net/?page=utilities&category=14&platform=&game=&author=&os=&level=&perpage=20&title=&desc=&utilsearch=Go
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 19, 2018, 07:46:22 pm
I've used the ROM map available on this page to determine the block of data where the dialogue is stored. As I scan through that data, it does not appear as an abbreviated form of actual text like your example of S.e.h prince. Instead, all the data in that section appears as 80-90 percent non letter entries, with occaisional letter entries interspersed.

As far as your point on forum etiquette, it's a fair one. I know I've been throwing a lot of separate topics onto this board lately. I actually do have a dedicated Dragon Quest thread under personal projects. I figured that thread would attract primarily readers interested in my Dragon Quest project, but posting here would catch a wider audience of people willing to help with the numerous technical obstacles I face as a very new person to this process. Perhaps it would be better to have a unified thread here that isn't really about my project but serves to keep my large mess of questions in one place?
Title: Re: Handling possible text compression
Post by: FCandChill on September 19, 2018, 08:09:27 pm
I've used the ROM map available on this page to determine the block of data where the dialogue is stored.

You didn't link to anything in your post, but I assume you meant to link here: https://datacrystal.romhacking.net/wiki/Dragon_Warrior_II:ROM_map

The info you referenced is under the section name Dragon Quest not Warrior. In other words ... the page covers the Japanese version, not the English version. That may be your issue.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 19, 2018, 08:54:53 pm
Fair point, but I have used a hex editor to carefully scroll through all the data in the rom, several times actually. On each search I come across the full list of items, monsters and spell names. The only other text I come across is the prologue text added for the US version intro. The dialogue text is no where to be found.

When I go through DW1 or even DW3, I easily locate the large block of general dialogue text, along with the items, monster names and spells.
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 02:00:55 am
I second the idea of a dedicated thread for all this: no need to make lots of new threads. :)

As for the text compression question, if I can find the time I can have a look myself. I know for a fact that Dragon Warrior doesn't have compression but I can't remember about DW2. I'd be a bit surprised if it did because neither it nor the first game have an overwhelming amount of text.

Maybe later today I'll be able to spend five minutes checking the text to see how it's stored in the ROM. Compression does happen in text heavy games - my efforts translating Time Stranger from 1986 found a neat dictionary compression scheme - but again, I'd be a little surprised were that the case here.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 20, 2018, 08:58:54 am
Sounds great Psyklax. Once this issue gets figured out I'll change the title of this post to something generalized and put all my technical questions here going forward.

Going back to the issue at hand, I also found it strange that the game would use compression--especially since there is a ton of empty space in the US rom. I initially thought this might be an issue of the game having the alphabet under multiple table entries. To test that, I went into FCEUX Nametable Viewer and checked the hex code of the letters showing up in dialogue text. The hex code for the letters in dialogue was exactly the same as the code for the letters in monster, item and spell names--with all of the latter showing up in a hex editor when I load up table data. Would this indicate compression or could another factor be causing this? 

I appreciate you taking a look at the rom!
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 10:43:22 am
I went into FCEUX Nametable Viewer and checked the hex code of the letters showing up in dialogue text. The hex code for the letters in dialogue was exactly the same as the code for the letters in monster, item and spell names--with all of the latter showing up in a hex editor when I load up table data. Would this indicate compression or could another factor be causing this?

I don't think simply looking at the nametable would indicate compression per se. The way I would do it is the more advanced method of debugging the game to see how the text gets from the ROM to the screen, and compression will become apparent then. A simpler way is the old fashioned relative search technique locating where the text is, and discovering the compression then.

I'll have a look later, anyway. I've already hacked this game to double experience and gold, so that's why I'm familiar with it.
Title: Re: Handling possible text compression
Post by: KingMike on September 20, 2018, 12:50:32 pm
I know Dragon Warrior II and III used ROM expansion so I wouldn't expect something beyond dictionary, because what's the point once they're expanding the ROM.

Yes, in DW4 they used Huffman which is tricky but that's because they couldn't really expand further. SOROM or whatever it was (the 512KB MMC1 mapper) was already about the limit.
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 06:00:12 pm
Well, you were right: DW2 uses a rather tricky compression method which I don't fully understand yet. I'm still looking at it, and it's not like what I've seen before. I'd say it's a bit like dictionary compression, except not... I'll continue until I've figured it out, but it's weird.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 20, 2018, 07:40:20 pm
Thank you so much for diving into this Psyklax. Why did those knuckleheads at Nintendo of America feel the need to give us this headache with all that free space at their disposal.  :banghead:
Title: Re: Handling possible text compression
Post by: Alchemic on September 21, 2018, 01:36:17 am
I did some digging around in DW2 a few years back. My notes are kind of a mess, but to summarize:
Code: [Select]
TBL for 0xB49B:
00-09 = 0-9
0A-23 = a-z
24-3D = A-Z
3E-57 = a-z again
58=A
59=(space)
5A=[SUN]
5B=[STAR]
5C=[MOON]
5D=[DROP]
5E=[HEART]
5F=(space)
60=(space)
61='
62="
63=->
64="
65='
66='
67='
68=.'
69=,
6A=-
6B=.
6C=&
6D=(space)
6E=?
6F=!
70=;

Text-dumping code in Python (https://github.com/Osteoclave/game-tools/blob/master/nes/dragonwarrior2_textdump.py)
Title: Re: Handling possible text compression
Post by: Psyklax on September 21, 2018, 03:56:14 am
Wow, Alchemic, that confirms what I saw: that it's pretty messed up. :D I got as far as noticing that each individual word is assembled letter-by-letter in SRAM before being passed off to the PPU, but it would've taken me time to get to the level of detail you've acquired. That has to be the most convoluted text handling scheme I've ever seen in a game.

I had a look so I can see what you're talking about there. It's still messing with my brain, so I think I'll leave it for now. :)
Title: Re: Handling possible text compression
Post by: abw on September 21, 2018, 08:57:00 am
Maybe later today I'll be able to spend five minutes checking the text to see how it's stored in the ROM.
If you actually had cracked that code in 5 minutes, you would have had my vote for the annual "God of ROMHacking" award :P.

I did some digging around in DW2 a few years back. My notes are kind of a mess, but to summarize:
[...]
Text-dumping code in Python (https://github.com/Osteoclave/game-tools/blob/master/nes/dragonwarrior2_textdump.py)
Nice work Alchemic! I'm sure somebody's next question is going to be: did you also happen to write an insertion script?

That has to be the most convoluted text handling scheme I've ever seen in a game.
The basic input -> output encoding is pretty simple, but I definitely agree that DW2's text engine could have been implemented in a much less complicated way. It took me a couple of days to work through it enough to get a decent script dump, but I can confirm that Alchemic's notes are indeed accurate.

I also found it strange that the game would use compression--especially since there is a ton of empty space in the US rom.
Why did those knuckleheads at Nintendo of America feel the need to give us this headache with all that free space at their disposal.  :banghead:
It's not always a question of how much free space in total, but where that free space is located. DW2's script is large enough that even with the compression scheme they added, it still doesn't fit inside a single PRG bank, which is why the script jumps from 0x17FE6 to 0xB7C2. Maybe they were trying to keep almost all of the text-related code and data confined to 2 PRG banks?
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 24, 2018, 07:04:44 am
@abw got it right. I'm definitely asking for someone to help me write a script for this. Wouldn't I need two scripts: one to uncompress the text into readable / editable form and another to recompress for reinsertion? I'm willing to keep the maximum length of each line the same if it means not having to worry about readjusting all the pointers.

In the mean time, I am keeping myself busy figuring out the best way to edit the monster, spell and item lists but I would be so very grateful to have a means of moving forward with the script one day soon.

Title: Re: Handling possible text compression
Post by: abw on September 24, 2018, 09:57:45 pm
@abw got it right. I'm definitely asking for someone to help me write a script for this. Wouldn't I need two scripts: one to uncompress the text into readable / editable form and another to recompress for reinsertion? I'm willing to keep the maximum length of each line the same if it means not having to worry about readjusting all the pointers.
If all else fails, abcde (http://www.romhacking.net/utilities/1392/) can definitely handle both extracting and inserting text like this along with updating the pointers, and you'll have all the original space at your disposal (there's no free space conveniently available after the original script ends, since the very next byte is code, so if you need more space than the original, it'll take a bit more work). As an added bonus, thanks to the original encoding being somewhat inefficient, abcde can actually re-insert the original script using $79 fewer bytes.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 25, 2018, 08:42:31 am
@abw

Am I to understand that your software can already handle unwinding the existing compression routine in DW2 without the need for a special script being written? Also, if I am to use the more efficient form of compression available through your software, it will automatically rework the code of the game to be able to read the new compression routine?

Something tells me it will be nowhere near this easy.  :laugh:
Title: Re: Handling possible text compression
Post by: abw on September 25, 2018, 06:10:59 pm
Am I to understand that your software can already handle unwinding the existing compression routine in DW2 without the need for a special script being written?
[...]
Something tells me it will be nowhere near this easy.  :laugh:
I'm not going to make any promises about how easy using abcde will be for anybody else, but... basically, yeah. Like I said earlier, the text encoding itself is actually pretty simple, it's just that DW2's code for parsing that encoding is fairly unpleasant. Give abcde the right table file, extract command file, and ROM, and watch it go. Stick a couple of extra lines in the script dump file it generates to turn it into a valid insert command file, and then run abcde again with the same table file, the insert command file (edited to have your new text, if so desired), and (a copy of) the ROM. That said, unless you start making ASM changes, you're still constrained by the total available space and text engine and stuff, so it's not a magic bullet.

Also, if I am to use the more efficient form of compression available through your software, it will automatically rework the code of the game to be able to read the new compression routine?
Haha, don't I wish it was that good :P. abcde doesn't make any changes to the game's code, it just uses a better insertion algorithm. The original data compression was sub-optimal; as one example, the game stores the string "'Welcome " as ['W][e][l][come][ ], with those 5 pieces taking up 10 + 5 + 5 + 10 + 5 = 35 bits. On the other hand, abcde is smarter and inserts the same string as ['][Welcome ] in 5 + 10 = 15 bits, for a savings of 20 bits. Little improvements like that that save 5 bits here, 10 bits there over the course of the entire script is how we end up with 121 bytes of reclaimed space.
Title: Re: General Hacking Questions
Post by: Chicken Knife on November 29, 2018, 08:09:01 am
I thought I'd bring this thread back to life as I'm back into my projects consistently and have a few things I need help with.

I'm nearly done with my Dragon Warrior 1 hack and my only issue left is that I want to fix some compromises I had to make porting over NPC sprite graphics from the famicom version due to left/right mirroring instructions in the DW1 for certain sprite tiles.

Does anyone have any good reading material to recommend that will present in a way that does a good job explaining itself to a non coder that will help me understand "sprite pointers" (if I'm using the right term) and will help me understand how to either disable the LR mirroring instructions I’m struggling with or simply redirect the game to alternative tiles for that sprite? If I can "repoint" them effectively like I do with text, I will have a ton of extra space since I turned all the 4 direction sprites into 1 direction sprites.

There are also other topics where I would like to do some reading and am having a hard time because I either can't find the resources or the resources I find are too technical:

I encountered a multiplier formula in Dragon Warrior 3 where the game takes the base Japanese gold and experience numbers saved for monsters saved in the IS game but applies a .25% boost, rounded up, to those values for what you actually earn in battle. I want to locate and disable that multiplier, restoring the straight Japanese values in gameplay. I assume a debugger would help me here. If so, any good reading material (or better yet videos) you guys would recommend for effectively using the FCEUX debugger for something like this?

Last, I've tried to play around with Atlas / Cartographer, ABCDE software in hopes of solving the DW2 script compression issue. The faq that comes with Atlas is too technical for me at this point and I haven't found much else out there. Any recommended reading?

Thanks guys!
Title: Re: General Hacking Questions
Post by: abw on November 29, 2018, 08:34:16 pm
Does anyone have any good reading material to recommend that will present in a way that does a good job explaining itself to a non coder that will help me understand "sprite pointers" (if I'm using the right term) and will help me understand how to either disable the LR mirroring instructions I’m struggling with or simply redirect the game to alternative tiles for that sprite?
Maybe I shouldn't comment since I don't usually do much graphics work, but when I do, I usually end up consulting Y0SHi's NES Documentation (http://www.romhacking.net/documents/120/), and although I'm pretty sure I originally had to read it a few times before it sunk in, it hasn't steered me wrong yet and it does have a couple of sections that talk about sprites.

Last, I've tried to play around with Atlas / Cartographer, ABCDE software in hopes of solving the DW2 script compression issue. The faq that comes with Atlas is too technical for me at this point and I haven't found much else out there. Any recommended reading?
Did you try abcde's readme or examples? Be honest - how bad are they really? I'm waaaay too deeply immersed in the source material to feel confident about my ability to pitch the documentation at a level that is useful without making other people barf, plus I implicitly assume that the end user is already familiar with Atlas/Cartographer :(.

I might have some recommended reading for you, though (based on abcde v0.0.2; provide your own "Dragon Warrior II (U) [!].nes") ;D:
Extraction:
Quote from: Cartographer.txt
#GAME NAME:      Dragon Warrior II (U) [!].nes

#BLOCK NAME:      Main Script, Part 1
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $B762
#POINTER TABLE STOP:   $B7BD
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#AUTO JUMP START:   $17FE7
#AUTO JUMP STOP:   $B7C2
#ATLAS PTRS:      Yes
#BASE POINTER:      $C010
#TABLE:         dw2_script.tbl
#COMMENTS:      Yes
#END BLOCK

#BLOCK NAME:      Main Script, Part 2
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $B7BE
#POINTER TABLE STOP:   $B7C1
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#ATLAS PTRS:      Yes
#BASE POINTER:      $10
#TABLE:         dw2_script.tbl
#COMMENTS:      Yes
#END BLOCK

Quote from: dw2_script.tbl
# NB: I didn't put much time into figuring out the control codes, so some of them are still unknown and some of them might be wrong. Caveat emptor!

/%00000=[end]\n\n
/%00001=.[end]\n\n
/%00010=?[’ ][FD][FD][end]\n\n
/%00011=[.’][end]\n\n
%00100=[FF]
%00101=y
%00110=c
%00111=o
%01000=d
%01001=e
%01010=f
%01011=g
%01100=h
%01101=i
%01110=j
%01111=
%10000=l
%10001=m
%10010=n
%10011=[line]\n
%10100=[.’]
%10101=[ ‘]
%10110=r
%10111=s
%11000=t
%11001=u
%11010=a
%11011=w
%11100=C0
%11101=C1
%11110=C2
%11111=C3

%1110000000=A
%1110000001=B
%1110000010=Ca
%1110000011=D
%1110000100=E
%1110000101=F
%1110000110=G
%1110000111=H
%1110001000=I
%1110001001=J
%1110001010=King
%1110001011=L
%1110001100=Moonbrooke
%1110001101=N
%1110001110=O
%1110001111=[item]
%1110010000=The
%1110010001=Rhone
%1110010010=S
%1110010011=;
%1110010100=U
%1110010101=”
%1110010110=Water Flying Cl
%1110010111=C
%1110011000=Y
%1110011001=Z
%1110011010=x
%1110011011=Village
%1110011100=z
%1110011101=[F9]
%1110011110=‟
%1110011111=K

%1110100000=v
%1110100001=q
%1110100010=[’ ][wait][line]\n
%1110100011=R
%1110100100=.
%1110100101=[FD][FD]
%1110100110=P
%1110100111=b
%1110101000=T
%1110101001=!
%1110101010=[sun]
%1110101011=[star]
%1110101100=[moon]
%1110101101=W
%1110101110=k
%1110101111=p
%1110110000=?
%1110110001=,
%1110110010=[monster]
%1110110011=....
%1110110100=:
%1110110101=[’ ]
%1110110110=-
%1110110111=[’ ]
%1110111000=[spell]
%1110111001=[letter]
%1110111010=[no voice]
%1110111011=[wait]
%1110111100=M
%1110111101=[name]
%1110111110=[number]
%1110111111=[FD]

%1111000000=Thou hast
%1111000001=hest
%1111000010=Midenhall
%1111000011=hou
%1111000100= of
%1111000101= is
%1111000110= thou has
%1111000111= and
%1111001000=to th
%1111001001= thee
%1111001010=ast
%1111001011= do
%1111001100=hat
%1111001101= shall
%1111001110= was
%1111001111=hou has
%1111010000=d the
%1111010001= has
%1111010010=gon
%1111010011=.[wait][line]\n
%1111010100= have
%1111010101=come to
%1111010110=ing
%1111010111= hast
%1111011000=ost thou
%1111011001=this
%1111011010= of the
%1111011011=Hargon
%1111011100=in the
%1111011101=thing
%1111011110=he
%1111011111= with

%1111100000=reasure
%1111100001=[ ‘]Hast
%1111100010=Erdrick
%1111100011=come
%1111100100=ere is
%1111100101=Welcome
%1111100110=rince
%1111100111= great
%1111101000=arr
%1111101001= for th
%1111101010=piece[(s)] of gold
%1111101011=[.’][wait][line]\n
%1111101100=But
%1111101101=here
%1111101110=can
%1111101111=ove
%1111110000=hee
%1111110001=not
%1111110010=for
%1111110011=one
%1111110100= any
%1111110101= to
%1111110110=descendant
%1111110111=Roge Fastfinger
%1111111000=all
%1111111001=thy
%1111111010=[ ‘]W
%1111111011=thank thee
%1111111100= it
%1111111101= tha
%1111111110= thou
%1111111111= the

Usage (replace \path\to\abcde\abcde.pl as appropriate):
Code: [Select]
perl \path\to\abcde\abcde.pl -m bin2text -cm abcde::Cartographer "Dragon Warrior II (U) [!].nes" Cartographer.txt Cartographer_out -s

Insertion:
Quote from: Atlas.txt
// Define, load, and activate a TABLE
#VAR(Table, TABLE)
#ADDTBL("dw2_script.tbl", Table)
#ACTIVETBL(Table)

// Jump to start of script
#JMP($14010)
#HDR($C010)
// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
#AUTOCMD($17FE7, #HDR($10))
#AUTOCMD($17FE7, #JMP($B7C2, $BE0F))

// the rest of Cartographer_out.txt goes here

Quote from: insert.bat
@copy /Y "Dragon Warrior II (U) [!].nes" "Dragon Warrior II (U) [new].nes" > nul

perl \path\to\abcde\abcde.pl -m text2bin -cm abcde::Atlas "Dragon Warrior II (U) [new].nes" Atlas.txt

pause
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on November 30, 2018, 02:54:28 am
@ abw

Wow! Talk about a silver platter  :beer:

I've done all the steps you've outlined but I'm getting this message after inputting: (from within abcde.pl's directory)
perl abcde.pl - bin2text -cm abcde::Cartographer "Dragon Warrior II - Edit.nes" Cartographer.txt Cartographer_out -s

Message after inputting:

error UTF-8 "\x92" does not map to Unicode at C:/Perl164/lib/Encode.pm line 228, <TABLE> line 3
when reading C:\Rom Editing\dw2_script.tbl line 3

I don't think me using the altered name of my hacked rom file would matter. Let me know if it does.
As far as Perl, I downloaded the software today in order to get this going. ActivePerl 5.26.1.2601 MS Win32-x64-404865 is the version.

PS

As far as Yoshi's NES doc, I think it's written far more for someone like you than someone like me. The density of it loses me in the same way as most of what I try to read on the NES Dev site. Truth be told, your abcde faq is much easier to follow, though I'm far from understanding all of it and it definitely does assume I understand Atlas / Cartographer. As far as the Yoshi faq, I'll have to keep coming back to this and whatever else I find until I figure out how to do the graphical work I need to.
Title: Re: General NES Hacking Questions
Post by: abw on November 30, 2018, 08:36:29 am
@ abw

Wow! Talk about a silver platter  :beer:
Heh, I had it handy from earlier in the thread. I used that stuff to make the King of Moonbrooke say "Oh noes, not the flying purple people eaters!" during the intro scene in order to test inserts :P.

error UTF-8 "\x92" does not map to Unicode at C:/Perl164/lib/Encode.pm line 228, <TABLE> line 3
when reading C:\Rom Editing\dw2_script.tbl line 3
Ah, sorry, I forgot to mention that you'll need to save the files (particularly the table file) encoded as UTF-8. Any modern text editor should give you that option when you save a file; even Notepad can save as UTF-8!

As far as Yoshi's NES doc, I think it's written far more for someone like you than someone like me. The density of it loses me in the same way as most of what I try to read on the NES Dev site.
Yeah, there's definitely a lot to take in, but knowing what's going on at the hardware level, what the different sections of PPU RAM control, and what all those writes to $20xx are doing can be pretty helpful when you're trying to track down some graphics logic.

Truth be told, your abcde faq is much easier to follow
Huzzah, mission partially accomplished!
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on November 30, 2018, 07:33:03 pm
@ abw

Worked like a charm.  :woot!:

I assume that a similar approach to how I edit text in hex editors would serve me well. I plan on working within the length of the existing strings, using blank entries at the end of lines of text if I happen to shorten in order to keep the length exactly the same. The thing that concerns me is the presence of so many control codes in DW2. As I go through the extraction document, I see the presence of plenty of those words in the script, but I don't see any indication that those words are produced by a control code in the document. I can keep Roge Fastfinger in mind easily enough, but with a ton of other common control code words like he, with, can, one, any etc. there is no way I'm going to be able to keep track of them all. Also, how do I know that the game consistently used control codes for those words instead of using them sporadically? It seems like it will be inevitable that I'll end up with a different data size and a considerable headache. Perhaps I could use the find command to highlight every instance of them in the text before I start editing in order to be wary of carefully maintaining string length. Any advice would be appreciated.

Also, going back again to Roge Fastfinger, I will be reverting his name to the Japanese Lagos. Is there any simple way to edit the control code words so I could swap out the name and use the same code? If not, I suppose I'd have to seriously abbreviate those lines to compensate.

Also, when reinsertion time arrives, does your system automatically note every control code word and insert them back into the game as the single byte control code instead of individual letter data? I'm curious.

Last question: I've heard on occasion that this kind of software automatically readjusts the text pointers and I could therefore enlarge some strings of text as long as the total amount of data is the same upon insertion. That seems too good to be true. Could it be?

Let me know your thoughts. I'm sure there is a lot I'm conceptually missing here.
Title: Re: General NES Hacking Questions
Post by: abw on December 01, 2018, 12:13:49 am
Worked like a charm.  :woot!:
Whoo-hoo!

I assume that a similar approach to how I edit text in hex editors would serve me well.
I plan on working within the length of the existing strings, using blank entries at the end of lines of text if I happen to shorten in order to keep the length exactly the same.
It seems like it will be inevitable that I'll end up with a different data size and a considerable headache.
*shudders* Nope, nope, and... wait for it... nope :P.
The fastest way is probably just to try it and see, but basically when you use a (decent) script insertion utility, you don't have to worry too much about things like the binary it inserts or updating pointers, since the utility takes care of all of that for you. Like I mentioned earlier, you're still constrained by the total available space and text engine and stuff like that, but as long as you don't try to insert impossible (combinations of) characters or too much text, you can do more-or-less whatever you want.

Line lengths will change; that's okay and you don't need to care about it.
Some text will get encoded as single characters, some as multiple characters; that's okay and you don't need to care about it.
String addresses will change and pointers to those strings will get updated; that's okay and you don't need to care about it.

Last question: I've heard on occasion that this kind of software automatically readjusts the text pointers and I could therefore enlarge some strings of text as long as the total amount of data is the same upon insertion. That seems too good to be true. Could it be?
Automatically recalculating pointer values is just one of the many benefits of using a script insertion utility over hex editing :).

Also, going back again to Roge Fastfinger, I will be reverting his name to the Japanese Lagos. Is there any simple way to edit the control code words so I could swap out the name and use the same code?
Yup. It's not quite as simple for DW2 as it would be for a game with a less ornery text engine, but it's still not too hard. The dictionary lives at ROM 0xB44B-0xB686, with each nybble (1 nybble = half a byte) of 0xB44B-0xB49A giving the length of the corresponding dictionary entry in 0xB49B-0xB686. So you could change "Roge Fastfinger" to "Lagos" at 0xB655, shift the rest of 0xB664-0xB686 up to just after the end of "Lagos", and then change the length of the "Roge Fastfinger" dictionary entry from 15 to 5 at 0xB496 (i.e. 0xAF -> 0xA5), leaving you with 10 bytes of unused space at 0xB67D-0xB686. Just make sure to update your table file to match whatever changes you make to the game's text encoding, or you're going to get a nasty surprise when you try extracting or inserting text!

Also, when reinsertion time arrives, does your system automatically note every control code word and insert them back into the game as the single byte control code instead of individual letter data? I'm curious.
Based on the table file(s) you provide, abcde is aware of all the possible ways of translating your text into the binary that the game needs, and it will convert your text into the shortest possible binary that represents your text. How? Magic :angel:.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 02, 2018, 04:10:31 pm
@abw

Ok, I worked furiously throughout the weekend on my script. Having just completed it, I ran the insertion routine, loaded up the game, and found the text to be totally mangled. Everyone is saying something different than what they should be saying. I don't think my data overran the alotted space. In general a direct translation of the famicom is shorter than the US version. I assume I would have gotten some kind of error message if it did.

The other possibility in my mind is that I was supposed to totally sort out the control codes prior reworking the script, which I did not do. Is that what you would suspect is the issue or do you have another idea?

I have a frightening suspicion that if control codes are the issue, I would have saved myself tremendous work prior to extraction and reworking the script.

Also, there were some instructions in the Atlas data that you quoted me where I wasn't sure if they were written to me or the program. For example:

// auto-commands for when DW2 does a mid-string bankswap and resets its read address:

PS I don't know if this helps, but all of the dialogue from everyone seems to be various battle commands

*EDIT

If it helps at all, this is the Atlas file where I tried removing what looked like your instructions from me and added my game script underneath. Take a look.

https://www.dropbox.com/s/t06zcf8qmmfhsno/Atlas.txt?dl=0
Title: Re: General NES Hacking Questions
Post by: abw on December 02, 2018, 10:52:18 pm
Ha, that's me skipping over important little details again :-[ (in my defence, they are documented). In my extract script, I set #COMMENTS: Yes, which makes every line of the script be output as a comment and thus ignored during insertion. So, since all the text is a comment, the only thing that actually changed would have been the pointers, and every pointer in the first block of pointers would point to the same first string starting at 0x14010, which happens to contain battle text. The second block of pointers would also point to the same RAM address ($8000), but with the wrong ROM bank visible in RAM, so I imagine any attempt to display one of those strings would end badly.

If you're only making a few changes to the script, you might prefer setting #COMMENTS: No instead and editing the text that way, or reformat the script dump using your text editor of choice (when translating, I like keeping the original text for reference and adding my new text just below); a simple regular expression search-and-replace operation or two should do the trick, or in the worst case you could manually delete the // before each line of script.

Also, there were some instructions in the Atlas data that you quoted me where I wasn't sure if they were written to me or the program. For example:

// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
Any line in the extract or insert command files that starts with // is treated as a comment, so the ones I added are just notes to explain what's going on. In this case, DW2 has some code that says "when the current read address reaches $BFD7, swap in a different bank and keep reading from $B7B2", so the #AUTOCMDs are there to handle that during insertion.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 02, 2018, 11:33:46 pm
@abw

As far as leaving out that bit, you are my patron saint of hacking. No grievance shall ever be held against you.  :laugh:

Ok. So if comments would have been set to No, then all the lines of text would have inserted as intended I assume. If I understand correctly, leaving comments set to Yes would be convenient because I could write the legitimate lines of text underneath them without the // and insert the data back in the rom effectively even with the old script appearing as comments.

So in my case, I took the output document and totally changed 98 percent of the text based on various resources to reflect a literal but fluid version of the Japanese text in contemporary English. I haven't slept much lately--and my wife may want to divorce me but that's neither here nor there. :D

So what would you suggest I do now? I often don't mind doing things the slow and painful way. Lifelong RPG gamer after all  :laugh: I'm thinking to go through the document and delete every instance of the characters // and leave whatever text comes after them. Would that do the job for reinsertion or do I have to add anything in their place?

There is no way I'm starting over with a new extraction. Somehow or another, this document has to get formatted correctly.



December 04, 2018, 02:07:08 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Ok. That was a question I could really answer myself through a little experimentation. I ran another extraction and just confirmed which // characters to remove.

Having done so, I reinserted and about 3/4 of my text was inserted correctly! I'll consider that a small victory haha. The other 1/4 text is either the games original text or bugged.

The error message I got in command prompt when I did the insertion was:

unable to tokenize; best attempt failed at input position 513 at C:\Rom Editing/abcde/Table/Table.pm Line 421 <COMMAND_FILE> line 1707. in text string starting at Atlas.txt line 1665.

I immediately thought this was an issue with my insert text. I went to that line (with word wrap turned off) and found there is nothing unusual at all about it. Line 1665 reads:

[ ‘]This is a sewing shop[.’][line]
Title: Re: General NES Hacking Questions
Post by: abw on December 04, 2018, 08:03:38 pm
As far as leaving out that bit, you are my patron saint of hacking. No grievance shall ever be held against you.  :laugh:
Heh, be careful what you promise >:D

I haven't slept much lately--and my wife may want to divorce me but that's neither here nor there. :D
I'm not going to admit to knowing what that feels like :D.


I took a closer look at this and noticed a couple of things:
- The dictionary does not include uppercase Q, V, or X as displayable characters, so you can't use those anywhere in your script without changing the dictionary. That unreadable mess of garbage just before the final error message abcde gave is supposed to show you how far along abcde got in the problem string before it couldn't go any farther, and as it happens 514 characters into the string starting at line 1665 is a Q. (yeah, the error messages suck)
- Some of the dictionary entries display different visually identical tiles, but I made them the same in my table file. In my translation of DW1, I originally made the mistake of assuming that just because the graphics were identical, it didn't matter which of the different tile IDs I used; this led to a moment of intense dismay when I realized that I had managed to accidentally disable the "speech" text printing sound effect!
- You did indeed make a lot of changes - replacing all the "thee"s and "thou"s and changing some proper nouns meant that a lot of the original dictionary entries went unused during insertion, which resulted in your new script not compressing small enough to fit in the available space. A few ad hoc tweaks to the dictionary was enough to sort that out and leave you with 218 bytes to spare; if you end up needing more room, updating the dictionary based on a full frequency analysis should give you much better compression results than me just eyeballing things.

Here's a copy of the updated insert script (https://drive.google.com/open?id=1rrX177cE49fE2R0CYk9CXhYgGJqmNgQZ) and table file (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) I used. I took the liberty of merging the original text into your new text as comments and fixing up a couple of typos that were breaking insertion; spell checking the new script is your responsibility :P. For the table file changes, you'll need to make the corresponding changes to the dictionary like I described a couple of posts ago (http://www.romhacking.net/forum/index.php?topic=27053.msg366909#msg366909).
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 05, 2018, 06:29:19 am
@abw, you didn't need to do all this work for me! I'm extremely grateful but I really don't want to be a burden here (Although I imagine watching me flop around like a fish out of water can be equally burdensome)

I think you've given me everything I need to do this insertion and further improvements to my script--except for one thing that I need clarification for.

In order to update the rom's dictionary with the new words and nybble lengths as you instructed above, I need you to clarify how I actually locate where to plug in the nybble that corresponds to each dictionary entry. Either I missed it or you didn't provide the detail as to how you knew 0x0B496 corresponded to the entry for Roge Fastfinger / Lagos. I'm not sure if this uses a counting system like the item/monster/spell lists. Hopefully not since that would give me a big headache as the entries here seem all mashed up together.
Title: Re: General NES Hacking Questions
Post by: abw on December 05, 2018, 07:35:57 pm
@abw, you didn't need to do all this work for me! I'm extremely grateful but I really don't want to be a burden here (Although I imagine watching me flop around like a fish out of water can be equally burdensome)
If it makes you feel any better, only a little bit of that work was purely for you ;). I'm slowly puttering away at my own translation whenever the mood strikes me, so a lot of the issues you're encountering now are issues I would no doubt encounter myself at a later date. Might as well deal with them now so that we can both benefit (and judging from the read count on this topic, I suspect it's more than just you and I that are interested in this).

I think you've given me everything I need to do this insertion and further improvements to my script--except for one thing that I need clarification for.
The lengths of the dictionary entries are stored in nybbles at 0xB44B-0xB49A and the values are stored in bytes at 0xB49B-0xB686; the table file I provided lists the dictionary entries in the same order they appear in ROM, and the table file you were using to edit the item/monster/spell lists will also let you see the dictionary values correctly. An example from partway through the dictionary might be helpful to see what's going on:
Code: [Select]
...
0x00B45B|$02:$B44B:11                            ; length of "A", length of "B"
0x00B45C|$02:$B44C:21                            ; length of "Ca", length of "D"
0x00B45D|$02:$B44D:11                            ; length of "E", length of "F"
0x00B45E|$02:$B44E:11                            ; length of "G", length of "H"
0x00B45F|$02:$B44F:11                            ; length of "I", length of "J"
0x00B460|$02:$B450:41                            ; length of "King", length of "L"
0x00B461|$02:$B451:A1                            ; length of "Moonbrooke", length of "N"
...
0x00B4C1|$02:$B4B1:24                            ; A
0x00B4C2|$02:$B4B2:25                            ; B
0x00B4C3|$02:$B4B3:26 0A                         ; Ca
0x00B4C5|$02:$B4B5:27                            ; D
0x00B4C6|$02:$B4B6:28                            ; E
0x00B4C7|$02:$B4B7:29                            ; F
0x00B4C8|$02:$B4B8:2A                            ; G
0x00B4C9|$02:$B4B9:2B                            ; H
0x00B4CA|$02:$B4BA:2C                            ; I
0x00B4CB|$02:$B4BB:2D                            ; J
0x00B4CC|$02:$B4BC:2E 12 17 10                   ; King
0x00B4D0|$02:$B4C0:2F                            ; L
0x00B4D1|$02:$B4C1:30 18 18 17 0B 1B 18 18 14 0E ; Moonbrooke
0x00B4DB|$02:$B4CB:31                            ; N
...
So if you wanted to, say, change the dictionary entry for King to say Queen instead, you'd change the length stored at 0xB460 from 41 to 51, change King to Queen starting at 0xB4CC, and shift everything from 0xB4D0 down by one byte. The dictionary has a fixed length and is followed immediately by code though, so in a case like this where you added a byte, you would need to remove a byte from somewhere else in the dictionary (pick "Water Flying Cl", it's only used 4 times in the entire script!).
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 05, 2018, 08:04:54 pm
@abw knowing that you tend to respond around this time I was in a big rush to message that I actually figured this out myself (well not really but still), finding that it was indeed a similar counting mechanism and making a little chart for myself just to make sure I understood it. Exhibit A: the little chart I was working on. hahaha

B490   piece(s) of gold   ???
B491   But[ ]         here
B492   can[ ]         ove
B493   hee         not
B494   for         one
B495   [ ]any         [ ]to[ ]
B496   Descendant       Lagos      
B497    all         thy      
B498   ?W         thank thee      
B499   [ ]it         [ ]thou      
B49A   [  ]the.      4

December 05, 2018, 08:07:19 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Now that I get the mechanics of this, I'm going to optimize the dictionary entries to match up to my translation trends as much as possible, which should allow me to expand text a little more as I go through the revision process should I desire.

I'm interested in what you are planning translation wise--more Latin for DW2? Also, now that we are discussing your translation, I'm curious what your textual basis was: the original US English translation or the Japanese?
Title: Re: General NES Hacking Questions
Post by: abw on December 07, 2018, 11:08:02 pm
Exhibit A: the little chart I was working on. hahaha
Haha, I see that's a nice little chart you have there :thumbsup:.

I'm interested in what you are planning translation wise--more Latin for DW2? Also, now that we are discussing your translation, I'm curious what your textual basis was: the original US English translation or the Japanese?
Yup. It's a dirty job, but somebody's got to do it :P. I generally use the English version of whatever I'm translating as my base, but will consult other English versions or other languages if the need arises; this happens most often for made-up enemy/item names, but sometimes I come across a line or two of dialogue that I like to clarify because it's a little too vague or allows for multiple valid interpretations.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 08, 2018, 11:13:07 pm
Ok, I have a report that is pretty darned good. While I was initially having some problems achieving a successful insertion with the updated Atlas.txt and table file you provided, I was able to narrow down the issue to one small mistake I made as I heavily edited the dictionary where a character length was off by one. Once I fixed that, I was able to do an insert that is 99% bug free (at least according to the appx 30% of game text that I've tested so far.) Here is the bug that's jumping out at me. The King of Cannock/Samaltria has some bugged text prior to his normal text. See the two pics below. The text seems to be from Hargon's Temple after you use the Charm/Eye of Rubiss to dispel the illusion

https://www.dropbox.com/s/txzrjqisvzwumuc/Dragon%20Warrior%20II%20-%20Edit_002.png?dl=0
https://www.dropbox.com/s/syx9sowi42u3j03/Dragon%20Warrior%20II%20-%20Edit_003.png?dl=0

After those lines, again he returns to his normal text, allows you to save, and remarks about his son. Very peculiar and I hope you have an idea how to fix it because I have no idea. Let's see if I find anything else like this, but I'm relieved that everything else has pointed to the right text so far. Here are links to my relevant files if you want to look at them regarding this issue. FYI my Atlas.txt script is significantly revised since I last sent it.

https://www.dropbox.com/s/5qo09huap2a5y2t/dw2_script.tbl?dl=0
https://www.dropbox.com/s/yn3qj00b536dxqt/Atlas.txt?dl=0
https://www.dropbox.com/s/fns0v7z1o6i8nxm/dw2_delocalized_052.ips?dl=0

One other thing. Regarding your Latin Translation project, are you planning to include any kind of uncensoring element? If you were I would be most grateful to enlist your efforts on that front. As you can see if you look in my patch I restored the Japanese church crosses and sprite for the priest. Unfortunately however, the priest is assigned the color palette of Princess Moonbrooke instead of what should be the Prince Midenhall/Laurasia palette. And then there's the matter of replacing the ghosts with original coffins but let me know if you're even interested in this graphical stuff. I must say that your long deceased Roman audience probably wouldn't have approved much of censorship--though the Latin speaking clergy would have demanded it I'm sure.  :D

*** Update

After my newest round of updates, King Samaltria's Kee Kee text went away. :o This is after it stuck around through the last 3 updates or so. I don't get it! Seems more like a bug in the game itself than a bug from the insertion

Title: Re: General NES Hacking Questions
Post by: abw on December 09, 2018, 01:58:13 pm
The King of Cannock/Samaltria has some bugged text prior to his normal text. See the two pics below. The text seems to be from Hargon's Temple after you use the Charm/Eye of Rubiss to dispel the illusion
[...]
After my newest round of updates, King Samaltria's Kee Kee text went away. :o This is after it stuck around through the last 3 updates or so. I don't get it! Seems more like a bug in the game itself than a bug from the insertion
This turns out to be a little bit of column A, a little bit of column B.

On the insert side, the game has a hardcoded value that tells it which pointer within the pointer table at ROM 0xB762-0xB7C1 starts reading from ROM bank 2 instead of ROM bank 5, but the insert script didn't update that value, so the pointer at ROM 0xB7BE started reading text from the wrong ROM bank, which threw off the end token counting before the script reached RAM $BFD7 and resulted in the wrong string being displayed. This is easily fixable by adding a COUNTER variable and an AUTOCMD for writing the counter like in the Dragon Quest IV example that ships with abcde, except Dragon Warrior II's script is only spread across 2 ROM banks instead of 6, so it's easier for II than IV.
Code: [Select]
// add this near the top of the insert script:
#VAR(pointerNum, COUNTER) // create a COUNTER variable named pointerNum
#CREATECTR(pointerNum, 8, 0) // pointerNum is an 8-bit value initialized to 0
#AUTOCMD($17FE7, #WLB(pointerNum, $3FA90)) // update the code that controls which pointer starts the next bank

// and then after every #W16 line in the insert script, add:
#INC(pointerNum, 1)

On the game side, there is a bug where the game takes the pointer to the start of the desired group of 16 strings and reads the first 2 bytes from the pointer's target *before* it runs through the code for checking whether the current string address has reached RAM $BFD7, so if you're unlucky enough to have a pointer point to RAM $BFD6 or $BFD7 like happened with your linked insert script, the text for that pointer gets seriously screwed up. There's currently no way for abcde to handle that situation for you automatically, so unfortunately you're just going to have to keep an eye on the pointer values, and if you see $BFD6 or $BFD7 come up, you can knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6 respectively to compensate. It means you lose 1 or 2 bytes of script space, but that's probably easier to deal with than making the ASM changes required to fix the bug in the original game.

Both of these issues only show up depending on where the dividing line between ROM banks 5 and 2 falls in your new script compared to the original script, which explains why they would come and go as you continue updating and re-inserting your script.

One other thing. Regarding your Latin Translation project, are you planning to include any kind of uncensoring element? If you were I would be most grateful to enlist your efforts on that front. As you can see if you look in my patch I restored the Japanese church crosses and sprite for the priest. Unfortunately however, the priest is assigned the color palette of Princess Moonbrooke instead of what should be the Prince Midenhall/Laurasia palette. And then there's the matter of replacing the ghosts with original coffins but let me know if you're even interested in this graphical stuff. I must say that your long deceased Roman audience probably wouldn't have approved much of censorship--though the Latin speaking clergy would have demanded it I'm sure.  :D
I generally try to stick pretty close to the source material for various reasons (e.g. nostalgia, insufficienct fluency in Japanese, etc.), so as a rule of thumb, if my source was censored, so is my translation; if my source was not censored, neither is my translation. I can do graphics hacking if I have to, but while I appreciate the results of other people's graphics hacks, making my own just doesn't have the same appeal as translating does.

As for the ancient Romans, well, they were no strangers to censorship ("censor" itself is a Latin word), but I think a bigger problem there would be the lack of hardware and compatible power sources to play the translation on :P.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 09, 2018, 02:31:57 pm
Thank you for looking at this!

While inserting the lines you quoted in my Atlas.txt are easy enough, if I am to undertand correctly, even with the COUNTER variable and AUTOCMD added, there is still a possibility of seeing these kinds of issues pop up. This leads me to the second fix:

if you see $BFD6 or $BFD7 come up, you can knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6 respectively to compensate. It means you lose 1 or 2 bytes of script space, but that's probably easier to deal with than making the ASM changes required to fix the bug in the original game.

I don't exactly understand the above instructions. Could you explain to me a little more clearly what you mean by "knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6"? First, how do I see when $BFD6 or $BFD7 come up and then how do I actually knock them back to the other addresses.

And finally, my attempt to rally you to the cause of graphics work went exactly as expected. I can't be blamed for trying though!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on December 09, 2018, 03:24:59 pm
Hey guys, with me finishing DW1, it's time for me to throw my hat into the text editing ring as well. I've been following this thread and I should be all caught up as far as having all the stuff I need. Big thanks to you abw for your help :)

abw, I noticed your table says you haven't figured out those control codes, but there's one in particular that's bugging me. In the python script posted on the first script there's a standalone byte, F2, which is a special "s" byte. It's used in the "piece<s> of gold" dictionary entry. I would take a good guess that this is the same as the EF byte in DW1 that adds or omits an S based of the number value used in the string. I'd hope to use this to change the battle message "[name][’ ]s HP is reduced by [number]." into "[name] takes [number] point(s) of damage". Is there a way I can update the table to use this standalone byte?
Title: Re: General NES Hacking Questions
Post by: abw on December 09, 2018, 04:44:32 pm
I don't exactly understand the above instructions. Could you explain to me a little more clearly what you mean by "knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6"? First, how do I see when $BFD6 or $BFD7 come up and then how do I actually knock them back to the other addresses.
Oh, sorry, I meant you'll need to check the values of the pointers in the pointer table at ROM 0xB762-0xB7C1 (based on your current script size, 0xB7BE is the most likely culprit) following an insert to see if any of the values are $BFD6 or $BFD7 (a.k.a. D6 BF or D7 BF in little endian), and if so, then adjust the AUTOCMD commands in the insert script to fire when the insert point reaches $17FE5 or $17FE6 instead of $17FE7.

Basically, we need to make sure the game never tries to read script data at or after RAM $BFD7 / ROM 0x17FE7, and since the game unconditionally reads the first two bytes of the string based on the pointer value, pointer values of $BFD6 or $BFD7 are bad.

And finally, my attempt to rally you to the cause of graphics work went exactly as expected. I can't be blamed for trying though!
Based on your work from DW1, I think you'll be able to manage the graphics just fine without me :P.

Hey guys, with me finishing DW1, it's time for me to throw my hat into the text editing ring as well. I've been following this thread and I should be all caught up as far as having all the stuff I need. Big thanks to you abw for your help :)
Welcome to the party! (There's music for that in this game!) I figured you'd show up here sooner or later ;).

abw, I noticed your table says you haven't figured out those control codes, but there's one in particular that's bugging me.
I also assume the F2 code in DW2 works the same way as EF in DW1, so if you wanted to re-jig the dictionary to make that code available separately as opposed to being usable only as part of the "piece[(s)] of gold" entry, feel free. The process I described a couple of posts ago (https://www.romhacking.net/forum/index.php?topic=27053.msg367159#msg367159) should work just as well for this dictionary update as for "King" -> "Queen".

The codes I haven't bothered to check out are the F9, FD, and FF codes. Based on its usage in the script, F9 is probably some sort of item ID, though it'll take some research to figure out if and how it differs from F7. Some light experimentation suggested that FD might be useless, and FF looks like it might be some kind of conditional end token, but again, I haven't really looked at them to find out, so those are just guesses.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 09, 2018, 05:06:50 pm
Based on your work from DW1, I think you'll be able to manage the graphics just fine without me :P.

Hah. I'm glad my DW1 work *looks* impressive but truthfully all of that was my very first foray into hacking and more a result of elbow grease than actual skill. Most of it was simply spamming tiles from the famicom version all over the tile grid with a ton of duplication. The in game engine is still having the sprites point left, right, up, down etc but all of those directional tiles were simply replaced with the front facing famicom tiles so that they would display appropriately. The issues were when the US game would use mirrored images of tiles and that would put me in a spot where the famicom sprites would inevitably get screwed up. I would have to do little tricks to make the sprites more symmetrical like change the old wizard guy's head so his cap no longer leans to one side, or remove the guards spear because his body had to by symmetrical, all to work around the fact I couldn't disable the mirroring or point the code to different tiles.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 23, 2018, 04:19:50 pm
@abw

Since I'm pretty much done with my DW2 retranslation (perpetual rephrasings aside), I thought I would go back and give the first Dragon Warrior a bit of the old in out (or should I say "out in") with abcde. I figured I would let you and choppasmith have some time to bring your DW2 projects forward before I start assaulting you guys with requests for help on running extract and insert routines on DW3.

I got the table file that Choppasmith used, (byte format as opposed to the bit format one you sent me for DW2--I assume they both will work) and was pleasantly reminded to save it in UTF-8 per your readme. For the Cartographer.txt, I got the text pointer data from data crystal. I removed the auto jump instructions as I figured that would be DW2 specific. Gave it a shot and I got a ton of the same error message in command prompt: substr outside of string at C:\Rom Editing/abcde/Table/Table.pm line 234. It did spit out a very limited cartographer out file which I'll post a bit of immediately below. No in game text was captured in the output.

//POINTER #16 @ $8032 - STRING #16 @ $16FFE

#W16($8032)

// current address: $16FFE


//POINTER #17 @ $8034 - STRING #17 @ $1769B

#W16($8034)

// current address: $1769B


//POINTER #18 @ $8036 - STRING #18 @ $17A75

#W16($8036)

// current address: $17A75

This is what my Cartographer.text looks like below: (FYI I don't really need the comments for existing script turned on. I'm really only refining and expanding what I already wrote while I consult outside of game resources for the Japanese text.)

#GAME NAME:      Dragon Warrior I - Edit.nes

#BLOCK NAME:      Main Script
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $8012
#POINTER TABLE STOP:   $8037
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#ATLAS PTRS:      Yes
#BASE POINTER:      $C010
#TABLE:         dw1_script.tbl
#COMMENTS:      No
#END BLOCK

I'm not sure if anything else needed to be changed. I figure pointer size would be the same. I'm not sure if DW1 also used 16 strings per pointer. Let me know if you think something is wrong here or if something needs to be tweaked with the table file for whatever reason. (attached below)

https://www.dropbox.com/s/h2uxum9jurl0jcb/dw1_script.tbl?dl=0
Title: Re: General NES Hacking Questions
Post by: abw on December 30, 2018, 04:12:13 pm
Try it again with "#BASE POINTER: $10" instead - the issue here is that a pointer value of e.g. $8028 + a base value of $C010 = $14038, but the entire ROM (including header) is only $14010 bytes, so we end up trying to read beyond the end of the file, which doesn't work so well. The other thing that might trip you up a bit is that the final pointer only has 10 strings instead of 16, so for a cleaner dump you'll want to split the main script in to two blocks:
Code: [Select]
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8012
#POINTER TABLE STOP: $8035
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 16
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

// only 10 strings for the final pointer
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8036
#POINTER TABLE STOP: $8037
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 10
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 30, 2018, 11:08:16 pm
Try it again with "#BASE POINTER: $10" instead - the issue here is that a pointer value of e.g. $8028 + a base value of $C010 = $14038, but the entire ROM (including header) is only $14010 bytes, so we end up trying to read beyond the end of the file, which doesn't work so well. The other thing that might trip you up a bit is that the final pointer only has 10 strings instead of 16, so for a cleaner dump you'll want to split the main script in to two blocks:
Code: [Select]
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8012
#POINTER TABLE STOP: $8035
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 16
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

// only 10 strings for the final pointer
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8036
#POINTER TABLE STOP: $8037
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 10
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

@abw, thank you as always for that and the explanation.

Unfortunately however the results were not successful. The output file contains a little bit of the script in a mashed together format with no delineation as I had with DW2. After the small section of script there is this massive block of information in byte format. Here's a copy if you want to take a glance. I'm scratching my head as to what would be causing this result.

https://www.dropbox.com/s/4tvfmtab1upy3vu/Cartographer_out.txt?dl=0

my command line instruction for output is basically the same as the one you recommended for DW2. Should any of that change in the case of this game?

perl abcde.pl -m bin2text -cm abcde::Cartographer "Dragon Warrior I - Edit.nes" Cartographer.txt Cartographer_out -s
Title: Re: General NES Hacking Questions
Post by: abw on December 31, 2018, 01:06:04 am
Ah, yes, that table file doesn't mark any of the table entries as end tokens, and with no end tokens there's nothing to say when to stop dumping, so each string keeps going until the end of the ROM :P. It also doesn't contain any newline markers, so even after fixing the end tokens you'd still end up with all 16 of each pointer's strings on one line, which didn't feel right to me. Here's the table file I've been using; feel free to adjust the \n linebreaks to match your desired format.
Code: [Select]
00=0
01=1
02=2
03=3
04=4
05=5
06=6
07=7
08=8
09=9
0A=a
0B=b
0C=c
0D=d
0E=e
0F=f
10=g
11=h
12=i
13=j
14=k
15=l
16=m
17=n
18=o
19=p
1A=q
1B=r
1C=s
1D=t
1E=u
1F=v
20=w
21=x
22=y
23=z
24=A
25=B
26=C
27=D
28=E
29=F
2A=G
2B=H
2C=I
2D=J
2E=K
2F=L
30=M
31=N
32=O
33=P
34=Q
35=R
36=S
37=T
38=U
39=V
3A=W
3B=X
3C=Y
3D=Z
3E=‟
3F=”
# 40 and 53 are visually identical, but 40 is used as a right single quotation mark while 53 is used as an apostrophe
40=’
41=*
42=[triangle-r]
43=[triangle-d]
44=:
45=[..]

47=.
48=,
49=-

4B=?
4C=!
4D=;
4E=)
4F=(
50=‘
# 51 is visually identical to 50, but does not trigger "speech" sound effects or auto-indentation
51=[‘; no sound, no indent]
52=[.’]
# 53 is the apostrophe version of 40
53='
54=[ ’]

5F=

F0=[ Point(s)]
F1=[(n )enemy name]
F3=[experience Point(s)]
F4=[enemy name]
F5=[number]
F6=[spell name]
F7=[item name]
F8=[HERO]
FB=[wait]
# FC is the end token used in the main script
/FC=[end]\n\n
FD=[line]\n
# FF is the end token used in the item/monster/spell lists
/FF=[/name]\n
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 31, 2018, 01:45:13 am
This worked perfectly. I'll have to study the format you used here for when I get through this revision and eventually move on to doing the same for DW3. I really want to develop the skill level to do this on my own, but the thing I'm most foggy on is what kind of process you use to figure out all these opcodes. Would  you do an initial extraction with just letters and the line breaks and then use context clues to flesh out the opcodes?
Title: Re: General NES Hacking Questions
Post by: abw on December 31, 2018, 01:17:34 pm
This worked perfectly.
Hurray!

I really want to develop the skill level to do this on my own, but the thing I'm most foggy on is what kind of process you use to figure out all these opcodes.
Every game is a world unto itself, so the only (as far as I know, anyway) guaranteed way to figure out how and where the game stores its text is by getting that text to display in a debugging emulator and tracing the code from the point where it writes the text to screen all the way backwards until you find the code that read the text from ROM. You'll need to be at least somewhat familiar with the console's hardware, the emulator's debugging features, and whatever variety of ASM the console uses, and it's usually a pretty time-consuming process, but it will give you highly detailed knowledge of how the game handles its text. Frequently, though, there are indeed easier ways.

Would  you do an initial extraction with just letters and the line breaks and then use context clues to flesh out the opcodes?
For games with simple text encodings like Dragon Warriors I and III, yeah, that's probably the fastest way to go about it. Making a table file based on how the individual character tiles are laid out in the PPU and then dumping a block of text (if you already know where the text and/or pointers are) or even loading the ROM and table file in a hex editor and running a quick visual scan through the ROM for blocks that look like text will go a long way towards getting a usable script dump. After that, it's usually not too hard to guess what any other codes appearing in the dump are supposed to do, especially if you're already familiar with what the text is supposed to say (e.g. due to playing the game obsessively as a child) or can find the string in-game. For ones you aren't too sure about, experimenting by removing them from the string or adding them elsewhere and observing how the game's behaviour changes will often shed more light on their purpose, and if all else fails, you can start diving in to the ASM.

For games with slightly trickier text encodings like the main scripts of Dragon Warriors II and IV, even if you know how the individual characters are laid out in the PPU, the game doesn't store the text that way in ROM and you're never going to be able to just guess how the text actually is stored within any reasonable time frame (unless somebody else has already done a bunch of work for you, thus allowing you to make better guesses), so the simple methods aren't going to work there and tracing through the game's code is probably the best way of finding out what in the world it's doing under the hood.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on January 01, 2019, 07:42:07 pm
Every game is a world unto itself, so the only (as far as I know, anyway) guaranteed way to figure out how and where the game stores its text is by getting that text to display in a debugging emulator and tracing the code from the point where it writes the text to screen all the way backwards until you find the code that read the text from ROM. You'll need to be at least somewhat familiar with the console's hardware, the emulator's debugging features, and whatever variety of ASM the console uses, and it's usually a pretty time-consuming process, but it will give you highly detailed knowledge of how the game handles its text. Frequently, though, there are indeed easier ways.

This brings me to a question. Over the last couple weeks I've been making a concerted effort to learn more about the NES architecture, study tutorials on the basics of ASM, and have spent more time playing with the FCEUX debugger. The 6502 NES ASM tutorials usually focus on teaching the basics of building an NES game from the ground up, which is good (if overwhelming) information but doesn't necessarily hone in on what I really want to be learning, which is what you highlighted--using the debugger to follow the data trails back to the rom code that dictates them. I've spent a lot of time looking for videos or tutorials around debugging and haven't come up with much. The only substantial information I've encountered is what was included in the FCEUX files themselves. Do you happen to recall anything in particular that helped you learn the process of tracing data back to its source? I think this is what would help me most not only with resolving some of these script editing issues but it would also help me track down how tile address instructions are stored in the code for the graphics work I need to do.
Title: Re: General NES Hacking Questions
Post by: abw on January 02, 2019, 10:39:17 am
Do you happen to recall anything in particular that helped you learn the process of tracing data back to its source?
Nothing specifically, alas. At a high level, the process is fairly simple: start with something you know (e.g. some text currently being displayed), figure out how that something got to be where it is / where it came from, and then keep repeating until "where it came from" is the ROM. The details can sometimes get messy, but FCEUX provides a pretty good collection of debugging tools for helping you do all that figuring out.

The Dragon Warrior code seems to be very fond of copying data back and forth between multiple staging areas in cartridge RAM before writing to the PPU, so even if you've mastered the use of FCEUX's Debugger, using its Trace Logger is probably still going to be a quicker way to find out useful information for these particular games. I wrote up a detailed description of tracking down DW1's item list and the pointer to it using FCUEX's Trace Logger in the Dragon Warrior 1 Spanish Translation (http://www.romhacking.net/forum/index.php?topic=26135.msg358879#msg358879) thread; you might find that helpful if you haven't read it already.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on January 06, 2019, 10:45:39 pm
@abw,

So I started the process of reinserting the script changes as I go. My main motivation for doing so was that I was hoping to be able to track how many characters worth of data remained available, as that indicator was very helpful while working on II.

After my first insert of my DW1 script, the command prompt curiously gave me no indication of remaining data afterwards. Other than that, the insertion seems to be working within the game. I believe I have some data available since I eliminated a ton of blank character spaces in the existing script. Am I missing something that turns that on this time around? This is the data I added to the top of my Atlas file.

// Define, load, and activate a TABLE
#VAR(Table, TABLE)
#ADDTBL("dw1_script.tbl", Table)
#ACTIVETBL(Table)

// Jump to start of script
#JMP($8038)
#HDR($10)

I didn't include the auto commands for mid-string swaps/counter variables, etc that were used in my Atlast file for DW2

I suppose I can pull the game up for myself in a hex editor to gauge how I'm doing with space but that command prompt indicator I always got before was very helpful.
Title: Re: General NES Hacking Questions
Post by: abw on January 07, 2019, 09:34:43 pm
Am I missing something that turns that on this time around?
Yup - without an upper bound on the insert range, there's no good way to calculate how much available space remains. abcde will just keep on inserting data until either it runs out of data to insert or your computer runs out of disk space (though I haven't tried that last one myself) :P. Try changing the jump command to e.g. "#JMP($8038, $BCBF)" and you should get some better feedback.

On a side note, I just submitted an update for abcde. Nothing major, but there are a couple of goodies that will be helpful if you end up doing some menu work on these games.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on February 06, 2019, 07:47:17 pm
So after a lot of editing (and I still wouldn't say it's 100% done) I managed to edit my script dump with the mobile script. It's 80K, about 50% bigger than the original.

I had to edit it a few times to fix some errors, thankfully abcde makes it easy to find what the troubled line is.

I then get this:

(https://i.imgur.com/DbrUmm6.png)

So far so good?


But then I get some scrambled text:

(https://i.imgur.com/DrJGHVe.png)

Looks like I need to double check my dictionary values. I hope that's my only problem.

EDIT: Yep dictionary and table needed a tiny tweak

(https://i.imgur.com/GxaFTL2.png)

Though, I think there's something funny with the special table provided in this thread. The combined .' for dialog works fine but the standalone closing quote/apostrophe just brings up the openign single quote instead

(https://i.imgur.com/9vw9uVS.png)

Also, getting some scrambled text. I wonder if this is from being over the limit. There's no way to immediately tell as said on the last page.

(https://i.imgur.com/8yK2JXO.png)

EDIT 2: Yeah the Midenhall castle NPC dialog is near the end of the first bank. If I'm reading it right, would using the script on the previous page be a way of transferring dialog/pointers to the second bank?
Title: Re: General NES Hacking Questions
Post by: abw on February 06, 2019, 10:28:36 pm
So after a lot of editing (and I still wouldn't say it's 100% done) I managed to edit my script dump with the mobile script. It's 80K, about 50% bigger than the original.

I had to edit it a few times to fix some errors, thankfully abcde makes it easy to find what the troubled line is.
You might want to grab the latest version (0.0.3) of abcde - I made some improvements like letting you know exactly how far over your limit the script goes. For your massive script, DW2 comes with 3 whole empty banks, so at least you shouldn't have to expand the ROM this time. If you haven't come across it yet, the code for switching between script banks starts at 0x3FE15 / $0F:$FE05, so that seems like a good place to start rewriting ASM.

Though, I think there's something funny with the special table provided in this thread. The combined .' for dialog works fine but the standalone closing quote/apostrophe just brings up the openign single quote instead
Yeah, like I said, I hadn't put much effort into the table file at that point, and I've noticed a couple of mistakes in it. I've uploaded my current table file here (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) if you're interested, but if you've been modifying the dictionary, you'll also want to ensure that your dictionary does in fact contain the correct value.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on February 13, 2019, 02:42:50 pm
You might want to grab the latest version (0.0.3) of abcde - I made some improvements like letting you know exactly how far over your limit the script goes. For your massive script, DW2 comes with 3 whole empty banks, so at least you shouldn't have to expand the ROM this time. If you haven't come across it yet, the code for switching between script banks starts at 0x3FE15 / $0F:$FE05, so that seems like a good place to start rewriting ASM.
Yeah, like I said, I hadn't put much effort into the table file at that point, and I've noticed a couple of mistakes in it. I've uploaded my current table file here (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) if you're interested, but if you've been modifying the dictionary, you'll also want to ensure that your dictionary does in fact contain the correct value.

Thanks for that, I've updated my table. I tried using the newest version of your abcde, I now get: "#JMP bounded by at $BE0F has -15738 ($-3D7A) space left" So I take it this means my script overwrites 15,738 bytes over the limit of the bank? Would cutting down my script temporarily be a good way to indicate where would be a good stopping point for the first bank? (I know I can't just go into the ROM and see where it overwrites data) BTW, maybe I'm missing something (and I'll be honest, I'm earnest in learning but a lot of this stuff still makes me go @_@) Why is it in the ROM Map, The Main Script Part 1 starts at B7C2 and Part 2 starts at 14010 but in the atlas script they're the other way around?
Title: Re: General NES Hacking Questions
Post by: abw on February 13, 2019, 06:38:46 pm
Thanks for that, I've updated my table. I tried using the newest version of your abcde, I now get: "#JMP bounded by at $BE0F has -15738 ($-3D7A) space left" So I take it this means my script overwrites 15,738 bytes over the limit of the bank? Would cutting down my script temporarily be a good way to indicate where would be a good stopping point for the first bank? (I know I can't just go into the ROM and see where it overwrites data)
If you want to get technical about it, the script space is already split between 2 banks, so that means you're 15,738 bytes over the total script space limit rather than the bank limit, but basically, yes. For finding out how much of your script fit into the first bank, you can check 0x3FA90 to see which pointer starts the next bank; if your new script is twice as big as the original, you'll probably have something like $17 there instead of $2E. Temporarily cutting down your script is probably the fastest way to find out how much fits into the original script space, though since it looks like you're going to need 3 banks anyway, you may wish to consider moving the script space from 0xB7C2-0xBE0F to one of the unused banks; some of DW2's bankswap routines are limited to swapping in one of the first 8 banks, so having lots of free space in bank 3 might come in handy if you plan on making other changes too.

BTW, maybe I'm missing something (and I'll be honest, I'm earnest in learning but a lot of this stuff still makes me go @_@) Why is it in the ROM Map, The Main Script Part 1 starts at B7C2 and Part 2 starts at 14010 but in the atlas script they're the other way around?
Oh, sorry, that was my fault - I was in "sort things by ROM address" mode :P. I've corrected the wiki now.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 01, 2019, 02:50:31 pm
Sorry for the late reply, just been finding no time to spare lately, but I'm still determined to learn and get this figured out.

So I found that I can fit up to "Pointer 34" in the original space (with 818 bytes left over which is plenty for possible edits and additions) leaving the text from Pointer 35-45 and the last little bit of "Script Part 2" left.

I also inserted just the script AFTER Pointer 34 into the original space and got 1401 bytes left. So by doing a little bit of math subtracting those numbers from 17,955 (the combined amount of bytes for script space according to the ROM Map) means the first part of my script that fits is 17,137 bytes and the remaining is 16,554 meaning my whole script is 33,691 bytes. So yeah despite different text file sizes, the actual script data is about double.

So I'm already a little confused here...

Firstly


For finding out how much of your script fit into the first bank, you can check 0x3FA90 to see which pointer starts the next bank; if your new script is twice as big as the original, you'll probably have something like $17 there instead of $2E.

So what am I looking at here (at 3FA90) exactly? It certainly says 2E. What does that signify?

And secondly, more a noobish question, but am I understanding correctly that the Wiki stating the game has 16 PRG-ROM Pages that are 16KB each referring to the banks? If I do switch banks for dialog, and yes I can see there are plenty of blank patches of data, is there an easy way to determine what bank is where aside from trying to divide the bytes of the rom by 16? I feel like I'm missing something glaringly obvious.

As far as doing anything else. The only thing I might do is what you and Chicken Knife talked about in moving the Monster Names. I had shortened the names just enough to fit, but having that extra little bit of space for having full names would be nice. Script is the priority for me though!

Anyway, once again, I appreciate the help and feel free to tell me just enough so I can figure it out while learning. Certainly don't want to come off as the type who just wants stuff done for them!
Title: Re: General NES Hacking Questions
Post by: abw on March 01, 2019, 06:52:28 pm
So what am I looking at here (at 3FA90) exactly? It certainly says 2E. What does that signify?
What you've got there is the key byte inside the routine for deciding which ROM bank needs to be swapped in to get the string the game wants to display:
Code: [Select]
; determine which ROM bank to load based on string index
; IN: A = low byte of string index, X = high byte of string index << 5 (don't ask me why), Y and C irrelevant
; OUT: no change to A or X, Y = index of ROM bank to swap in to read the string from, C not used, but does indicate whether we ended up choosing bank 5 (clear) or 2 (set)
; control flow target (from $FA68)
0x03FA7E|$0F:$FA6E:A0 05    LDY #$05 ; default to bank 5
0x03FA80|$0F:$FA70:48      PHA ; save the original string index in A
0x03FA81|$0F:$FA71:29 F0    AND #$F0 ; useless op
0x03FA83|$0F:$FA73:4A      LSR
0x03FA84|$0F:$FA74:4A      LSR
0x03FA85|$0F:$FA75:4A      LSR
0x03FA86|$0F:$FA76:4A      LSR
0x03FA87|$0F:$FA77:85 10    STA $10 ; 16 strings per pointer, so low byte of string index >> 4 = low nybble of pointer index
0x03FA89|$0F:$FA79:8A      TXA
0x03FA8A|$0F:$FA7A:29 E0    AND #$E0
0x03FA8C|$0F:$FA7C:4A      LSR ; A = high nybble of pointer index
0x03FA8D|$0F:$FA7D:05 10    ORA $10 ; glue the high and low nybbles of the pointer index together into a single byte
0x03FA8F|$0F:$FA7F:C9 2E    CMP #$2E
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead
; control flow target (from $FA81)
0x03FA95|$0F:$FA85:8C C6 60 STY $60C6
0x03FA98|$0F:$FA88:68      PLA ; restore the original string index in A
; control flow target (from $D12B)
; external bank control flow target (from $06:$9542, $06:$9550)
0x03FA99|$0F:$FA89:60      RTS
If you added the #INC commands I mentioned in an earlier post (http://www.romhacking.net/forum/index.php?topic=27053.msg367369#msg367369), that #$2E should have been updated to reflect the new dividing line between bank 5 and 2.

The good news is that it looks like you are pretty close to being able to fit your script into just 2 banks, and if you can manage that, then all you need to do is insert the second half of your script into one of the unused banks and update 0x3FA94 to the new bank number. Do you think you can update the dictionary to get an extra 3% compression?

And secondly, more a noobish question, but am I understanding correctly that the Wiki stating the game has 16 PRG-ROM Pages that are 16KB each referring to the banks?
Yup!

is there an easy way to determine what bank is where aside from trying to divide the bytes of the rom by 16? I feel like I'm missing something glaringly obvious.
Math is the way (especially when you're flipping between games with different bank sizes), but this shouldn't be something you need a calculator for. 16 KB = $4000, so (ignoring the $10 byte iNES header), $0000 - $3FFF is bank 0, $4000 - $7FFF is bank 1, $8000 - $BFFF is bank 2, $C000 - $FFFF is bank 3, and so on. So if you have an address like 0x3FA90, you can look at the 3F part and say something to yourself like "3 x 4 = 12, F is in bank 3, and 12 + 3 = 15" to know that 0x3FA90 is in ROM bank 15.

Anyway, once again, I appreciate the help and feel free to tell me just enough so I can figure it out while learning. Certainly don't want to come off as the type who just wants stuff done for them!
Heh, the first game was free, but you're going to have to work for the second one :D.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 02, 2019, 12:23:48 pm
Heh, the first game was free, but you're going to have to work for the second one :D.

Sounds fair :) I'll get this slowly but surely.

Okay, so, in updating my dictionary, I had to update the table because I was still unable to use apostrophes and single closing quotes. Part of my new dictionary table looks like this (taken from your latest table)

Code: [Select]
%1110110101='
%1110110110=-
%1110110111=’

I do the insert.bat and it stops at a line at Pointer 26

Code: [Select]
C:\Users\mog11\Downloads\abcde_v0_0_3>perl abcde.pl -m text2bin -cm abcde::Atlas "DRAGONW2.nes" Atlas.txt
unable to tokenize; best attempt failed at input position 160 at ^ indicator in
]'s coffin.[end-FC]‘'Tis more than a ma
]'s coffin.[end-FC]ΓÇÿ^
(does your table file contain a ""?) in text string starting at Atlas.txt line 1127!

And here's the line as it appears in my atlas.txt

Code: [Select]
puts it back in [name]'s coffin.[end-FC]

‘'Tis more than a man could ask to know such elation at so advanced an age!’[wait][line]

Really baffling what happened here. It's not like this is the first apostrophe in the text.

Also, while I updated the dictionary to save a few bytes (adding a space for "gold", "Goddess", and "key") I found that I missed a bunch of post-final boss NPC dialog that needed adding, so droping it 3% might be out unfortunately.
Title: Re: General NES Hacking Questions
Post by: abw on March 02, 2019, 06:13:33 pm
I do the insert.bat and it stops at a line at Pointer 26

Code: [Select]
C:\Users\mog11\Downloads\abcde_v0_0_3>perl abcde.pl -m text2bin -cm abcde::Atlas "DRAGONW2.nes" Atlas.txt
unable to tokenize; best attempt failed at input position 160 at ^ indicator in
]'s coffin.[end-FC]‘'Tis more than a ma
]'s coffin.[end-FC]ΓÇÿ^
(does your table file contain a ""?) in text string starting at Atlas.txt line 1127!

And here's the line as it appears in my atlas.txt

Code: [Select]
puts it back in [name]'s coffin.[end-FC]

‘'Tis more than a man could ask to know such elation at so advanced an age!’[wait][line]

Really baffling what happened here. It's not like this is the first apostrophe in the text.
Hmm, that does look odd. Based on that output, it looks like your system is defaulting to code page 437 (where "ΓÇÿ" = E2 80 98 = the UTF-8 encoding of U+2018 = "‘"), which means "┬ì" = C2 8D = the UTF-8 encoding of U+008D = Reverse Line Feed, but I have no idea how that managed to get in there. So, it looks like you've got an invisible character sitting between the "‘" and the "'" for some reason (you should be able to verify that hypothesis by viewing your insert script with a hex editor); try deleting that section and re-type the "‘'".

Also, here's a tip from abcde's readme: if you do "chcp 65001" in cmd before running abcde (e.g. at the start of a .bat file), you should get abcde's UTF-8 output showing up as UTF-8 characters instead of whatever code page your system defaults to... though in a case like this involving a non-printable control code, having gobbledygook is probably more helpful in tracking down the cause of the error!

Also, while I updated the dictionary to save a few bytes (adding a space for "gold", "Goddess", and "key") I found that I missed a bunch of post-final boss NPC dialog that needed adding, so droping it 3% might be out unfortunately.
Ah, if you have more text to add anyway, then yeah, trying to fit everything into 2 banks might be more work than just using 3 banks. So much for only needing a 1-byte hack :P.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 05, 2019, 11:14:58 am
Okay figured out the weird little script bug (you were right, there was some weird hex between the opening quote and the apostrophe). And with the recent dialog added 34,221 bytes.

Just to be sure. Your abcde script enters the script in both areas, right? I just want to make sure my math is right. The space remaining that I get when inserting part of it is the total amount between the two sections? Not just one? Because it seems like the best free space to use as a third bank is 1CC30 which is over 13K.

Old Script Part 1: 3FD7 Length (16, 343)
Old Script Part 2: 64E Length (1,614)
Total: 17,957

And to use a third bank I'm basically going to have to duplicate this part of the code:

Code: [Select]
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead

And change the pointer index and bank so that it's something like pointer index < #$22 (Pointer 34) then have something for Pointer 35-45 and then have a line for code 2E (46) equal and greater in value. Is that right?
Title: Re: General NES Hacking Questions
Post by: abw on March 05, 2019, 05:18:32 pm
Okay figured out the weird little script bug (you were right, there was some weird hex between the opening quote and the apostrophe).
Hurray!

Just to be sure. Your abcde script enters the script in both areas, right? I just want to make sure my math is right. The space remaining that I get when inserting part of it is the total amount between the two sections? Not just one?
Yup, yup, and yup.

Because it seems like the best free space to use as a third bank is 1CC30 which is over 13K.
I'd probably go with one (or two if you need another one) of the completely unused banks starting at 0x30010, 0x34010, or 0x38010 for a full 16k each, but if you can fit all your script into the free space starting around 0x1CC30, then great.

And to use a third bank I'm basically going to have to duplicate this part of the code:

Code: [Select]
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead

And change the pointer index and bank so that it's something like pointer index < #$22 (Pointer 34) then have something for Pointer 35-45 and then have a line for code 2E (46) equal and greater in value. Is that right?
Basically, yeah. You'll just need to find a home for that code, but bank F is already pretty full. There should be enough unused space for your needs at 0x3FFA7 - 0x3FFC9 (0x3FFCA is used!), though. It also possible there's other code elsewhere that might need to be updated too; I haven't done that analysis for you this time either ;).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 10, 2019, 02:36:55 pm
Oh Happy Day! Feeling overwhelmed by the idea of having to move important code around, I took another crack at editing the dictionary, swapping out entries with about 15 or so uses with ones much higher and I got my script down to 32,378 bytes! And I can totally trim the fat in my script if need be. That should be much easier! What a relief!

So, if I keep bank 5 as default and go with your suggestion of using the space at 30010 (thanks btw, I didn't really think of looking that far, I didn't think there'd be full unused banks) which would be bank... 10, right? That would change the value of 3FA94 to 0A.

Now I just need to add the bit from your earlier post so it can split the script properly. Edit: yes the JMP is updated to reflect the dumping to the new bank.


Code: [Select]
// add this near the top of the insert script:
#VAR(pointerNum, COUNTER) // create a COUNTER variable named pointerNum
#CREATECTR(pointerNum, 8, 0) // pointerNum is an 8-bit value initialized to 0
#AUTOCMD($17FE7, #WLB(pointerNum, $3FA90)) // update the code that controls which pointer starts the next bank

// and then after every #W16 line in the insert script, add:
#INC(pointerNum, 1)

// Jump to start of script
#JMP($14010)
#HDR($C010)
// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
#AUTOCMD($17FE7, #HDR($10))
#AUTOCMD($17FE7, #JMP($30010, $3400F))

Edit: yes the JMP is updated to reflect the dumping to the new bank. I seem to be close. Once trying with my full script (as opposed to pieces to determine length and where the best split would be) I get "#JMP bounded by at $3400F has -1223 ($-4C7) space left" so it seems maybe it's still a bit too big? Did I make a miscalculation somewhere? Either way, 1K to trim down should be easy and inconsequential.
Title: Re: General NES Hacking Questions
Post by: abw on March 10, 2019, 05:05:58 pm
Oh Happy Day! Feeling overwhelmed by the idea of having to move important code around, I took another crack at editing the dictionary, swapping out entries with about 15 or so uses with ones much higher and I got my script down to 32,378 bytes! And I can totally trim the fat in my script if need be. That should be much easier! What a relief!
Nice!

So, if I keep bank 5 as default and go with your suggestion of using the space at 30010 (thanks btw, I didn't really think of looking that far, I didn't think there'd be full unused banks) which would be bank... 10, right? That would change the value of 3FA94 to 0A.
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

If you're still having space issues, don't forget about the single-character dictionary entries - if you're e.g. using "p" more than "w", you could kick "w" into the 10-bit range and move "p" into the 5-bit range to get better compression. For English text, you can probably also replace the "q" entry with "qu".
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 10, 2019, 05:24:29 pm
Nice!
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

Well, as said a few posts ago, I split my script into two parts to determine how much of it would fit. I took the bytes remaining as reported by abcde and subtracted from the total space available and did it again for the second part then combined the values which is how I got the 32,378 bytes. I had done the same thing before and I was getting over 34,000 bytes, so even if my math is somehow off it's still a significant decrease.

Nice!
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

If you're still having space issues, don't forget about the single-character dictionary entries - if you're e.g. using "p" more than "w", you could kick "w" into the 10-bit range and move "p" into the 5-bit range to get better compression. For English text, you can probably also replace the "q" entry with "qu".
Yeah that could work. I would assume, in layman's terms, that the first set of entries use less data than the 4 "C" tables? Or does each table use extra bits of data the further you go down? (Edit : forget that I'm dumb)I honestly still don't quite understand how those work, I initially thought it was like a special table where'd you use two bytes to load a dictionary value, and pretty much every word/letter from the dictionary wound be preceded by the corresponding value and then stuff from the first table fills in the rest with no dictionary value needed.

Edit : Just read the Wiki, I guess it works like I said above but in bits, not bytes?
Title: Re: General NES Hacking Questions
Post by: abw on March 10, 2019, 06:12:36 pm
Yeah that could work. I would assume, in layman's terms, that the first set of entries use less data than the 4 "C" tables? Or does each table use extra bits of data the further you go down? (Edit : forget that I'm dumb)I honestly still don't quite understand how those work, I initially thought it was like a special table where'd you use two bytes to load a dictionary value, and pretty much every word/letter from the dictionary wound be preceded by the corresponding value and then stuff from the first table fills in the rest with no dictionary value needed.

Edit : Just read the Wiki, I guess it works like I said above but in bits, not bytes?
Basically, yeah. I had this typed up before I saw your edit, so I'll post it anyway just in case it's still helpful:
--
Sort of, except as far as the game's concerned, the individual tokens are 5 bits each, not 8, and 4 of the entries from the first 5-bit table are used to switch to the corresponding "C" table. You could split that single table file into multiple table files if you wanted to; I just prefer having all the entries in a single table.

You can tell how much space each entry takes up by looking at its left-hand side. In a hexadecimal table, "80" takes up one byte (or 2 nybbles or 8 bits, depending on how you want to think of it), "80FF" takes up two bytes (/4 nybbles/16 bits), and so on; in a binary table, "11011" takes up 5 bits, "1110101111" takes up 10 bits, and so on. This will be an even bigger issue if you decide to tackle DW4, since it uses a Huffman encoding where the binary representation of individual characters ranges between 3 bits (e.g. for "e") and 18 bits (e.g. for "8").
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 11, 2019, 03:18:44 pm
Basically, yeah. I had this typed up before I saw your edit, so I'll post it anyway just in case it's still helpful:
--
Sort of, except as far as the game's concerned, the individual tokens are 5 bits each, not 8, and 4 of the entries from the first 5-bit table are used to switch to the corresponding "C" table. You could split that single table file into multiple table files if you wanted to; I just prefer having all the entries in a single table.

You can tell how much space each entry takes up by looking at its left-hand side. In a hexadecimal table, "80" takes up one byte (or 2 nybbles or 8 bits, depending on how you want to think of it), "80FF" takes up two bytes (/4 nybbles/16 bits), and so on; in a binary table, "11011" takes up 5 bits, "1110101111" takes up 10 bits, and so on. This will be an even bigger issue if you decide to tackle DW4, since it uses a Huffman encoding where the binary representation of individual characters ranges between 3 bits (e.g. for "e") and 18 bits (e.g. for "8").

Thanks, once I saw that the programmer's calculator had a binary converter, it just clicked.

Anyway, just from doing the above suggestion I'm down another 400 bytes! And just after some needed editing of the intro I'm now down to 760 bytes over the limit. I was really loose and liberal with my initial script (like gratuitous line breaks, pauses, and spaces), so it should be easy to trim the fat without sacrificing much. Although when I tried dumping with the new second address, I'd get horrible garbage dialog. The intro is only the 8th pointer in bank 5, so I'm not too sure what happened there. I know I have to update the ROM with the new second bank, but I'm not sure why dialog in the beginning wouldn't work properly. Did I miss something (other than updating the second bank?)

A couple other weird issues

Still having trouble with closing single quotes for some reason.

(https://i.imgur.com/w2bUVKS.png)

I double checked the ROM and I do indeed have $66 in the dictionary. I also have an entry with "!’" but that has Apostrophes ($67) work fine though

Also

(https://i.imgur.com/VrR1E2T.png)

Not sure what's going on here. There's a couple of windows like this with the oddly premature uncalled for line break. I'm assuming the window counts how many bits are used per line as opposed to individual letters/spaces/characters? Any suggestions? Or is it just a weird game programming/abcde limitation?
Title: Re: General NES Hacking Questions
Post by: abw on March 11, 2019, 08:03:16 pm
Hurray! The proper choice of dictionary entries makes a huge difference to the compression ratio. Just make sure you leave the first 5 end tokens and the 4 "C" table switches alone (since the game cares about those) and fill out the rest of the main 5-bit table with your 23 best tokens.

As for your issues, it's kind of hard to say for sure without seeing the files. If dumping the inserted data didn't work but the game still displays it correctly, then something is wrong with your dump script. For the closing single quotes, my next guess would be an error in the dictionary lengths, though a length error would also mess up all the dictionary entries following the one with the wrong length. The random line breaks is definitely strange too, and not an issue I've come across myself; I've definitely inserted longer strings than that without getting inappropriate line breaks. I haven't looked at the code for it, but just based on observation, DW2's line wrapping algorithm seems pretty solid. Any chance it always happens around the words "What" or "is"? If so, that could indicate that you've got a line break instead of a space in one of the dictionary entries - just a guess!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 16, 2019, 06:27:57 pm
I got it! I had used Pointer Tables to extract, edit, and insert the dictionary entries. Yeah, I know it's clunky, but easy for me. It was changing blank spaces (5F) to [no voice] (59). Once I changed it in the ROM it was fine. Similar problem with the closing single quotes and I fixed that too. I was hoping that this could've shrunk the script size, but, alas, it didn't. Oh well, more editing work for me!

Congrats on your Latin translation release, abw! I had looked at the Read Me and you said one of your improvements was editing the main party status windows to show (was this battle or field or both?) full character names instead of just showing the first four letters.

When working on DW1 menus, the thought of doing that for later entries struck me as a "would be nice if possible". You mentioned it was more than a one byte hack. Were there other entries for the window you had to change besides length? Or is it like monster/item/spell/etc names where you had to find that value that affected display length? Btw, I know it'd be easy to tell the magic byte that affects monster name length like DW1, but you did explain how you did it and I have every intention to try myself as part of my learning.
Title: Re: General NES Hacking Questions
Post by: abw on March 17, 2019, 05:53:07 pm
I got it!
Congrats!

Congrats on your Latin translation release, abw! I had looked at the Read Me and you said one of your improvements was editing the main party status windows to show (was this battle or field or both?) full character names instead of just showing the first four letters.

When working on DW1 menus, the thought of doing that for later entries struck me as a "would be nice if possible". You mentioned it was more than a one byte hack. Were there other entries for the window you had to change besides length? Or is it like monster/item/spell/etc names where you had to find that value that affected display length?
Thanks! If you check the screenshots, you'll see that full character names are displayed both in and out of battle; I think I made changes for displaying full names in a total of 37 different menus, which typically also involved widening and repositioning them and sometimes also involved making updates to the menu wiping process; the WEAPON/ARMOR/SHILD/HELMET menus were particularly irritating to deal with in that respect. For the code changes, the basic problem is that DW2 stores the first 4 characters of each hero's name in one spot and the last 4 characters in a completely different spot (probably due to the English version's extra 4 characters being bolted on to the original 4 characters present in the Japanese version), so you can't just find the code that says "read 4 bytes from X" and update one byte to say "read 8 bytes from X", you need to add more code (which needs more space) to read 4 bytes from X and glue them together with 4 bytes from Y.

For the later games in the series, the mini status menu was rearranged into 1 column per hero instead of 1 row per hero, so widening the columns probably wouldn't work out very well due to lack of screen space if everybody has a long name, but you could switch the layout back to rows if you wanted. It's just a question of how much work would be involved.

Btw, I know it'd be easy to tell the magic byte that affects monster name length like DW1, but you did explain how you did it and I have every intention to try myself as part of my learning.
Trace logger is your friend there too!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 16, 2019, 03:08:52 pm
Basck from vacation and ready to tackle this!

Okay, I'm having a strange pointer issue (at least I think it's Pointer related). I got my script (with around 19 bytes to spare!)

When the game loads the intro, instead of loading the line

"This is the royal castle of the kingdom of Moonbrooke." which is Pointer 7, String 14 it loads instead, "‘We shall await thy return most eagerly!’" Which is a couple lines up at String 12. Granted I removed the "Hold Reset" dialog to save space but I double checked and I don't see any missing end-FC marks I might've missed that would cause the issues. What's weird is, in the next pointer, the first two lines load just fine but everything after is mixed up.

Did I mess up the formatting of my atlas.txt somehow? It's right here
http://www.mediafire.com/file/e7kqqhf720plzpf/atlas.txt/file

(note because of the trouble Chicken Knife had with his script mentioned earlier in this thread, I opted to extract the script with comments:off so it's just the new dialog)

I know I read your post, abw, a couple pages back about the ROM flaw that pointers could get mixed up, but I'm not sure I understand what you mean by checking them. In DW1, I could load up the ROM in Windhex, look at the start of the string, the address, then compare it with the pointer, but I'm not sure how I can do that with DW2's compression.
Title: Re: General NES Hacking Questions
Post by: darthvaderx on April 16, 2019, 05:26:13 pm
Is it possible to run Lua Script in Mesen in the same way as in FCEUX? I've tried to run Metroid in every way but none worked.
Title: Re: General NES Hacking Questions
Post by: abw on April 16, 2019, 09:56:29 pm
When the game loads the intro, instead of loading the line

"This is the royal castle of the kingdom of Moonbrooke." which is Pointer 7, String 14 it loads instead, "‘We shall await thy return most eagerly!’" Which is a couple lines up at String 12.
I'm not seeing anything immediately obvious that would cause that kind of error, and a quick test with a version of the original table file tweaked to include your new characters gets me the expected text displayed during the entire intro sequence, so I suspect your problem lies elsewhere. Getting an earlier string from the same pointer makes me think the game might be counting more end tokens than you want it to... what does your table file look like? If you altered any of the first 5 entries, that might explain this behaviour.

I know I read your post, abw, a couple pages back about the ROM flaw that pointers could get mixed up, but I'm not sure I understand what you mean by checking them.
Basically you just pop open the ROM in a hex editor, look at 0xB772-0xB7D0, and check whether any of those pointers point to $BFD6 or $BFD7. If they do, then you'll need to move the auto-jump point (0x17FE7) back a byte or two in order to force the affected pointer to point into the second script bank instead of letting it read non-script data from the first script bank and make a mess of your text.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 18, 2019, 04:42:11 pm
I'm not seeing anything immediately obvious that would cause that kind of error, and a quick test with a version of the original table file tweaked to include your new characters gets me the expected text displayed during the entire intro sequence, so I suspect your problem lies elsewhere. Getting an earlier string from the same pointer makes me think the game might be counting more end tokens than you want it to... what does your table file look like? If you altered any of the first 5 entries, that might explain this behaviour.
Basically you just pop open the ROM in a hex editor, look at 0xB772-0xB7D0, and check whether any of those pointers point to $BFD6 or $BFD7. If they do, then you'll need to move the auto-jump point (0x17FE7) back a byte or two in order to force the affected pointer to point into the second script bank instead of letting it read non-script data from the first script bank and make a mess of your text.

Okay that went better than I hoped!  :laugh:

So, when you suggested the problem (thanks for that, it's easy for me to overlook little things like that), it was indeed something I did with my table/dictionary. I thought I'd swap the standalone [FF] with the standalone Period that was further down. Once I swapped it back, the intro text displayed fine again. Lesson learned. Adding a note in my copy of the table: DO NOT MESS WITH THE FIRST 5 VALUES! Only problem was I went from just being under the limit by about 20 bytes, to being OVER the limit by about 60. So I had to scour the dictionary again to see if there was some way to get the size back down. After adding a space to the comma entry and replacing the unused ; by realizing I had a lot of words and phrases that ended in "eth" and boom that was enough to put me WELL under with about 140 bytes to spare. WOW! I'm ecstatic! I should be able to restore the little bit I had to cut (mostly consists of long gratuitous laughs, pauses, and a post-final boss NPC line).

Anyway when I asked about changing the second text bank at 3FA94, I figured having the second block at $3400F would make it Bank 10 (0A) but when you replied with an explanation of how the bank swapping work, you mentioned Bank 12 and I'm not sure if you were just giving an unrelated example, subtly correcting me, or just made a typo. I'm getting some messed up text by a couple of the Midenhall Castle NPCs using either value (either blank windows or a sparse random letter) so i figured I'll have to look at those pointers, but I want to make sure I'm not doing this with the wrong bank value.

EDIT: These are my values between b772-B7D0

16 90 73 92 7D 92 87 92 91 92 9B 92 A5 92 AF 92 B9 92 02 94 19 95 3E 96 12 98 37 99 12 9C 62 9E 35 A1 0A A4 CE A7 AB AB 5E AF 68 AF 72 AF 7C AF 86 AF 42 B4 37 BB 43 02 B1 07 93 0E 4A 13 FC 17 7D 1B 88 1E 1C 22 54 27 A0 2D 9D 31 16 35 0A 39 DF B4 EB A0 4D BB 17 F4 F9 C3 E2 57 C0 E7 8F 9D

So, nothing wrong there, then?

Edit 2: So am I understating right that my problem is probably between Pointer 32 and 35 which is pointer B7A2 and B7A8 respectively? Since B7A8 points to the new bank? Pointer 32 has a lot of the Midenhall dialog, some of which is displayed properly. Sorry to say, I'm still puzzled here.
Title: Re: General NES Hacking Questions
Post by: abw on April 19, 2019, 12:01:43 am
Okay that went better than I hoped!  :laugh:
[...]
Adding a note in my copy of the table: DO NOT MESS WITH THE FIRST 5 VALUES!
Hurray! To be fair, I did actually mention that a few posts back (http://www.romhacking.net/forum/index.php?topic=27053.msg372393#msg372393), but I'm glad you've got it working now!

Anyway when I asked about changing the second text bank at 3FA94, I figured having the second block at $3400F would make it Bank 10 (0A) but when you replied with an explanation of how the bank swapping work, you mentioned Bank 12 and I'm not sure if you were just giving an unrelated example, subtly correcting me, or just made a typo.
Heh, that one I managed to write down correctly for a change :P. Banks $0C (starting at 0x030010), $0D (0x034010), and $0E (0x038010) are DW2's 3 empty banks.

I'm getting some messed up text by a couple of the Midenhall Castle NPCs using either value (either blank windows or a sparse random letter) so i figured I'll have to look at those pointers, but I want to make sure I'm not doing this with the wrong bank value.
Yup, that makes sense. The main script's pointer table actually starts at 0xB762, so you're missing the first 8 pointers, but they're probably fine so I'm not too concerned about that. The pointer table also stops at 0xB7C1, so the other 16 bytes you included here are just the start of the original second script bank's text and aren't really relevant. If you look at the rest of the pointer values, you'll see they keep increasing from $9016 at 0xB772 up to $BB37 at 0xB7A6 and then reset (that'll be the first pointer following the auto-jump) to $0243 at 0xB7A8 and start increasing again up to $390A at 0xB7C0, so those pointer values are all off by $8000 and giving you who knows what from system RAM instead of the data you want from cartridge ROM (as an added bonus, the $2002 PPU register is sensitive to reads, so if you're really lucky, you might also get some other graphical mayhem happening :D). Checking your insert script, I see that you've updated where to jump to when the insertion point reaches 0x17FE7 but didn't update the corresponding header value, which is throwing the pointer value calculations off. Try changing that to #AUTOCMD($17FE7, #HDR($28010)) instead and you should get the right pointer values.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 19, 2019, 12:05:26 pm
It occurred to me just before reading your reply that it was probably the pointer values being off for the new bank (you did post that "formula" for pointer values a while back). Had no idea that was what the HDR command in the script was for. Lesson learned for the future, but yep, that did it, it's working beautifully. Just got to start testing and tweaking and then doing all the menu stuff. Man that was nuts but that's a huge hurdle cleared for me thank you so much! :beer:

Edit: Chicken Knife gave me his atlas.txt so I could compare translations and I realized you have the "pointer formula" right in there. Doh! XP
Title: Re: General NES Hacking Questions
Post by: abw on April 20, 2019, 12:51:11 pm
Lesson learned for the future, but yep, that did it, it's working beautifully. Just got to start testing and tweaking and then doing all the menu stuff. Man that was nuts but that's a huge hurdle cleared for me thank you so much! :beer:
Sweet! Let's hope the testing and tweaking goes well too :).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 22, 2019, 01:54:37 pm
Okay sorry to post in this thread so soon, but I'm genuinely stumped by the plural rules (I foolishly thought it was a bunch of suffixes in a row, kinda like the dictionary)  . With new monster names need a couple of tweaks not covered by the current plural rules

New names

Mummy Boy > Boys
Hunter Mech > Mechs
Madusa > Madusae
Dragonfry > No change
Gigantes > No change (though I'm not 100% sure if a plural is even needed here, I don't think Gigantes are encountered in groups)
Cyclops > No change (actually I just read here (https://www.grammarphobia.com/blog/2013/08/cyclops.html) that Square-Enix goofed and that the correct plural for a cyclops is "Cyclopes", though I suppose if need be, Cyclopses isn't entirely wrong and off limits)
Technically Man o' War is listed as "Men o' War" as its plural but I figure with the way the system counts from the end and adds to the end Man o' Wars would be just fine

The good news is that the new names also free up the need for a few rules

ch > ches (only the Hunter Mech is used and that's a case where it only needs the s added)
dead > no change
ngo > no change
f > ves (unused anyway)
i > ies
Mouse/mouse > Mice/mice (they're rats now)
rus>rii
sh>shes


I mean I get that there's essentially two parts of that code. One looks at the end letters and then points to other part that removes/adds letters at the end. Even for something like editing the "ngo" if I wanted to change it to "fry" I only see the "n" and "g" to change, I don't see the o

Code: [Select]
; -o pluralization handler: -ngo -> -ngo, -o -> -os
0x01C845|$07:$8835:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
0x01C848|$07:$8838:C9 10    CMP #$10 ; "g"
0x01C84A|$07:$883A:D0 E7    BNE $8823 ; if not -go, append "s"
0x01C84C|$07:$883C:BD EF 60 LDA $60EF,X ; read third-last letter of monster name
0x01C84F|$07:$883F:C9 17    CMP #$17 ; "n"
0x01C851|$07:$8841:D0 E0    BNE $8823 ; if not -ngo, append "s"
0x01C853|$07:$8843:F0 EB    BEQ $8830 ; if -ngo, plural = singular

I guess what really confuses me are the if/if not suffix lines. How do I change THOSE?

Edit: Also doing Menus (no trouble understanding those :) ) but are you sure those pointers you listed on the wiki are right, abw? I'm getting some really broken up strings like

Nevermind, I misread the text start value as 7656 instead of 76E6 which was throwing me off.
Title: Re: General NES Hacking Questions
Post by: abw on April 22, 2019, 08:20:23 pm
Okay sorry to post in this thread so soon, but I'm genuinely stumped by the plural rules (I foolishly thought it was a bunch of suffixes in a row, kinda like the dictionary)  .
Nope, the pluralization rules are implemented as actual code, not just a table of suffixes or similar, so the bad news is you're going to have to roll up your sleeves and dig in to it a bit in order to make your changes. On the other hand, since you're going to be making changes anyway, you're free to make all the changes you want - you've got all the space from 0x01C801 to 0x01C8F3 to do whatever you want, and if that isn't enough space, most of the rest of the bank is empty. When this code starts running, the singular monster name has already been laid out in RAM at $6119 for you, and by the end, you just need to write the plural form to RAM starting at $60F1 and SEC before you RTS. You can use the debugger to set a breakpoint at the start and step through the code to watch exactly what it does.

For -ngo specifically, it works like this:

$87FC: LDA $60F1,X    ; read the final letter of the monster name into A (in this case that's "o", a.k.a. #$18; I think this is the part you missed)
$87FF: CMP #$18       ; among other things, this sets the Z (zero) processor flag based on whether A is #$18 ("o") or not
$8801: BEQ $8835      ; since Z is set, BEQ follows the branch

; at this point we know the last letter was "o", so we'll check the second-last letter
$8835: LDA $60F0,X    ; read the second-last letter the of monster name into A (in this case that's "g", a.k.a. #$10)
$8838: CMP #$10       ; as before, this sets the Z flag based on whether A is #$10 ("g") or not
$883A: BNE $8823      ; since A is #$10, Z is not set and we don't take this branch

; at this point we know the last two letters were "go", so we'll check the third-last letter
$883C: LDA $60EF,X    ; read the third-last letter of the monster name (in this case that's "n", a.k.a. #$17)
$883F: CMP #$17       ; as before, this sets the Z flag based on whether A is #$17 ("n") or not
$8841: BNE $8823      ; since A is #$17, Z is not set and we don't take this branch
$8843: BEQ $8830      ; ... which means we do take this branch
$8830: SEC            ; the calling code cares about whether C is set or not
$8831: RTS            ; and at this point we're done, not having changed a single byte of the monster name
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 22, 2019, 10:51:19 pm
Nope, the pluralization rules are implemented as actual code, not just a table of suffixes or similar, so the bad news is you're going to have to roll up your sleeves and dig in to it a bit in order to make your changes. On the other hand, since you're going to be making changes anyway, you're free to make all the changes you want - you've got all the space from 0x01C801 to 0x01C8F3 to do whatever you want, and if that isn't enough space, most of the rest of the bank is empty. When this code starts running, the singular monster name has already been laid out in RAM at $6119 for you, and by the end, you just need to write the plural form to RAM starting at $60F1 and SEC before you RTS. You can use the debugger to set a breakpoint at the start and step through the code to watch exactly what it does.

For -ngo specifically, it works like this:

$87FC: LDA $60F1,X    ; read the final letter of the monster name into A (in this case that's "o", a.k.a. #$18; I think this is the part you missed)
$87FF: CMP #$18       ; among other things, this sets the Z (zero) processor flag based on whether A is #$18 ("o") or not
$8801: BEQ $8835      ; since Z is set, BEQ follows the branch

; at this point we know the last letter was "o", so we'll check the second-last letter
$8835: LDA $60F0,X    ; read the second-last letter the of monster name into A (in this case that's "g", a.k.a. #$10)
$8838: CMP #$10       ; as before, this sets the Z flag based on whether A is #$10 ("g") or not
$883A: BNE $8823      ; since A is #$10, Z is not set and we don't take this branch

; at this point we know the last two letters were "go", so we'll check the third-last letter
$883C: LDA $60EF,X    ; read the third-last letter of the monster name (in this case that's "n", a.k.a. #$17)
$883F: CMP #$17       ; as before, this sets the Z flag based on whether A is #$17 ("n") or not
$8841: BNE $8823      ; since A is #$17, Z is not set and we don't take this branch
$8843: BEQ $8830      ; ... which means we do take this branch
$8830: SEC            ; the calling code cares about whether C is set or not
$8831: RTS            ; and at this point we're done, not having changed a single byte of the monster name


Thanks that explains it a little better. I now have an idea of what to do, I just don't know the commands. I realize fixing "Mechs" would be the easiest thing, I would just have to remove the line that checked the second to last letter for a c $8875-8878

But I looked at your notes on the Wiki and sort of brainstormed what I WANT to do.

Fixing S rules for Gigantes, Magus
Code: [Select]
; read second-last letter of monster name
; "u"
; if not -us, append "es"
; read third-last letter of monster name
; "g"
; if not -gus, append "es"
; read second-last letter of monster name
; "e"
; if not -es, append "es"
; read third-last letter of monster name
; "t"
; if -tes, don't change $8830
; "i"
; replace -us with -i
; Go to $8830
; replace final letter with "i"
; append "es"


Fixing Mummy Boy and Dragonfry
Code: [Select]
; "i"
; read second-last letter of monster name
; "o"
; if "-oy", append "s"
; "r"
; if "-ry", don't change ($8830)

Fixing Madusa

Code: [Select]
; read second-last letter of monster name
; "s"
; if not -sa, append "s"
; read third-last letter of monster name
; "u"
; "e"append e
; Go to $8830

It's something like that right? Again I get the programming gist of it, but I'm not sure what hex notation does what.

On a different note, thankfully window editing has gone quite well. I can see what you're talking about a few posts ago with the US "addition" of extra names. By expanding the Battle Command window, I realized I could display full names at the top of the window now, but strangely any unused character spaces still "cover" the top bar of the window. That's a real shame, otherwise it's neat! I'm thinking of keeping the change.

Edit: So as brought to my attention by laser lambert, apparently the special [(s)] (F2) doesn't work the same way as DW1. If used for "points", whenever there's 1, it just cuts off the string (experience) or doesn't display the whole window at all (points of damage in battle). Don't want to be a bother (I really really don't I swear! ^^'') But even if I used the Trace Logger/Debugger to find the routine I wouldn't know what to look for, I'm mostly just curious, but is there any way to fix that?
Title: Re: General NES Hacking Questions
Post by: abw on April 24, 2019, 12:06:32 am
When I was doing this for Latin, I took my list of singular monster names (I excluded the unique bosses since they never need plural forms), sorted it right-to-left so that I had all the same suffixes grouped together, wrote out the correct plural form for each name, did some extra Latin-specific stuff you won't need to deal with for English, and then wrote out the singular -> plural transformation rules I needed. Once I had that, writing the code was fairly straightforward; it's a little bit long and boring, but the individual sections are pretty simple. This is an oversimplification (particularly for CMP, which does multiple other things simultaneously), but for the purposes of this exercise, you can treat CMP #$hex as "check if A is #$hex", BEQ $addr as "if it is, go to $addr", and BNE $addr as "if it isn't, go to $addr".

As with script editing, I recommend writing code as code rather than typing things in to a hex editor, so you'll want to grab an assembler (anything that works for 65816 should also work for 6502, so pick your favourite SNES or NES assembler; I was actually using Asar (http://www.romhacking.net/utilities/964/), but I've been meaning to give xkas-plus (http://www.romhacking.net/forum/index.php?topic=19640.0) a try), and if you haven't done any ASM before, you'll also want some reference material (I like the handy instruction descriptions in chapter 18 of Programming the 65816 (http://www.romhacking.net/documents/423/), but feel free to consult other resources if that one doesn't suit your style).

Also, if you get the chance to be clever, take it! E.g. if you only have one monster name that ends in "r" and you're writing the code to handle names that end in "r", you don't have to check that the 2nd-last letter is "a" or the 3rd-last letter is "W" or anything else, you can immediately just change the 9th-last letter to "e" to get "Man o' War" pluralized to "Men o' War".

By expanding the Battle Command window, I realized I could display full names at the top of the window now, but strangely any unused character spaces still "cover" the top bar of the window. That's a real shame, otherwise it's neat! I'm thinking of keeping the change.
Check out the screenshots from my translation - I also added some code for changing trailing spaces in hero names to top borders when the names are displayed as part of a border :P.

Edit: So as brought to my attention by laser lambert, apparently the special [(s)] (F2) doesn't work the same way as DW1. If used for "points", whenever there's 1, it just cuts off the string (experience) or doesn't display the whole window at all (points of damage in battle). Don't want to be a bother (I really really don't I swear! ^^'') But even if I used the Trace Logger/Debugger to find the routine I wouldn't know what to look for, I'm mostly just curious, but is there any way to fix that?
Much like the [name] control code, [(s)] just tests the 16-bit value at $8F; the code for that is actually right before the pluralization code:
Code: [Select]
; data -> code
; if $8F-$90 == #$0001, print "s" + [end-FA] to $60F1 and SEC, else print [end-FA] and CLC
; indirect control flow target
; from $02:$BE37 via $8006
0x01C7E8|$07:$87D8:A5 90    LDA $90
0x01C7EA|$07:$87DA:D0 05    BNE $87E1 ; if $90 > 0, add "s"
0x01C7EC|$07:$87DC:A4 8F    LDY $8F
0x01C7EE|$07:$87DE:88      DEY
0x01C7EF|$07:$87DF:F0 0C    BEQ $87ED ; if $90 == 0 and $8F - 1 == 0 (i.e. $8F == 1), do not add "s"
; control flow target (from $87DA)
0x01C7F1|$07:$87E1:A9 1C    LDA #$1C ; "s"
0x01C7F3|$07:$87E3:8D F1 60 STA $60F1
0x01C7F6|$07:$87E6:A9 FA    LDA #$FA ; [end-FA]
0x01C7F8|$07:$87E8:8D F2 60 STA $60F2
0x01C7FB|$07:$87EB:38      SEC
0x01C7FC|$07:$87EC:60      RTS

; control flow target (from $87DF)
0x01C7FD|$07:$87ED:A9 FA    LDA #$FA ; [end-FA]
0x01C7FF|$07:$87EF:18      CLC
0x01C800|$07:$87F0:60      RTS
The [number] control code, which is used for both experience and gold gains, also uses $8F-$90, so I would have expected [(s)] to work for both of those situations.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 24, 2019, 06:41:42 am
When I was doing this for Latin, I took my list of singular monster names (I excluded the unique bosses since they never need plural forms), sorted it right-to-left so that I had all the same suffixes grouped together, wrote out the correct plural form for each name, did some extra Latin-specific stuff you won't need to deal with for English, and then wrote out the singular -> plural transformation rules I needed. Once I had that, writing the code was fairly straightforward; it's a little bit long and boring, but the individual sections are pretty simple. This is an oversimplification (particularly for CMP, which does multiple other things simultaneously), but for the purposes of this exercise, you can treat CMP #$hex as "check if A is #$hex", BEQ $addr as "if it is, go to $addr", and BNE $addr as "if it isn't, go to $addr".

As with script editing, I recommend writing code as code rather than typing things in to a hex editor, so you'll want to grab an assembler (anything that works for 65816 should also work for 6502, so pick your favourite SNES or NES assembler; I was actually using Asar (http://www.romhacking.net/utilities/964/), but I've been meaning to give xkas-plus (http://www.romhacking.net/forum/index.php?topic=19640.0) a try), and if you haven't done any ASM before, you'll also want some reference material (I like the handy instruction descriptions in chapter 18 of Programming the 65816 (http://www.romhacking.net/documents/423/), but feel free to consult other resources if that one doesn't suit your style).

Also, if you get the chance to be clever, take it! E.g. if you only have one monster name that ends in "r" and you're writing the code to handle names that end in "r", you don't have to check that the 2nd-last letter is "a" or the 3rd-last letter is "W" or anything else, you can immediately just change the 9th-last letter to "e" to get "Man o' War" pluralized to "Men o' War".

Thanks. I was worried my post came across too "Wahh I don't wanna! Just do it for me please." which wasn't what I intended at all. But that's exactly the info I was hoping for!

Quote
Check out the screenshots from my translation - I also added some code for changing trailing spaces in hero names to top borders when the names are displayed as part of a border :P.

Ah, you sly dog! Guess I'll have to download your translation and see what you did.

Speaking of. I was earnestly going to try to find monster and spell name length and used your explanation in that previous DW1 thread. But looking it over it confused me more than I thought it would.

In your example you said I could look up HEAL by setting a Read Breakpoint at $B5E6. The thing is looking back at the rom, I can't see what B5E6 is supposed to be. The pointer value was something similar but different like B8B6 (sorry away from my computer right now, something like that).

Am I missing a real obvious conversion or something? I thought the magic RAM to ROM conversion number was 3FF0?

Quote
Much like the [name] control code, [(s)] just tests the 16-bit value at $8F; the code for that is actually right before the pluralization code:
Code: [Select]
; data -> code
; if $8F-$90 == #$0001, print "s" + [end-FA] to $60F1 and SEC, else print [end-FA] and CLC
; indirect control flow target
; from $02:$BE37 via $8006
0x01C7E8|$07:$87D8:A5 90    LDA $90
0x01C7EA|$07:$87DA:D0 05    BNE $87E1 ; if $90 > 0, add "s"
0x01C7EC|$07:$87DC:A4 8F    LDY $8F
0x01C7EE|$07:$87DE:88      DEY
0x01C7EF|$07:$87DF:F0 0C    BEQ $87ED ; if $90 == 0 and $8F - 1 == 0 (i.e. $8F == 1), do not add "s"
; control flow target (from $87DA)
0x01C7F1|$07:$87E1:A9 1C    LDA #$1C ; "s"
0x01C7F3|$07:$87E3:8D F1 60 STA $60F1
0x01C7F6|$07:$87E6:A9 FA    LDA #$FA ; [end-FA]
0x01C7F8|$07:$87E8:8D F2 60 STA $60F2
0x01C7FB|$07:$87EB:38      SEC
0x01C7FC|$07:$87EC:60      RTS

; control flow target (from $87DF)
0x01C7FD|$07:$87ED:A9 FA    LDA #$FA ; [end-FA]
0x01C7FF|$07:$87EF:18      CLC
0x01C800|$07:$87F0:60      RTS
The [number] control code, which is used for both experience and gold gains, also uses $8F-$90, so I would have expected [(s)] to work for both of those situations.

Thanks again for this. How strange but I guess that's why the table only uses it for Gold and nothing else, maybe the devs just couldn't get it working properly? In a way I don't find it with fretting over. I could just tweak the script and then change my "point(s)" dictionary entry and be able to save MORE space that way.
Title: Re: General NES Hacking Questions
Post by: abw on April 24, 2019, 08:09:25 pm
Thanks. I was worried my post came across too "Wahh I don't wanna! Just do it for me please." which wasn't what I intended at all. But that's exactly the info I was hoping for!
We aim to please ;D.

Ah, you sly dog! Guess I'll have to download your translation and see what you did.
You won't be able to completely copy my code since I cannibalized the free space I created by shortening "ADVENTURE LOG" to "VOLUMEN", but the ASM for handling menu control codes $98 - $9F starts at 0x3ED8A; the original game ran the same code for all of $9A - $9F, so I stole $9B - $9F for the "names in border" code.

Speaking of. I was earnestly going to try to find monster and spell name length and used your explanation in that previous DW1 thread. But looking it over it confused me more than I thought it would.

In your example you said I could look up HEAL by setting a Read Breakpoint at $B5E6. The thing is looking back at the rom, I can't see what B5E6 is supposed to be. The pointer value was something similar but different like B8B6 (sorry away from my computer right now, something like that).

Am I missing a real obvious conversion or something? I thought the magic RAM to ROM conversion number was 3FF0?
Sorry, part of the problem here is that I can't type properly :'(. I've updated that post to use the correct breakpoint of $BE56 as shown in the debugger snapshot instead of $B5E6; after that, it goes back to the relationship between RAM and ROM addresses: the read breakpoint uses RAM address $BE56, which for DW1 corresponds to a ROM offset of 0x3E66 (not the address you're looking for), 0x7E66 (pick me because I say "HEAL"!), or 0xBE66 (also not the address you're looking for), or theoretically 0xFE66 (but this is reserved for the fixed bank, so it's also not the address you're looking for). Does it make any more sense like this?
DW1 ROM Offset     DW1 RAM Address   DW1 RAM to ROM Conversion Number
$0010 - $400F      $8000 - $BFFF     -$7FF0
$4010 - $800F      $8000 - $BFFF     -$3FF0
$8010 - $C00F      $8000 - $BFFF     +$10
$C010 - $1000F     $C000 - $FFFF     +$4010
Thanks again for this. How strange but I guess that's why the table only uses it for Gold and nothing else, maybe the devs just couldn't get it working properly? In a way I don't find it with fretting over. I could just tweak the script and then change my "point(s)" dictionary entry and be able to save MORE space that way.
Wait a minute... Am I missing something, or is getting 1 piece of gold in this game actually impossible? Slimes are worth 2 gold, unique saleable items go for 2 gold, and even buying an Antidote Herb with a Golden Card still gets you a discount of 2 gold...

Hmm, yup, that F2 is just plain broken when $8F-$90 is #$0001. Instead of
Code: [Select]
LDA #$FA
CLC
RTS
which as you've seen results in the rest of the string getting cut off, it should be doing
Code: [Select]
LDA #$FA
STA $60F1
SEC
RTS
So the good news is: it's not you, it's the game :P. Replacing the CLC/RTS with BNE $87E3 should be enough to fix that up.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 25, 2019, 02:47:18 pm
Okay, really embarrassing question but I can't seem to open the file in either program you recommended. Asar seems to ONLY accept SNES roms and xkas plus seems to only accept dissembled code. Since you've said you've been using the former, I feel like I'm missing something incredibly obvious.

Edit: Also, I thought I was getting your explanation of Read Breakpoints. I thought to find Monster Length I'd place a Read Breakpoint for Slime which the Pointer points to as memory at B718 (to avoid possible confusion I'll just say that with Chicken Knife sharing his atlas script I thought I'd use the work you guys did in moving the Item and Monster names). I figured it'd have to be CPU memory because PPU memory doesn't let me put in B718 (I assume 3ff0 is the maximum?) And yet when I try to go in to a battle with a Slime, the Debugger doesn't trip anything. And yes I made sure it's enabled before you ask :p It's an easy thing to miss, so I wouldn't blame you if you asked.
Title: Re: General NES Hacking Questions
Post by: abw on April 25, 2019, 05:43:04 pm
Okay, really embarrassing question but I can't seem to open the file in either program you recommended. Asar seems to ONLY accept SNES roms and xkas plus seems to only accept dissembled code. Since you've said you've been using the former, I feel like I'm missing something incredibly obvious.

Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:
Quote from: test.asm
; Example NES 6502 ASM file: writes a small infinite loop.
; Put this file in the same directory as asar and execute it with e.g.
;   copy /Y nul test.bin
;   asar -nocheck test.asm test.bin
; After that, test.bin should contain 64 KB of #$00 followed by "A9 00 4C 00 80"

norom   ; stop Asar from trying to apply SNES memory mapping to this NES code
org $10010   ; set the ROM file insertion point to 0x10010
base $8000   ; set the starting RAM address to $8000

loop:
   LDA #$00
   JMP loop

Edit: Also, I thought I was getting your explanation of Read Breakpoints. I thought to find Monster Length I'd place a Read Breakpoint for Slime which the Pointer points to as memory at B718 (to avoid possible confusion I'll just say that with Chicken Knife sharing his atlas script I thought I'd use the work you guys did in moving the Item and Monster names). I figured it'd have to be CPU memory because PPU memory doesn't let me put in B718 (I assume 3ff0 is the maximum?) And yet when I try to go in to a battle with a Slime, the Debugger doesn't trip anything. And yes I made sure it's enabled before you ask :p It's an easy thing to miss, so I wouldn't blame you if you asked.
Yeah, unless you're specifically looking for graphics stuff, a CPU breakpoint is probably what you want. The original monster list was at 0x1B728 a.k.a. $06:$B718, but I moved my monster list to 0x1D050, a.k.a. $07:$9040, so if you're looking for Slime, that's where it'll be.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 27, 2019, 07:40:19 pm
Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:Yeah, unless you're specifically looking for graphics stuff, a CPU breakpoint is probably what you want. The original monster list was at 0x1B728 a.k.a. $06:$B718, but I moved my monster list to 0x1D050, a.k.a. $07:$9040, so if you're looking for Slime, that's where it'll be.

I guess I figured what was there at the pointer would still work. Thanks.

Well I got the game to stop and saw
 0F:F47B:8D A0 60  STA $60A0 = #$0B

And thanks to laserlambert's testing I know that the line 1 of monster names gets cut off at 11 letters, so I figure that's GOTTA be it right? Am I right in thinking maybe this ISN'T a hardcoded value but something that's loaded in memory? I also know that the second line for monsters is limited to 9 letters which is at least 1 letter short for my new monster names and even when trying to make a breakpoint based on the Monster Line 2 pointer, I got a break but couldn't find anythign resembling a 9 letter limit. I even went BACK to DW1 and tried to recreate the steps you did in finding the length limit of HEAL. made a breakpoint and had it stop when casting and found

01:A868:AE E2 64  LDX $64E2 = #$0F

Note this is from my hack, so i figured this would have to be the new 15 letter limit for spells. If so, why does this use LDX? And how did you turn that into the ROM address of $77E9? Or am I just not looking at the right thing at all?
Title: Re: General NES Hacking Questions
Post by: abw on April 28, 2019, 09:21:17 am
This sounds okay as far as it goes, but you haven't hit ROM yet, so you need to keep following the trail a bit further. Once you find the value you're looking for being read from somewhere in the $8000 - $FFFF range, then you can stop and convert the RAM address to a ROM address (/ get FCEUX to do it for you if you want).

0F:F47B:8D A0 60  STA $60A0 = #$0B
This shows that the game is about to store whatever the current value of A is to $60A0 (which was #$0B just before that instruction executed), so you'll need to find out where $60A0 became #$0B in the first place.

01:A868:AE E2 64  LDX $64E2 = #$0F
Similarly, this shows that the game is loading X with the value of $64E2, which happens to be #$0F; you'll need to find out how $64E2 became #$0F.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on May 04, 2019, 06:40:23 pm
Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:

Sorry, I did this and ran it, it made the bin file but I'm still getting the "Not an SNES ROM Error" You might have to dumb it down even more for me. (9_6)

This sounds okay as far as it goes, but you haven't hit ROM yet, so you need to keep following the trail a bit further. Once you find the value you're looking for being read from somewhere in the $8000 - $FFFF range, then you can stop and convert the RAM address to a ROM address (/ get FCEUX to do it for you if you want).
This shows that the game is about to store whatever the current value of A is to $60A0 (which was #$0B just before that instruction executed), so you'll need to find out where $60A0 became #$0B in the first place.
Similarly, this shows that the game is loading X with the value of $64E2, which happens to be #$0F; you'll need to find out how $64E2 became #$0F.

Okay so I used trace logger to log the data from the world map to the start of a battle to the breakpoint. Still not seeing what I want. I tried to put an Execution Breakpoint on 60A0, but I don't think I'm doing it right. When doubleclicking in the debugger it just gives me "K==#00" as the "condition" and when I try to change it I get an invalid condition.

Really sorry about this. It's hard not to feel like that guy who was trying to translate DW1 into Spanish. It's frustrating to feel so clueless, but this is uncharted territory for me and I'm determined to pick up SOMETHING new for later games.

On another note, the Assembly guide you posted is really handy, thanks! It's nice to know what all those 3 letter terms mean.
Title: Re: General NES Hacking Questions
Post by: abw on May 15, 2019, 06:06:35 pm
Sorry for the delay in responding, I've been offline for the past couple of weeks!

Sorry, I did this and ran it, it made the bin file but I'm still getting the "Not an SNES ROM Error" You might have to dumb it down even more for me. (9_6)
I'm not sure how much further down I can go :P. Go to the directory containing Asar, copy the sample ASM I provided into a new file named test.asm, make an empty file named test.bin, and then open a command prompt in that directory and run "asar -nocheck test.asm test.bin". Works like a charm for me.

Okay so I used trace logger to log the data from the world map to the start of a battle to the breakpoint. Still not seeing what I want. I tried to put an Execution Breakpoint on 60A0, but I don't think I'm doing it right. When doubleclicking in the debugger it just gives me "K==#00" as the "condition" and when I try to change it I get an invalid condition.
For $60A0, you'd want a write breakpoint since you're looking for places where the game writes #$0B to $60A0. Execute breakpoints fire when the code at the address you set the breakpoint for gets executed (e.g. the F47B in "0F:F47B:8D A0 60  STA $60A0 = #$0B") and read/write breakpoints fire when the address you set the breakpoint for gets modified by some code (e.g. 60A0 is being written to in "0F:F47B:8D A0 60  STA $60A0 = #$0B").

With the trace log, you don't necessarily need to set any breakpoints; they just help to reduce the size of the log file you need to look through. If I do the same thing as you with an unaltered ROM, making a trace log from the world map to the start of a battle, it's a huge file but I can search it for $B718 (the start of the monster list) to get:
Code: [Select]
        $F422:A0 00     LDY #$00                                     A:18 X:00 Y:00 S:F7 P:nvUBdIzc
        $F424:AE A0 60  LDX $60A0 = #$0B                             A:18 X:00 Y:00 S:F7 P:nvUBdIZc
        $F427:B1 57     LDA ($57),Y @ $B718 = #$36                   A:18 X:0B Y:00 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $010A = #$5F                   A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:36 X:0B Y:01 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:36 X:0A Y:01 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B719 = #$15                   A:36 X:0A Y:01 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0109 = #$5F                   A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:15 X:0A Y:02 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:15 X:09 Y:02 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71A = #$12                   A:15 X:09 Y:02 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0108 = #$5F                   A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:12 X:09 Y:03 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:12 X:08 Y:03 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71B = #$16                   A:12 X:08 Y:03 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0107 = #$5F                   A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:16 X:08 Y:04 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:16 X:07 Y:04 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71C = #$0E                   A:16 X:07 Y:04 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0106 = #$5F                   A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:0E X:07 Y:05 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:0E X:06 Y:05 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71D = #$FF                   A:0E X:06 Y:05 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:FF X:06 Y:05 S:F7 P:NvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:FF X:06 Y:05 S:F7 P:nvUBdIZC
        $F434:60        RTS (from $F3FE) --------------------------- A:FF X:06 Y:05 S:F7 P:nvUBdIZC
which shows that the game is copying data from $B718-$B71D to $010A-$0106 (stored backwards) until it reads a #$FF (monster name end token) or X reaches #$00, and that X was set based on $60A0.

Spoiler:
As a side note, a little further down, you'll see the game copies the monster name from $00FF,X to $6119,Y, where it will eventually get used by the [name] control code:
Code: [Select]
      $FCE8:AE A0 60  LDX $60A0 = #$0B                             A:FF X:06 Y:00 S:F9 P:nvUBdIZc
      $FCEB:BD FF 00  LDA $00FF,X @ $010A = #$36                   A:FF X:0B Y:00 S:F9 P:nvUBdIzc
      $FCEE:99 19 61  STA $6119,Y @ $6119 = #$25                   A:36 X:0B Y:00 S:F9 P:nvUBdIzc
      $FCF1:C8        INY                                          A:36 X:0B Y:00 S:F9 P:nvUBdIzc
      $FCF2:CA        DEX                                          A:36 X:0B Y:01 S:F9 P:nvUBdIzc
      $FCF3:D0 F6     BNE $FCEB                                    A:36 X:0A Y:01 S:F9 P:nvUBdIzc
...

Searching backwards in the trace log for $60A0, the very first result is this:
Code: [Select]
     $FC92:A9 0B     LDA #$0B                                     A:00 X:00 Y:00 S:FA P:nvUBdIZc
     $FC94:8D A0 60  STA $60A0 = #$01                             A:0B X:00 Y:00 S:FA P:nvUBdIzc
So $60A0 got its value from A, and A got its value set based on $FC93 (the #$0B part of "LDA #$0B"), which unlike $60A0 comes from ROM i.e. 0x3FCA3. Ta-da!

With that as a guide, see if you can find where the maximum length of the second "line" of monster names in the main dialogue box is set (hint: it's #$09 and it's not too far away from where the maximum length of the first "line" is set), and then see if you can track down where the lengths for each of the two lines in the monster menu list get set (hint: same values as the dialogue box lengths, but set in a different area of the code; they'll still be in your trace log, though).

Really sorry about this. It's hard not to feel like that guy who was trying to translate DW1 into Spanish. It's frustrating to feel so clueless, but this is uncharted territory for me and I'm determined to pick up SOMETHING new for later games.

On another note, the Assembly guide you posted is really handy, thanks! It's nice to know what all those 3 letter terms mean.
Yeah, if you're not used to this kind of thing, it can take a while to really sink in. Just keep at it and you'll get the hang of it sooner or later!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on May 18, 2019, 07:11:35 am
Hey! Glad to see your back and that you're okay! I was genuinely worried for a bit there that something bad might've happened that would've taken you out of the picture. While I'm sure I could've found help, you're a cool guy and it would've been a bummer to not be able to finish this while learning how to do it on my own. But anyway...

I'm not sure how much further down I can go :P. Go to the directory containing Asar, copy the sample ASM I provided into a new file named test.asm, make an empty file named test.bin, and then open a command prompt in that directory and run "asar -nocheck test.asm test.bin". Works like a charm for me.

I wonder if it's a Windows thing, are you on 10? When I try to run that very same command from the command line, it seems to work for a second but just takes me back to the command line with a modified test.bin and trying to run asar again just gives me the usual.


Searching backwards in the trace log for $60A0, the very first result is this:
Code: [Select]
     $FC92:A9 0B     LDA #$0B                                     A:00 X:00 Y:00 S:FA P:nvUBdIZc
     $FC94:8D A0 60  STA $60A0 = #$01                             A:0B X:00 Y:00 S:FA P:nvUBdIzc
So $60A0 got its value from A, and A got its value set based on $FC93 (the #$0B part of "LDA #$0B"), which unlike $60A0 comes from ROM i.e. 0x3FCA3. Ta-da!

With that as a guide, see if you can find where the maximum length of the second "line" of monster names in the main dialogue box is set (hint: it's #$09 and it's not too far away from where the maximum length of the first "line" is set), and then see if you can track down where the lengths for each of the two lines in the monster menu list get set (hint: same values as the dialogue box lengths, but set in a different area of the code; they'll still be in your trace log, though).

So on one hand I DID find the second line monster value of 9 at 3FCBF, though I'm not sure how you turned FC93 to 3FCA3. I mean yeah you added 30010 but where did THAT come from? It doesn't quite match up with what you were talking about RAM to ROM addresses on the last page.

And I made a honest effort, but I can't seem to find what you're talking about for the Monster List window. I DID find the subroutine FCE8 in my trace log and while trying to understand it still makes my eyes go @_@ I can understand enough that there's two sections concerning whether or not the Monster needs that second line printed in the window. And I can see that it loads the value as X as opposed to A in the main dialog window. But can't seem to find anything in my log about the value being stored in 60A0.
Title: Re: General NES Hacking Questions
Post by: abw on May 18, 2019, 12:57:54 pm
Hey! Glad to see your back and that you're okay! I was genuinely worried for a bit there that something bad might've happened that would've taken you out of the picture. While I'm sure I could've found help, you're a cool guy and it would've been a bummer to not be able to finish this while learning how to do it on my own. But anyway...
Yeah, every now and then I go offline for a couple of weeks for IRL stuff, though one time it was for an entire year!

I wonder if it's a Windows thing, are you on 10? When I try to run that very same command from the command line, it seems to work for a second but just takes me back to the command line with a modified test.bin and trying to run asar again just gives me the usual.
I'm actually on Windows 7 (cuz eww 8 and 10), but if you're getting a modified test.bin, then it sounds like Asar is working. Try changing test.asm and see if you get a different test.bin.

So on one hand I DID find the second line monster value of 9 at 3FCBF,
Nice job!

though I'm not sure how you turned FC93 to 3FCA3. I mean yeah you added 30010 but where did THAT come from? It doesn't quite match up with what you were talking about RAM to ROM addresses on the last page.
If the Trace Logger included bank number, $FC93 would show up as $0F:$FC93, and $0F * $4000 - $8000 - $01 * $4000 + $FC93 + $10 = $3FCA3 (i.e. <ROM bank number> * <bank size> - <base RAM-to-ROM offset> - <RAM bank number> * <bank size> + <RAM address> + <iNES header size>). Or find $FC93 in the Hex Editor -> right click -> Go Here In ROM File.

And I made a honest effort, but I can't seem to find what you're talking about for the Monster List window. I DID find the subroutine FCE8 in my trace log and while trying to understand it still makes my eyes go @_@
Ah, $FCE8's not so bad :P. When you're trying to wrap your head around a block of code, remember that the Debugger and Trace Logger give you two different views of the same thing; sometimes it's easier to understand what's going on when looking at one instead of the other. Here's the basic code you'll see in the Debugger:
Code: [Select]
0F:FCE8:AE A0 60 LDX $60A0
0F:FCEB:BD FF 00 LDA $00FF,X
0F:FCEE:99 19 61 STA $6119,Y
0F:FCF1:C8      INY
0F:FCF2:CA      DEX
0F:FCF3:D0 F6    BNE $FCEB
0F:FCF5:60      RTS

and here's a commented version:
Spoiler:
Code: [Select]
; copy $60A0 bytes of data from $00FF,X to $6119,Y
; X is used as a read index, Y as a write index
; data gets copied in reverse order
; IN:
; A/X/C = irrelevant
; Y = current write index
; OUT:
; A = last byte copied (but calling code doesn't care)
; X = 0
; Y = current write index; this is important since the calling code needs to remember the write index from the first segment when dealing with the second segment
; C = unchanged
; control flow target (from $FC9D, $FCBA)
0x03FCF8|$0F:$FCE8:AE A0 60 LDX $60A0 ; initialize the read index to the value of $60A0
; control flow target (from $FCF3)
0x03FCFB|$0F:$FCEB:BD FF 00 LDA $00FF,X ; read data from $00FF,X
0x03FCFE|$0F:$FCEE:99 19 61 STA $6119,Y ; write data to $6119,Y
0x03FD01|$0F:$FCF1:C8      INY ; increment write index
0x03FD02|$0F:$FCF2:CA      DEX ; decrement read index
0x03FD03|$0F:$FCF3:D0 F6    BNE $FCEB ; if the read index is not 0, loop back to $FCEB
0x03FD05|$0F:$FCF5:60      RTS ; otherwise the read index is 0, so we're done

I can understand enough that there's two sections concerning whether or not the Monster needs that second line printed in the window. And I can see that it loads the value as X as opposed to A in the main dialog window. But can't seem to find anything in my log about the value being stored in 60A0.
If you keep looking for reads on $B718, you should notice that the game scans through the monster name list a few times while starting a battle; the last one is for the monster list menu (you can also easily isolate this one by starting a trace log just before pressing FIGHT, as the monster menu gets redrawn after that point). At the spot where the breakpoint fires, you'll be in the same block of code as for the main dialogue window, but coming from a different place (the stack will show something like FA,F3,C5,EF,..., which means the last JSR before you got to this code ended at $F3FA [so started at $F3F8], and the JSR before that ended at $EFC5 [$EFC3]). Searching backwards for $60A0 from there should quickly get you to:
Code: [Select]
              $EFA9:A9 0B     LDA #$0B                                     A:00 X:18 Y:07 S:F1 P:nvUBdIZc
              $EFAB:8D A0 60  STA $60A0 = #$00                             A:0B X:18 Y:07 S:F1 P:nvUBdIzc
and searching forwards will eventually (after a long series of other uses for $60A0) get you the maximum length of the second line too (or if you look in the Debugger, the code for handling the second line of monster names is only a few lines of ASM away from the code for handling the first line).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 13, 2019, 09:43:40 pm
Okay, so first of all, sorry for the delay. I mean yeah I work but a big reason is I just got nice CPU upgrade so I've been finally able to play DQXI among other things, and I think the last time I looked at this (about a week ago) my brain was like "Nope, not today..." but the Smash reveal made me go... "yeah I need to get back to this." If anything I'm just eager to get to 3 and finish the NES Erdrick trilogy at the very very least.

Anyway. Good news is I not only figured out the monster name menu length (though to your credit, you made it pretty easy in your last post) but I ALSO got the spell length for dialog.

(https://i.imgur.com/CC0s7l2.png)
(yeah I swapped out Heal for Holy Protection as a quick way to test)

I'm not sure if I want to mess with the menu length and keep it abbreviated like I did DW1 (because I know unlike DW1 not only the spell menus are done differently, but there are many more long spell names too). Otherwise that's all taken care of.

Bad news it, and I'm so sorry, but the ASM stuff is still stumping me. More, the asar usage than anything else.
I'm actually on Windows 7 (cuz eww 8 and 10), but if you're getting a modified test.bin, then it sounds like Asar is working. Try changing test.asm and see if you get a different test.bin.

Is the bin file supposed to be viewable in text form? Because opening it up in Notepad+ just gives me junk so I have no idea what to look for.
Title: Re: General NES Hacking Questions
Post by: abw on June 14, 2019, 09:21:41 am
Okay, so first of all, sorry for the delay. I mean yeah I work but a big reason is I just got nice CPU upgrade so I've been finally able to play DQXI among other things, and I think the last time I looked at this (about a week ago) my brain was like "Nope, not today..." but the Smash reveal made me go... "yeah I need to get back to this." If anything I'm just eager to get to 3 and finish the NES Erdrick trilogy at the very very least.
Heh, I hear DQXI is a leading cause of delay among DQ NES hackers ;).

Anyway. Good news is I not only figured out the monster name menu length (though to your credit, you made it pretty easy in your last post) but I ALSO got the spell length for dialog.
Congrats! It sounds like you must be getting close to finishing with this game - what's still left?

Bad news it, and I'm so sorry, but the ASM stuff is still stumping me. More, the asar usage than anything else.
Is the bin file supposed to be viewable in text form? Because opening it up in Notepad+ just gives me junk so I have no idea what to look for.
Just as viewable as any other ROM, i.e. not very :P. Like I said earlier (http://www.romhacking.net/forum/index.php?topic=27053.msg374493#msg374493), that sample ASM file should generate a file with 64 KB of zero bytes followed by the bytes "A9 00 4C 00 80", so open it up in a hex editor, scroll to the very bottom, and if you see those bytes, then it's working.

For real world usage, you'd want to adjust the org/base values to the ROM/RAM addresses you want to write to, replace the useless infinite loop ASM I concocted for the sample with whatever code you actually want to insert, and then run it against a real ROM instead of an empty file.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 21, 2019, 09:45:23 pm
Heh, I hear DQXI is a leading cause of delay among DQ NES hackers ;).

The funny thing about playing XI while working on II is spotting the references. The Puff Puff girl in Gondolia says the same thing as the girl in Lianport/Rippleport word for word from the mobile version. That was a real A-ha moment for me and I'm hoping doing these script ports more people will be able to see that.

Quote
Congrats! It sounds like you must be getting close to finishing with this game - what's still left?

Honestly outside fixing the buggy  plural s and of course the monster name ASM, just need to edit the uncompressed Prologue text and get the new graphics from Chicken Knife's hack I'll be done. I have the new menus ready to go for insertion.

Speaking of A-Ha moment, I finally get how asar works. Man I feel dumb. I was expecting some kind of fancy interface like PS2Dis or something where you can just edit the lines of code as you go.

Am I right in thinking I could just copy the Pluralization rules from the Wiki into the test.asm (replace the loop code) file but change org to $01C805 (code where it starts to check monster count) and base to $4000 (?) and then just "revise" the code to my liking and then run "asar -nocheck test.asm (DW2 rom)" Is that what you do?
Title: Re: General NES Hacking Questions
Post by: abw on June 22, 2019, 04:46:25 pm
Speaking of A-Ha moment, I finally get how asar works. Man I feel dumb. I was expecting some kind of fancy interface like PS2Dis or something where you can just edit the lines of code as you go.
Ah, yeah, it's not that fancy, alas. Quite possibly there are better assembler options out there; Asar is just the one I'm used to and it hasn't yet irritated me enough to go looking for alternatives.

Am I right in thinking I could just copy the Pluralization rules from the Wiki into the test.asm (replace the loop code) file but change org to $01C805 (code where it starts to check monster count) and base to $4000 (?) and then just "revise" the code to my liking and then run "asar -nocheck test.asm (DW2 rom)" Is that what you do?
Sort of. You'll want to set the base value (RAM address) to $87F5 and then trim out the non-ASM ROM address, RAM address, and assembled code bytes from the wiki, leaving just the opcodes and data bytes (and probably the comments, because why not?). Unless you can manage to keep the byte counts between control flow targets identical to the original code or enjoy manually counting bytes and updating lots of pointers every time you make a change, you will also want to convert the control flow addresses into labels and use those, so it'll look something like this:
Code: [Select]
LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$18 ; "o"
BEQ o_handler ; -ngo -> -ngo, -o -> -os
CMP #$0F ; "f"
BEQ f_handler ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ y_handler ; -y -> -ies
CMP #$12 ; "i"
BEQ i_handler ; -i -> -ies
CMP #$1C ; "s"
BEQ s_handler ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ h_handler ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ n_handler ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0E ; "e"
BEQ e_handler ; -mouse -> -mice, -Mouse -> -Mice, -e -> es
CMP #$0D ; "d"
BEQ d_handler ; -dead -> -dead, -d -> -ds
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
done:
SEC
RTS

o_handler:
...

Keep in mind that other unrelated code starts at $88E4, so make sure not to overwrite that; fortunately bank 7 is mostly empty, so if you need more space, you can JMP $8C13 and keep going from there.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 25, 2019, 01:31:06 pm
Okay, so, here it is!

NOTE: I'm keeping addresses and bytes here for referential purposes. I'm well aware from your last post that these should be removed for insertion.



Code: [Select]
LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$0F ; "f"
BEQ $885E ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ $8874 ; -y -> -ies
CMP #$12 ; "i"
BEQ $8863 ; -i -> -ies
CMP #$1C ; "s"
BEQ $8835 ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ $886B ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ $889C ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ e_handler ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ d_handler ; Man O' War -> Men O' War
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
done:
SEC
RTS

; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
$8835:  LDA $60F0,X ; read second-last letter of monster name
$8838:  CMP #$0E ; "e"
$883A:  BEQ $8830 ; if -es, singular=plural
$883C:  CMP #$0A ; "a"
$883F:  BEQ $8830 ; if -as, singular=plural
$8841:  CMP #$19 ; "p"
$8843:  BEQ $8830 ; if -ps, singular=plural
$8845:  LDA $60EF,X ; read third-last letter of monster name
$8848:  CMP #$10 ; "g"
$884A:  BEQ $8855   ; Shortcut here, if g_s (Magus) replace us with i
$884C:  BNE $885D ; If not either Cyclops or Atlas (or Magus), add -es

; append "es" to monster name
$884E:E8      INX
$884F:A9 0E    LDA #$0E ; "e"
$8851:9D F1 60 STA $60F1,X ; append "e" to monster name
$8854:D0 BE    BNE $8823 ; append "s" to monster name; note that this branch is always taken



$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS

; -f pluralization handler: -f -> -ves (no need to have this, but if there's space, might keep it for possible hacks)
$885E:A9 1F    LDA #$1F ; "v"
$8860:4C 6C 88 JMP $886C ; replace "f" with "v" then append "es"


; -i pluralization handler: -i -> -ies (same as above)
$8863:A9 12    LDA #$12 ; "i"
; unused control flow target (from $8867)
$8865:9D F1 60 STA $60F1,X ; replace final letter with "i"
$8868:4C 5D 88 JMP $884E ; append "es"

; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
$886B:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
$886E:C9 1C    CMP #$1C ; "s"
$8870:F0 E0    BEQ $885D ; if -sh, append "es"
$8872:D0 A4    BNE $8823 ; else, append "s"

; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
$8874 LDA $60F0,X ; read second-last letter of monster name
$8877 CMP #$18 ; "o"
$8879 BEQ $8823 ; if -oy, append "s"
$887B CMP #$18 ; "r"
$887D BEQ $8830       ; if -ry, no change
$887F BNE $886C       ; otherwise replace y with -ies

; -a pluralization handler: needed for madusa -> madusae
$8881 LDA $60F0,X ; read second-last letter of monster name
$8884 CMP #$1C ; "s"
$8886 BEQ             ; if -sa add "e"
$8888 BNE $8823       ; if not -sa append "s"

;adding "e"
$888A INX
$888B LDA #$0E ; "e"
$888D STA $60F1,X ; append "e" to monster name
$8890 JMP 8829

; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?

; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
$889C:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
$889F:C9 0A    CMP #$0A ; "a"
$88A1:D0 9D    BNE $8823 ; if not -an, append "s"
$88A3:BD EF 60 LDA $60EF,X ; read third-last letter of monster name
$88A6:C9 16    CMP #$16 ; "m"
$88A8:F0 04    BEQ $8891 ; -man -> -men
$88AA:C9 30    CMP #$30 ; "M"
$88AC:D0 92    BNE $8823 ; if not -Man, append "s"
; control flow target (from $888B)
$88AE:A9 0E    LDA #$0E ; "e"
$88B0:9D F0 60 STA $60F0,X ; replace second-last letter of monster name
; control flow target (from $88DF)
$88B3:38      SEC
$88B4:60      RTS

Not neading the -dead or -mouse codes freed up a bunch of space. So much so, I thought I'd just keep some of the extra codes I didn't necessarily need like f-ves or i-ies in case someone wants to use my script port as a base for a hack that needs new monster names.

I just had a couple of confused points. I'm not quite sure how the replace text code works so I'm a little stumped at changing the old -us to -ii code to just -us to -i (Magus to Magi)

Code: [Select]
$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS

and I just wanted to run my Man O' War to Men O' War code by you

Code: [Select]
; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
Title: Re: General NES Hacking Questions
Post by: abw on June 25, 2019, 08:39:22 pm
I just had a couple of confused points. I'm not quite sure how the replace text code works so I'm a little stumped at changing the old -us to -ii code to just -us to -i (Magus to Magi)

Code: [Select]
$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS
There's nothing magic going on here; try setting an execute breakpoint at $8853, getting into a fight with some Magic Vampirii (or temporarily rename e.g. Slime to Slrus and get into a fight with some of those), and step through the code, watching what happens around $60F1 in the Hex Editor.

Does this make it any clearer?
Spoiler:
Code: [Select]
0x01C863|$07:$8853:A9 12    LDA #$12 ; "i"
0x01C865|$07:$8855:9D F0 60 STA $60F0,X ; replace the second-last letter ("u") with "i"
0x01C868|$07:$8858:9D F1 60 STA $60F1,X ; replace the last letter ("s") with "i"
0x01C86B|$07:$885B:38      SEC ; let calling code know to read a #$FA-terminated string from $60F1 instead of a single byte from A
0x01C86C|$07:$885C:60      RTS

and I just wanted to run my Man O' War to Men O' War code by you

Code: [Select]
; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
Close, but as it's written that code will still end pluralizing Man O' War to Man O' War, since it takes the ninth-last letter and writes it to the ninth-last letter; try doing "LDA #$0E" before the "STA $60E9,X".

Once you've finished labelling the control flow targets, try assembling and inserting it! Your code is definitely shorter than the original (and way shorter than mine for Latin!), so this probably won't be an issue, but just in case, keep in mind that all those branches use a 1-byte signed displacement from the end of the operand, which basically means if you need to move farther than -127/+128 bytes, you'll have to use a JMP instead. Assuming that works, it's time to test it out - here's hoping the game doesn't crash on you :D.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 26, 2019, 09:42:21 pm
There's nothing magic going on here; try setting an execute breakpoint at $8853, getting into a fight with some Magic Vampirii (or temporarily rename e.g. Slime to Slrus and get into a fight with some of those), and step through the code, watching what happens around $60F1 in the Hex Editor.

Does this make it any clearer?
Spoiler:
Code: [Select]
0x01C863|$07:$8853:A9 12    LDA #$12 ; "i"
0x01C865|$07:$8855:9D F0 60 STA $60F0,X ; replace the second-last letter ("u") with "i"
0x01C868|$07:$8858:9D F1 60 STA $60F1,X ; replace the last letter ("s") with "i"
0x01C86B|$07:$885B:38      SEC ; let calling code know to read a #$FA-terminated string from $60F1 instead of a single byte from A
0x01C86C|$07:$885C:60      RTS
Close, but as it's written that code will still end pluralizing Man O' War to Man O' War, since it takes the ninth-last letter and writes it to the ninth-last letter; try doing "LDA #$0E" before the "STA $60E9,X".

Once you've finished labelling the control flow targets, try assembling and inserting it! Your code is definitely shorter than the original (and way shorter than mine for Latin!), so this probably won't be an issue, but just in case, keep in mind that all those branches use a 1-byte signed displacement from the end of the operand, which basically means if you need to move farther than -127/+128 bytes, you'll have to use a JMP instead. Assuming that works, it's time to test it out - here's hoping the game doesn't crash on you :D.

First of all, really smacking my forehead at the obvious lack of a LDA command for some of these. I think I forgot to copy that line over for the Wiki but also, it's really easy for me to overlook something obvious.  :crazy:

Not sure what I did, but my first time seemed to work okay aside from a couple of mistakes, but now after checking my Jump/Branch values I seem to have made it worse. A lot of my test names tend to show one or two letters like "Three y appeared" instead of two "Iron Dragonfry appeared"

(https://i.imgur.com/7AxRlHp.png)

I was able to parse your example asm script above, filling in the byte values appropriately, but I did change stuff like done: and add_s: assuming those were meant to be comments. Did I mess somethign up there?

This is my current script as it's being inserted (also yeah I should have known base meant was a RAM address. Here I thought it was some kind of hardware value that needed to be changed to something signifying NES hardware instead of SNES hardware. Live and learn.

Code: [Select]
norom   ; stop Asar from trying to apply SNES memory mapping to this NES code
org $01C805   ; set the ROM file insertion point to 0x10010
base $87F5   ; set the starting RAM address to $8000

LDY $8F ; number of monsters in the current group
DEY
BEQ $882C ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$0F ; "f"
BEQ $8856 ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ $886C ; -y -> -ies
CMP #$12 ; "i"
BEQ $885B ; -i -> -ies
CMP #$1C ; "s"
BEQ $882E ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ $8863 ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ $8899 ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ $887A ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ $888C ; Man O' War -> Men O' War
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
; add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
; done:
SEC
RTS

; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
LDA $60F0,X ; read second-last letter of monster name
CMP #$0E ; "e"
BEQ $882C ; if -es, singular=plural
CMP #$0A ; "a"
BEQ $882C ; if -as, singular=plural
CMP #$19 ; "p"
BEQ $882C ; if -ps, singular=plural
LDA $60EF,X ; read third-last letter of monster name
CMP #$10 ; "g"
BEQ $884E   ; Shortcut here, if g_s (Magus) replace us with i
BNE $8846 ; If not either Cyclops or Atlas (or Magus), add -es

; append "es" to monster name
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
BNE $881F ; append "s" to monster name; note that this branch is always taken

; replace us with i
LDA #$12    ; "i"
STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
SEC
RTS

; -f pluralization handler: -f -> -ves (no need to have this, but if there's space, might keep it for possible hacks)
LDA #$1F ; "v"
JMP $885D ; replace "f" with "v" then append "es"


; -i pluralization handler: -i -> -ies (same as above)
LDA #$12 ; "i"
; unused control flow target (from $8867)
STA $60F1,X ; replace final letter with "i"
JMP $8846 ; append "es"

; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ $8846 ; if -sh, append "es"
BNE $881F ; else, append "s"

; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
LDA $60F0,X ; read second-last letter of monster name
CMP #$18 ; "o"
BEQ $881F ; if -oy, append "s"
CMP #$1B ; "r"
BEQ $882C       ; if -ry, no change
BNE $885D      ; otherwise replace y with -ies

; -a pluralization handler: needed for madusa -> madusae
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ $8883   ; if -sa add "e"
BNE $881F       ; if not -sa append "s"

;adding "e"
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
JMP $8825

; -r pluralization handler: needed for man o' war
LDA $60E9,X     ; read ninth from end letter of monster name
CMP #$0A       ; "a"
BNE $881F       ; if not "a" append "s"
LDA #$0E        ; "e"
STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?

; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
LDA $60F0,X ; read second-last letter of monster name
CMP #$0A ; "a"
BNE $881F ; if not -an, append "s"
LDA $60EF,X ; read third-last letter of monster name
CMP #$16 ; "m"
BEQ $88AB   ; -man -> -men
CMP #$30 ; "M"
BNE $881F ; if not -Man, append "s"
; control flow target (from $888B)
LDA #$0E ; "e"
STA $60F0,X ; replace second-last letter of monster name
; control flow target (from $88DF)
SEC
RTS