Romhacking.net

Romhacking => ROM Hacking Discussion => Topic started by: Chicken Knife on September 19, 2018, 08:25:57 am

Title: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on September 19, 2018, 08:25:57 am
So I've mostly finished my script revisions to Dragon Warrior 1 and I'm now looking at taking on the sequel. There isn't a dedicated table file posted, but when I plug in the table data from the first game, I'm able to read the text for spells, items and monster names in my hex editor. The curious thing is that the bulk of in game dialogue text does not show up in a readable form.

Glancing over information for later Dragon Warrior games, it looks like text compression exists for the dialogue in the 4th game. Am I most likely facing a compression issue in DW2?

If so, are there any good universal tools that would help me with the process of extracting, decoding, and reinserting the text?
Title: Re: Handling possible text compression
Post by: FCandChill on September 19, 2018, 05:43:03 pm
Chances are, this game uses DTE compression. It's very common in NES games. Can you find data that sort of ressembles dialogue like "S.e.h prince." ("Save the princess" compressed)?

Also, you might want to have a dedicated thread for your Dragon Quest questions ... just a suggestion.

September 19, 2018, 05:43:36 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
As for tools, check here https://www.romhacking.net/?page=utilities&category=14&platform=&game=&author=&os=&level=&perpage=20&title=&desc=&utilsearch=Go
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 19, 2018, 07:46:22 pm
I've used the ROM map available on this page to determine the block of data where the dialogue is stored. As I scan through that data, it does not appear as an abbreviated form of actual text like your example of S.e.h prince. Instead, all the data in that section appears as 80-90 percent non letter entries, with occaisional letter entries interspersed.

As far as your point on forum etiquette, it's a fair one. I know I've been throwing a lot of separate topics onto this board lately. I actually do have a dedicated Dragon Quest thread under personal projects. I figured that thread would attract primarily readers interested in my Dragon Quest project, but posting here would catch a wider audience of people willing to help with the numerous technical obstacles I face as a very new person to this process. Perhaps it would be better to have a unified thread here that isn't really about my project but serves to keep my large mess of questions in one place?
Title: Re: Handling possible text compression
Post by: FCandChill on September 19, 2018, 08:09:27 pm
I've used the ROM map available on this page to determine the block of data where the dialogue is stored.

You didn't link to anything in your post, but I assume you meant to link here: https://datacrystal.romhacking.net/wiki/Dragon_Warrior_II:ROM_map

The info you referenced is under the section name Dragon Quest not Warrior. In other words ... the page covers the Japanese version, not the English version. That may be your issue.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 19, 2018, 08:54:53 pm
Fair point, but I have used a hex editor to carefully scroll through all the data in the rom, several times actually. On each search I come across the full list of items, monsters and spell names. The only other text I come across is the prologue text added for the US version intro. The dialogue text is no where to be found.

When I go through DW1 or even DW3, I easily locate the large block of general dialogue text, along with the items, monster names and spells.
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 02:00:55 am
I second the idea of a dedicated thread for all this: no need to make lots of new threads. :)

As for the text compression question, if I can find the time I can have a look myself. I know for a fact that Dragon Warrior doesn't have compression but I can't remember about DW2. I'd be a bit surprised if it did because neither it nor the first game have an overwhelming amount of text.

Maybe later today I'll be able to spend five minutes checking the text to see how it's stored in the ROM. Compression does happen in text heavy games - my efforts translating Time Stranger from 1986 found a neat dictionary compression scheme - but again, I'd be a little surprised were that the case here.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 20, 2018, 08:58:54 am
Sounds great Psyklax. Once this issue gets figured out I'll change the title of this post to something generalized and put all my technical questions here going forward.

Going back to the issue at hand, I also found it strange that the game would use compression--especially since there is a ton of empty space in the US rom. I initially thought this might be an issue of the game having the alphabet under multiple table entries. To test that, I went into FCEUX Nametable Viewer and checked the hex code of the letters showing up in dialogue text. The hex code for the letters in dialogue was exactly the same as the code for the letters in monster, item and spell names--with all of the latter showing up in a hex editor when I load up table data. Would this indicate compression or could another factor be causing this? 

I appreciate you taking a look at the rom!
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 10:43:22 am
I went into FCEUX Nametable Viewer and checked the hex code of the letters showing up in dialogue text. The hex code for the letters in dialogue was exactly the same as the code for the letters in monster, item and spell names--with all of the latter showing up in a hex editor when I load up table data. Would this indicate compression or could another factor be causing this?

I don't think simply looking at the nametable would indicate compression per se. The way I would do it is the more advanced method of debugging the game to see how the text gets from the ROM to the screen, and compression will become apparent then. A simpler way is the old fashioned relative search technique locating where the text is, and discovering the compression then.

I'll have a look later, anyway. I've already hacked this game to double experience and gold, so that's why I'm familiar with it.
Title: Re: Handling possible text compression
Post by: KingMike on September 20, 2018, 12:50:32 pm
I know Dragon Warrior II and III used ROM expansion so I wouldn't expect something beyond dictionary, because what's the point once they're expanding the ROM.

Yes, in DW4 they used Huffman which is tricky but that's because they couldn't really expand further. SOROM or whatever it was (the 512KB MMC1 mapper) was already about the limit.
Title: Re: Handling possible text compression
Post by: Psyklax on September 20, 2018, 06:00:12 pm
Well, you were right: DW2 uses a rather tricky compression method which I don't fully understand yet. I'm still looking at it, and it's not like what I've seen before. I'd say it's a bit like dictionary compression, except not... I'll continue until I've figured it out, but it's weird.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 20, 2018, 07:40:20 pm
Thank you so much for diving into this Psyklax. Why did those knuckleheads at Nintendo of America feel the need to give us this headache with all that free space at their disposal.  :banghead:
Title: Re: Handling possible text compression
Post by: Alchemic on September 21, 2018, 01:36:17 am
I did some digging around in DW2 a few years back. My notes are kind of a mess, but to summarize:
Code: [Select]
TBL for 0xB49B:
00-09 = 0-9
0A-23 = a-z
24-3D = A-Z
3E-57 = a-z again
58=A
59=(space)
5A=[SUN]
5B=[STAR]
5C=[MOON]
5D=[DROP]
5E=[HEART]
5F=(space)
60=(space)
61='
62="
63=->
64="
65='
66='
67='
68=.'
69=,
6A=-
6B=.
6C=&
6D=(space)
6E=?
6F=!
70=;

Text-dumping code in Python (https://github.com/Osteoclave/game-tools/blob/master/nes/dragonwarrior2_textdump.py)
Title: Re: Handling possible text compression
Post by: Psyklax on September 21, 2018, 03:56:14 am
Wow, Alchemic, that confirms what I saw: that it's pretty messed up. :D I got as far as noticing that each individual word is assembled letter-by-letter in SRAM before being passed off to the PPU, but it would've taken me time to get to the level of detail you've acquired. That has to be the most convoluted text handling scheme I've ever seen in a game.

I had a look so I can see what you're talking about there. It's still messing with my brain, so I think I'll leave it for now. :)
Title: Re: Handling possible text compression
Post by: abw on September 21, 2018, 08:57:00 am
Maybe later today I'll be able to spend five minutes checking the text to see how it's stored in the ROM.
If you actually had cracked that code in 5 minutes, you would have had my vote for the annual "God of ROMHacking" award :P.

I did some digging around in DW2 a few years back. My notes are kind of a mess, but to summarize:
[...]
Text-dumping code in Python (https://github.com/Osteoclave/game-tools/blob/master/nes/dragonwarrior2_textdump.py)
Nice work Alchemic! I'm sure somebody's next question is going to be: did you also happen to write an insertion script?

That has to be the most convoluted text handling scheme I've ever seen in a game.
The basic input -> output encoding is pretty simple, but I definitely agree that DW2's text engine could have been implemented in a much less complicated way. It took me a couple of days to work through it enough to get a decent script dump, but I can confirm that Alchemic's notes are indeed accurate.

I also found it strange that the game would use compression--especially since there is a ton of empty space in the US rom.
Why did those knuckleheads at Nintendo of America feel the need to give us this headache with all that free space at their disposal.  :banghead:
It's not always a question of how much free space in total, but where that free space is located. DW2's script is large enough that even with the compression scheme they added, it still doesn't fit inside a single PRG bank, which is why the script jumps from 0x17FE6 to 0xB7C2. Maybe they were trying to keep almost all of the text-related code and data confined to 2 PRG banks?
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 24, 2018, 07:04:44 am
@abw got it right. I'm definitely asking for someone to help me write a script for this. Wouldn't I need two scripts: one to uncompress the text into readable / editable form and another to recompress for reinsertion? I'm willing to keep the maximum length of each line the same if it means not having to worry about readjusting all the pointers.

In the mean time, I am keeping myself busy figuring out the best way to edit the monster, spell and item lists but I would be so very grateful to have a means of moving forward with the script one day soon.

Title: Re: Handling possible text compression
Post by: abw on September 24, 2018, 09:57:45 pm
@abw got it right. I'm definitely asking for someone to help me write a script for this. Wouldn't I need two scripts: one to uncompress the text into readable / editable form and another to recompress for reinsertion? I'm willing to keep the maximum length of each line the same if it means not having to worry about readjusting all the pointers.
If all else fails, abcde (http://www.romhacking.net/utilities/1392/) can definitely handle both extracting and inserting text like this along with updating the pointers, and you'll have all the original space at your disposal (there's no free space conveniently available after the original script ends, since the very next byte is code, so if you need more space than the original, it'll take a bit more work). As an added bonus, thanks to the original encoding being somewhat inefficient, abcde can actually re-insert the original script using $79 fewer bytes.
Title: Re: Handling possible text compression
Post by: Chicken Knife on September 25, 2018, 08:42:31 am
@abw

Am I to understand that your software can already handle unwinding the existing compression routine in DW2 without the need for a special script being written? Also, if I am to use the more efficient form of compression available through your software, it will automatically rework the code of the game to be able to read the new compression routine?

Something tells me it will be nowhere near this easy.  :laugh:
Title: Re: Handling possible text compression
Post by: abw on September 25, 2018, 06:10:59 pm
Am I to understand that your software can already handle unwinding the existing compression routine in DW2 without the need for a special script being written?
[...]
Something tells me it will be nowhere near this easy.  :laugh:
I'm not going to make any promises about how easy using abcde will be for anybody else, but... basically, yeah. Like I said earlier, the text encoding itself is actually pretty simple, it's just that DW2's code for parsing that encoding is fairly unpleasant. Give abcde the right table file, extract command file, and ROM, and watch it go. Stick a couple of extra lines in the script dump file it generates to turn it into a valid insert command file, and then run abcde again with the same table file, the insert command file (edited to have your new text, if so desired), and (a copy of) the ROM. That said, unless you start making ASM changes, you're still constrained by the total available space and text engine and stuff, so it's not a magic bullet.

Also, if I am to use the more efficient form of compression available through your software, it will automatically rework the code of the game to be able to read the new compression routine?
Haha, don't I wish it was that good :P. abcde doesn't make any changes to the game's code, it just uses a better insertion algorithm. The original data compression was sub-optimal; as one example, the game stores the string "'Welcome " as ['W][e][l][come][ ], with those 5 pieces taking up 10 + 5 + 5 + 10 + 5 = 35 bits. On the other hand, abcde is smarter and inserts the same string as ['][Welcome ] in 5 + 10 = 15 bits, for a savings of 20 bits. Little improvements like that that save 5 bits here, 10 bits there over the course of the entire script is how we end up with 121 bytes of reclaimed space.
Title: Re: General Hacking Questions
Post by: Chicken Knife on November 29, 2018, 08:09:01 am
I thought I'd bring this thread back to life as I'm back into my projects consistently and have a few things I need help with.

I'm nearly done with my Dragon Warrior 1 hack and my only issue left is that I want to fix some compromises I had to make porting over NPC sprite graphics from the famicom version due to left/right mirroring instructions in the DW1 for certain sprite tiles.

Does anyone have any good reading material to recommend that will present in a way that does a good job explaining itself to a non coder that will help me understand "sprite pointers" (if I'm using the right term) and will help me understand how to either disable the LR mirroring instructions I’m struggling with or simply redirect the game to alternative tiles for that sprite? If I can "repoint" them effectively like I do with text, I will have a ton of extra space since I turned all the 4 direction sprites into 1 direction sprites.

There are also other topics where I would like to do some reading and am having a hard time because I either can't find the resources or the resources I find are too technical:

I encountered a multiplier formula in Dragon Warrior 3 where the game takes the base Japanese gold and experience numbers saved for monsters saved in the IS game but applies a .25% boost, rounded up, to those values for what you actually earn in battle. I want to locate and disable that multiplier, restoring the straight Japanese values in gameplay. I assume a debugger would help me here. If so, any good reading material (or better yet videos) you guys would recommend for effectively using the FCEUX debugger for something like this?

Last, I've tried to play around with Atlas / Cartographer, ABCDE software in hopes of solving the DW2 script compression issue. The faq that comes with Atlas is too technical for me at this point and I haven't found much else out there. Any recommended reading?

Thanks guys!
Title: Re: General Hacking Questions
Post by: abw on November 29, 2018, 08:34:16 pm
Does anyone have any good reading material to recommend that will present in a way that does a good job explaining itself to a non coder that will help me understand "sprite pointers" (if I'm using the right term) and will help me understand how to either disable the LR mirroring instructions I’m struggling with or simply redirect the game to alternative tiles for that sprite?
Maybe I shouldn't comment since I don't usually do much graphics work, but when I do, I usually end up consulting Y0SHi's NES Documentation (http://www.romhacking.net/documents/120/), and although I'm pretty sure I originally had to read it a few times before it sunk in, it hasn't steered me wrong yet and it does have a couple of sections that talk about sprites.

Last, I've tried to play around with Atlas / Cartographer, ABCDE software in hopes of solving the DW2 script compression issue. The faq that comes with Atlas is too technical for me at this point and I haven't found much else out there. Any recommended reading?
Did you try abcde's readme or examples? Be honest - how bad are they really? I'm waaaay too deeply immersed in the source material to feel confident about my ability to pitch the documentation at a level that is useful without making other people barf, plus I implicitly assume that the end user is already familiar with Atlas/Cartographer :(.

I might have some recommended reading for you, though (based on abcde v0.0.2; provide your own "Dragon Warrior II (U) [!].nes") ;D:
Extraction:
Quote from: Cartographer.txt
#GAME NAME:      Dragon Warrior II (U) [!].nes

#BLOCK NAME:      Main Script, Part 1
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $B762
#POINTER TABLE STOP:   $B7BD
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#AUTO JUMP START:   $17FE7
#AUTO JUMP STOP:   $B7C2
#ATLAS PTRS:      Yes
#BASE POINTER:      $C010
#TABLE:         dw2_script.tbl
#COMMENTS:      Yes
#END BLOCK

#BLOCK NAME:      Main Script, Part 2
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $B7BE
#POINTER TABLE STOP:   $B7C1
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#ATLAS PTRS:      Yes
#BASE POINTER:      $10
#TABLE:         dw2_script.tbl
#COMMENTS:      Yes
#END BLOCK

Quote from: dw2_script.tbl
# NB: I didn't put much time into figuring out the control codes, so some of them are still unknown and some of them might be wrong. Caveat emptor!

/%00000=[end]\n\n
/%00001=.[end]\n\n
/%00010=?[’ ][FD][FD][end]\n\n
/%00011=[.’][end]\n\n
%00100=[FF]
%00101=y
%00110=c
%00111=o
%01000=d
%01001=e
%01010=f
%01011=g
%01100=h
%01101=i
%01110=j
%01111=
%10000=l
%10001=m
%10010=n
%10011=[line]\n
%10100=[.’]
%10101=[ ‘]
%10110=r
%10111=s
%11000=t
%11001=u
%11010=a
%11011=w
%11100=C0
%11101=C1
%11110=C2
%11111=C3

%1110000000=A
%1110000001=B
%1110000010=Ca
%1110000011=D
%1110000100=E
%1110000101=F
%1110000110=G
%1110000111=H
%1110001000=I
%1110001001=J
%1110001010=King
%1110001011=L
%1110001100=Moonbrooke
%1110001101=N
%1110001110=O
%1110001111=[item]
%1110010000=The
%1110010001=Rhone
%1110010010=S
%1110010011=;
%1110010100=U
%1110010101=”
%1110010110=Water Flying Cl
%1110010111=C
%1110011000=Y
%1110011001=Z
%1110011010=x
%1110011011=Village
%1110011100=z
%1110011101=[F9]
%1110011110=‟
%1110011111=K

%1110100000=v
%1110100001=q
%1110100010=[’ ][wait][line]\n
%1110100011=R
%1110100100=.
%1110100101=[FD][FD]
%1110100110=P
%1110100111=b
%1110101000=T
%1110101001=!
%1110101010=[sun]
%1110101011=[star]
%1110101100=[moon]
%1110101101=W
%1110101110=k
%1110101111=p
%1110110000=?
%1110110001=,
%1110110010=[monster]
%1110110011=....
%1110110100=:
%1110110101=[’ ]
%1110110110=-
%1110110111=[’ ]
%1110111000=[spell]
%1110111001=[letter]
%1110111010=[no voice]
%1110111011=[wait]
%1110111100=M
%1110111101=[name]
%1110111110=[number]
%1110111111=[FD]

%1111000000=Thou hast
%1111000001=hest
%1111000010=Midenhall
%1111000011=hou
%1111000100= of
%1111000101= is
%1111000110= thou has
%1111000111= and
%1111001000=to th
%1111001001= thee
%1111001010=ast
%1111001011= do
%1111001100=hat
%1111001101= shall
%1111001110= was
%1111001111=hou has
%1111010000=d the
%1111010001= has
%1111010010=gon
%1111010011=.[wait][line]\n
%1111010100= have
%1111010101=come to
%1111010110=ing
%1111010111= hast
%1111011000=ost thou
%1111011001=this
%1111011010= of the
%1111011011=Hargon
%1111011100=in the
%1111011101=thing
%1111011110=he
%1111011111= with

%1111100000=reasure
%1111100001=[ ‘]Hast
%1111100010=Erdrick
%1111100011=come
%1111100100=ere is
%1111100101=Welcome
%1111100110=rince
%1111100111= great
%1111101000=arr
%1111101001= for th
%1111101010=piece[(s)] of gold
%1111101011=[.’][wait][line]\n
%1111101100=But
%1111101101=here
%1111101110=can
%1111101111=ove
%1111110000=hee
%1111110001=not
%1111110010=for
%1111110011=one
%1111110100= any
%1111110101= to
%1111110110=descendant
%1111110111=Roge Fastfinger
%1111111000=all
%1111111001=thy
%1111111010=[ ‘]W
%1111111011=thank thee
%1111111100= it
%1111111101= tha
%1111111110= thou
%1111111111= the

Usage (replace \path\to\abcde\abcde.pl as appropriate):
Code: [Select]
perl \path\to\abcde\abcde.pl -m bin2text -cm abcde::Cartographer "Dragon Warrior II (U) [!].nes" Cartographer.txt Cartographer_out -s

Insertion:
Quote from: Atlas.txt
// Define, load, and activate a TABLE
#VAR(Table, TABLE)
#ADDTBL("dw2_script.tbl", Table)
#ACTIVETBL(Table)

// Jump to start of script
#JMP($14010)
#HDR($C010)
// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
#AUTOCMD($17FE7, #HDR($10))
#AUTOCMD($17FE7, #JMP($B7C2, $BE0F))

// the rest of Cartographer_out.txt goes here

Quote from: insert.bat
@copy /Y "Dragon Warrior II (U) [!].nes" "Dragon Warrior II (U) [new].nes" > nul

perl \path\to\abcde\abcde.pl -m text2bin -cm abcde::Atlas "Dragon Warrior II (U) [new].nes" Atlas.txt

pause
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on November 30, 2018, 02:54:28 am
@ abw

Wow! Talk about a silver platter  :beer:

I've done all the steps you've outlined but I'm getting this message after inputting: (from within abcde.pl's directory)
perl abcde.pl - bin2text -cm abcde::Cartographer "Dragon Warrior II - Edit.nes" Cartographer.txt Cartographer_out -s

Message after inputting:

error UTF-8 "\x92" does not map to Unicode at C:/Perl164/lib/Encode.pm line 228, <TABLE> line 3
when reading C:\Rom Editing\dw2_script.tbl line 3

I don't think me using the altered name of my hacked rom file would matter. Let me know if it does.
As far as Perl, I downloaded the software today in order to get this going. ActivePerl 5.26.1.2601 MS Win32-x64-404865 is the version.

PS

As far as Yoshi's NES doc, I think it's written far more for someone like you than someone like me. The density of it loses me in the same way as most of what I try to read on the NES Dev site. Truth be told, your abcde faq is much easier to follow, though I'm far from understanding all of it and it definitely does assume I understand Atlas / Cartographer. As far as the Yoshi faq, I'll have to keep coming back to this and whatever else I find until I figure out how to do the graphical work I need to.
Title: Re: General NES Hacking Questions
Post by: abw on November 30, 2018, 08:36:29 am
@ abw

Wow! Talk about a silver platter  :beer:
Heh, I had it handy from earlier in the thread. I used that stuff to make the King of Moonbrooke say "Oh noes, not the flying purple people eaters!" during the intro scene in order to test inserts :P.

error UTF-8 "\x92" does not map to Unicode at C:/Perl164/lib/Encode.pm line 228, <TABLE> line 3
when reading C:\Rom Editing\dw2_script.tbl line 3
Ah, sorry, I forgot to mention that you'll need to save the files (particularly the table file) encoded as UTF-8. Any modern text editor should give you that option when you save a file; even Notepad can save as UTF-8!

As far as Yoshi's NES doc, I think it's written far more for someone like you than someone like me. The density of it loses me in the same way as most of what I try to read on the NES Dev site.
Yeah, there's definitely a lot to take in, but knowing what's going on at the hardware level, what the different sections of PPU RAM control, and what all those writes to $20xx are doing can be pretty helpful when you're trying to track down some graphics logic.

Truth be told, your abcde faq is much easier to follow
Huzzah, mission partially accomplished!
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on November 30, 2018, 07:33:03 pm
@ abw

Worked like a charm.  :woot!:

I assume that a similar approach to how I edit text in hex editors would serve me well. I plan on working within the length of the existing strings, using blank entries at the end of lines of text if I happen to shorten in order to keep the length exactly the same. The thing that concerns me is the presence of so many control codes in DW2. As I go through the extraction document, I see the presence of plenty of those words in the script, but I don't see any indication that those words are produced by a control code in the document. I can keep Roge Fastfinger in mind easily enough, but with a ton of other common control code words like he, with, can, one, any etc. there is no way I'm going to be able to keep track of them all. Also, how do I know that the game consistently used control codes for those words instead of using them sporadically? It seems like it will be inevitable that I'll end up with a different data size and a considerable headache. Perhaps I could use the find command to highlight every instance of them in the text before I start editing in order to be wary of carefully maintaining string length. Any advice would be appreciated.

Also, going back again to Roge Fastfinger, I will be reverting his name to the Japanese Lagos. Is there any simple way to edit the control code words so I could swap out the name and use the same code? If not, I suppose I'd have to seriously abbreviate those lines to compensate.

Also, when reinsertion time arrives, does your system automatically note every control code word and insert them back into the game as the single byte control code instead of individual letter data? I'm curious.

Last question: I've heard on occasion that this kind of software automatically readjusts the text pointers and I could therefore enlarge some strings of text as long as the total amount of data is the same upon insertion. That seems too good to be true. Could it be?

Let me know your thoughts. I'm sure there is a lot I'm conceptually missing here.
Title: Re: General NES Hacking Questions
Post by: abw on December 01, 2018, 12:13:49 am
Worked like a charm.  :woot!:
Whoo-hoo!

I assume that a similar approach to how I edit text in hex editors would serve me well.
I plan on working within the length of the existing strings, using blank entries at the end of lines of text if I happen to shorten in order to keep the length exactly the same.
It seems like it will be inevitable that I'll end up with a different data size and a considerable headache.
*shudders* Nope, nope, and... wait for it... nope :P.
The fastest way is probably just to try it and see, but basically when you use a (decent) script insertion utility, you don't have to worry too much about things like the binary it inserts or updating pointers, since the utility takes care of all of that for you. Like I mentioned earlier, you're still constrained by the total available space and text engine and stuff like that, but as long as you don't try to insert impossible (combinations of) characters or too much text, you can do more-or-less whatever you want.

Line lengths will change; that's okay and you don't need to care about it.
Some text will get encoded as single characters, some as multiple characters; that's okay and you don't need to care about it.
String addresses will change and pointers to those strings will get updated; that's okay and you don't need to care about it.

Last question: I've heard on occasion that this kind of software automatically readjusts the text pointers and I could therefore enlarge some strings of text as long as the total amount of data is the same upon insertion. That seems too good to be true. Could it be?
Automatically recalculating pointer values is just one of the many benefits of using a script insertion utility over hex editing :).

Also, going back again to Roge Fastfinger, I will be reverting his name to the Japanese Lagos. Is there any simple way to edit the control code words so I could swap out the name and use the same code?
Yup. It's not quite as simple for DW2 as it would be for a game with a less ornery text engine, but it's still not too hard. The dictionary lives at ROM 0xB44B-0xB686, with each nybble (1 nybble = half a byte) of 0xB44B-0xB49A giving the length of the corresponding dictionary entry in 0xB49B-0xB686. So you could change "Roge Fastfinger" to "Lagos" at 0xB655, shift the rest of 0xB664-0xB686 up to just after the end of "Lagos", and then change the length of the "Roge Fastfinger" dictionary entry from 15 to 5 at 0xB496 (i.e. 0xAF -> 0xA5), leaving you with 10 bytes of unused space at 0xB67D-0xB686. Just make sure to update your table file to match whatever changes you make to the game's text encoding, or you're going to get a nasty surprise when you try extracting or inserting text!

Also, when reinsertion time arrives, does your system automatically note every control code word and insert them back into the game as the single byte control code instead of individual letter data? I'm curious.
Based on the table file(s) you provide, abcde is aware of all the possible ways of translating your text into the binary that the game needs, and it will convert your text into the shortest possible binary that represents your text. How? Magic :angel:.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 02, 2018, 04:10:31 pm
@abw

Ok, I worked furiously throughout the weekend on my script. Having just completed it, I ran the insertion routine, loaded up the game, and found the text to be totally mangled. Everyone is saying something different than what they should be saying. I don't think my data overran the alotted space. In general a direct translation of the famicom is shorter than the US version. I assume I would have gotten some kind of error message if it did.

The other possibility in my mind is that I was supposed to totally sort out the control codes prior reworking the script, which I did not do. Is that what you would suspect is the issue or do you have another idea?

I have a frightening suspicion that if control codes are the issue, I would have saved myself tremendous work prior to extraction and reworking the script.

Also, there were some instructions in the Atlas data that you quoted me where I wasn't sure if they were written to me or the program. For example:

// auto-commands for when DW2 does a mid-string bankswap and resets its read address:

PS I don't know if this helps, but all of the dialogue from everyone seems to be various battle commands

*EDIT

If it helps at all, this is the Atlas file where I tried removing what looked like your instructions from me and added my game script underneath. Take a look.

https://www.dropbox.com/s/t06zcf8qmmfhsno/Atlas.txt?dl=0
Title: Re: General NES Hacking Questions
Post by: abw on December 02, 2018, 10:52:18 pm
Ha, that's me skipping over important little details again :-[ (in my defence, they are documented). In my extract script, I set #COMMENTS: Yes, which makes every line of the script be output as a comment and thus ignored during insertion. So, since all the text is a comment, the only thing that actually changed would have been the pointers, and every pointer in the first block of pointers would point to the same first string starting at 0x14010, which happens to contain battle text. The second block of pointers would also point to the same RAM address ($8000), but with the wrong ROM bank visible in RAM, so I imagine any attempt to display one of those strings would end badly.

If you're only making a few changes to the script, you might prefer setting #COMMENTS: No instead and editing the text that way, or reformat the script dump using your text editor of choice (when translating, I like keeping the original text for reference and adding my new text just below); a simple regular expression search-and-replace operation or two should do the trick, or in the worst case you could manually delete the // before each line of script.

Also, there were some instructions in the Atlas data that you quoted me where I wasn't sure if they were written to me or the program. For example:

// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
Any line in the extract or insert command files that starts with // is treated as a comment, so the ones I added are just notes to explain what's going on. In this case, DW2 has some code that says "when the current read address reaches $BFD7, swap in a different bank and keep reading from $B7B2", so the #AUTOCMDs are there to handle that during insertion.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 02, 2018, 11:33:46 pm
@abw

As far as leaving out that bit, you are my patron saint of hacking. No grievance shall ever be held against you.  :laugh:

Ok. So if comments would have been set to No, then all the lines of text would have inserted as intended I assume. If I understand correctly, leaving comments set to Yes would be convenient because I could write the legitimate lines of text underneath them without the // and insert the data back in the rom effectively even with the old script appearing as comments.

So in my case, I took the output document and totally changed 98 percent of the text based on various resources to reflect a literal but fluid version of the Japanese text in contemporary English. I haven't slept much lately--and my wife may want to divorce me but that's neither here nor there. :D

So what would you suggest I do now? I often don't mind doing things the slow and painful way. Lifelong RPG gamer after all  :laugh: I'm thinking to go through the document and delete every instance of the characters // and leave whatever text comes after them. Would that do the job for reinsertion or do I have to add anything in their place?

There is no way I'm starting over with a new extraction. Somehow or another, this document has to get formatted correctly.



December 04, 2018, 02:07:08 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Ok. That was a question I could really answer myself through a little experimentation. I ran another extraction and just confirmed which // characters to remove.

Having done so, I reinserted and about 3/4 of my text was inserted correctly! I'll consider that a small victory haha. The other 1/4 text is either the games original text or bugged.

The error message I got in command prompt when I did the insertion was:

unable to tokenize; best attempt failed at input position 513 at C:\Rom Editing/abcde/Table/Table.pm Line 421 <COMMAND_FILE> line 1707. in text string starting at Atlas.txt line 1665.

I immediately thought this was an issue with my insert text. I went to that line (with word wrap turned off) and found there is nothing unusual at all about it. Line 1665 reads:

[ ‘]This is a sewing shop[.’][line]
Title: Re: General NES Hacking Questions
Post by: abw on December 04, 2018, 08:03:38 pm
As far as leaving out that bit, you are my patron saint of hacking. No grievance shall ever be held against you.  :laugh:
Heh, be careful what you promise >:D

I haven't slept much lately--and my wife may want to divorce me but that's neither here nor there. :D
I'm not going to admit to knowing what that feels like :D.


I took a closer look at this and noticed a couple of things:
- The dictionary does not include uppercase Q, V, or X as displayable characters, so you can't use those anywhere in your script without changing the dictionary. That unreadable mess of garbage just before the final error message abcde gave is supposed to show you how far along abcde got in the problem string before it couldn't go any farther, and as it happens 514 characters into the string starting at line 1665 is a Q. (yeah, the error messages suck)
- Some of the dictionary entries display different visually identical tiles, but I made them the same in my table file. In my translation of DW1, I originally made the mistake of assuming that just because the graphics were identical, it didn't matter which of the different tile IDs I used; this led to a moment of intense dismay when I realized that I had managed to accidentally disable the "speech" text printing sound effect!
- You did indeed make a lot of changes - replacing all the "thee"s and "thou"s and changing some proper nouns meant that a lot of the original dictionary entries went unused during insertion, which resulted in your new script not compressing small enough to fit in the available space. A few ad hoc tweaks to the dictionary was enough to sort that out and leave you with 218 bytes to spare; if you end up needing more room, updating the dictionary based on a full frequency analysis should give you much better compression results than me just eyeballing things.

Here's a copy of the updated insert script (https://drive.google.com/open?id=1rrX177cE49fE2R0CYk9CXhYgGJqmNgQZ) and table file (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) I used. I took the liberty of merging the original text into your new text as comments and fixing up a couple of typos that were breaking insertion; spell checking the new script is your responsibility :P. For the table file changes, you'll need to make the corresponding changes to the dictionary like I described a couple of posts ago (http://www.romhacking.net/forum/index.php?topic=27053.msg366909#msg366909).
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 05, 2018, 06:29:19 am
@abw, you didn't need to do all this work for me! I'm extremely grateful but I really don't want to be a burden here (Although I imagine watching me flop around like a fish out of water can be equally burdensome)

I think you've given me everything I need to do this insertion and further improvements to my script--except for one thing that I need clarification for.

In order to update the rom's dictionary with the new words and nybble lengths as you instructed above, I need you to clarify how I actually locate where to plug in the nybble that corresponds to each dictionary entry. Either I missed it or you didn't provide the detail as to how you knew 0x0B496 corresponded to the entry for Roge Fastfinger / Lagos. I'm not sure if this uses a counting system like the item/monster/spell lists. Hopefully not since that would give me a big headache as the entries here seem all mashed up together.
Title: Re: General NES Hacking Questions
Post by: abw on December 05, 2018, 07:35:57 pm
@abw, you didn't need to do all this work for me! I'm extremely grateful but I really don't want to be a burden here (Although I imagine watching me flop around like a fish out of water can be equally burdensome)
If it makes you feel any better, only a little bit of that work was purely for you ;). I'm slowly puttering away at my own translation whenever the mood strikes me, so a lot of the issues you're encountering now are issues I would no doubt encounter myself at a later date. Might as well deal with them now so that we can both benefit (and judging from the read count on this topic, I suspect it's more than just you and I that are interested in this).

I think you've given me everything I need to do this insertion and further improvements to my script--except for one thing that I need clarification for.
The lengths of the dictionary entries are stored in nybbles at 0xB44B-0xB49A and the values are stored in bytes at 0xB49B-0xB686; the table file I provided lists the dictionary entries in the same order they appear in ROM, and the table file you were using to edit the item/monster/spell lists will also let you see the dictionary values correctly. An example from partway through the dictionary might be helpful to see what's going on:
Code: [Select]
...
0x00B45B|$02:$B44B:11                            ; length of "A", length of "B"
0x00B45C|$02:$B44C:21                            ; length of "Ca", length of "D"
0x00B45D|$02:$B44D:11                            ; length of "E", length of "F"
0x00B45E|$02:$B44E:11                            ; length of "G", length of "H"
0x00B45F|$02:$B44F:11                            ; length of "I", length of "J"
0x00B460|$02:$B450:41                            ; length of "King", length of "L"
0x00B461|$02:$B451:A1                            ; length of "Moonbrooke", length of "N"
...
0x00B4C1|$02:$B4B1:24                            ; A
0x00B4C2|$02:$B4B2:25                            ; B
0x00B4C3|$02:$B4B3:26 0A                         ; Ca
0x00B4C5|$02:$B4B5:27                            ; D
0x00B4C6|$02:$B4B6:28                            ; E
0x00B4C7|$02:$B4B7:29                            ; F
0x00B4C8|$02:$B4B8:2A                            ; G
0x00B4C9|$02:$B4B9:2B                            ; H
0x00B4CA|$02:$B4BA:2C                            ; I
0x00B4CB|$02:$B4BB:2D                            ; J
0x00B4CC|$02:$B4BC:2E 12 17 10                   ; King
0x00B4D0|$02:$B4C0:2F                            ; L
0x00B4D1|$02:$B4C1:30 18 18 17 0B 1B 18 18 14 0E ; Moonbrooke
0x00B4DB|$02:$B4CB:31                            ; N
...
So if you wanted to, say, change the dictionary entry for King to say Queen instead, you'd change the length stored at 0xB460 from 41 to 51, change King to Queen starting at 0xB4CC, and shift everything from 0xB4D0 down by one byte. The dictionary has a fixed length and is followed immediately by code though, so in a case like this where you added a byte, you would need to remove a byte from somewhere else in the dictionary (pick "Water Flying Cl", it's only used 4 times in the entire script!).
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 05, 2018, 08:04:54 pm
@abw knowing that you tend to respond around this time I was in a big rush to message that I actually figured this out myself (well not really but still), finding that it was indeed a similar counting mechanism and making a little chart for myself just to make sure I understood it. Exhibit A: the little chart I was working on. hahaha

B490   piece(s) of gold   ???
B491   But[ ]         here
B492   can[ ]         ove
B493   hee         not
B494   for         one
B495   [ ]any         [ ]to[ ]
B496   Descendant       Lagos      
B497    all         thy      
B498   ?W         thank thee      
B499   [ ]it         [ ]thou      
B49A   [  ]the.      4

December 05, 2018, 08:07:19 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Now that I get the mechanics of this, I'm going to optimize the dictionary entries to match up to my translation trends as much as possible, which should allow me to expand text a little more as I go through the revision process should I desire.

I'm interested in what you are planning translation wise--more Latin for DW2? Also, now that we are discussing your translation, I'm curious what your textual basis was: the original US English translation or the Japanese?
Title: Re: General NES Hacking Questions
Post by: abw on December 07, 2018, 11:08:02 pm
Exhibit A: the little chart I was working on. hahaha
Haha, I see that's a nice little chart you have there :thumbsup:.

I'm interested in what you are planning translation wise--more Latin for DW2? Also, now that we are discussing your translation, I'm curious what your textual basis was: the original US English translation or the Japanese?
Yup. It's a dirty job, but somebody's got to do it :P. I generally use the English version of whatever I'm translating as my base, but will consult other English versions or other languages if the need arises; this happens most often for made-up enemy/item names, but sometimes I come across a line or two of dialogue that I like to clarify because it's a little too vague or allows for multiple valid interpretations.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 08, 2018, 11:13:07 pm
Ok, I have a report that is pretty darned good. While I was initially having some problems achieving a successful insertion with the updated Atlas.txt and table file you provided, I was able to narrow down the issue to one small mistake I made as I heavily edited the dictionary where a character length was off by one. Once I fixed that, I was able to do an insert that is 99% bug free (at least according to the appx 30% of game text that I've tested so far.) Here is the bug that's jumping out at me. The King of Cannock/Samaltria has some bugged text prior to his normal text. See the two pics below. The text seems to be from Hargon's Temple after you use the Charm/Eye of Rubiss to dispel the illusion

https://www.dropbox.com/s/txzrjqisvzwumuc/Dragon%20Warrior%20II%20-%20Edit_002.png?dl=0
https://www.dropbox.com/s/syx9sowi42u3j03/Dragon%20Warrior%20II%20-%20Edit_003.png?dl=0

After those lines, again he returns to his normal text, allows you to save, and remarks about his son. Very peculiar and I hope you have an idea how to fix it because I have no idea. Let's see if I find anything else like this, but I'm relieved that everything else has pointed to the right text so far. Here are links to my relevant files if you want to look at them regarding this issue. FYI my Atlas.txt script is significantly revised since I last sent it.

https://www.dropbox.com/s/5qo09huap2a5y2t/dw2_script.tbl?dl=0
https://www.dropbox.com/s/yn3qj00b536dxqt/Atlas.txt?dl=0
https://www.dropbox.com/s/fns0v7z1o6i8nxm/dw2_delocalized_052.ips?dl=0

One other thing. Regarding your Latin Translation project, are you planning to include any kind of uncensoring element? If you were I would be most grateful to enlist your efforts on that front. As you can see if you look in my patch I restored the Japanese church crosses and sprite for the priest. Unfortunately however, the priest is assigned the color palette of Princess Moonbrooke instead of what should be the Prince Midenhall/Laurasia palette. And then there's the matter of replacing the ghosts with original coffins but let me know if you're even interested in this graphical stuff. I must say that your long deceased Roman audience probably wouldn't have approved much of censorship--though the Latin speaking clergy would have demanded it I'm sure.  :D

*** Update

After my newest round of updates, King Samaltria's Kee Kee text went away. :o This is after it stuck around through the last 3 updates or so. I don't get it! Seems more like a bug in the game itself than a bug from the insertion

Title: Re: General NES Hacking Questions
Post by: abw on December 09, 2018, 01:58:13 pm
The King of Cannock/Samaltria has some bugged text prior to his normal text. See the two pics below. The text seems to be from Hargon's Temple after you use the Charm/Eye of Rubiss to dispel the illusion
[...]
After my newest round of updates, King Samaltria's Kee Kee text went away. :o This is after it stuck around through the last 3 updates or so. I don't get it! Seems more like a bug in the game itself than a bug from the insertion
This turns out to be a little bit of column A, a little bit of column B.

On the insert side, the game has a hardcoded value that tells it which pointer within the pointer table at ROM 0xB762-0xB7C1 starts reading from ROM bank 2 instead of ROM bank 5, but the insert script didn't update that value, so the pointer at ROM 0xB7BE started reading text from the wrong ROM bank, which threw off the end token counting before the script reached RAM $BFD7 and resulted in the wrong string being displayed. This is easily fixable by adding a COUNTER variable and an AUTOCMD for writing the counter like in the Dragon Quest IV example that ships with abcde, except Dragon Warrior II's script is only spread across 2 ROM banks instead of 6, so it's easier for II than IV.
Code: [Select]
// add this near the top of the insert script:
#VAR(pointerNum, COUNTER) // create a COUNTER variable named pointerNum
#CREATECTR(pointerNum, 8, 0) // pointerNum is an 8-bit value initialized to 0
#AUTOCMD($17FE7, #WLB(pointerNum, $3FA90)) // update the code that controls which pointer starts the next bank

// and then after every #W16 line in the insert script, add:
#INC(pointerNum, 1)

On the game side, there is a bug where the game takes the pointer to the start of the desired group of 16 strings and reads the first 2 bytes from the pointer's target *before* it runs through the code for checking whether the current string address has reached RAM $BFD7, so if you're unlucky enough to have a pointer point to RAM $BFD6 or $BFD7 like happened with your linked insert script, the text for that pointer gets seriously screwed up. There's currently no way for abcde to handle that situation for you automatically, so unfortunately you're just going to have to keep an eye on the pointer values, and if you see $BFD6 or $BFD7 come up, you can knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6 respectively to compensate. It means you lose 1 or 2 bytes of script space, but that's probably easier to deal with than making the ASM changes required to fix the bug in the original game.

Both of these issues only show up depending on where the dividing line between ROM banks 5 and 2 falls in your new script compared to the original script, which explains why they would come and go as you continue updating and re-inserting your script.

One other thing. Regarding your Latin Translation project, are you planning to include any kind of uncensoring element? If you were I would be most grateful to enlist your efforts on that front. As you can see if you look in my patch I restored the Japanese church crosses and sprite for the priest. Unfortunately however, the priest is assigned the color palette of Princess Moonbrooke instead of what should be the Prince Midenhall/Laurasia palette. And then there's the matter of replacing the ghosts with original coffins but let me know if you're even interested in this graphical stuff. I must say that your long deceased Roman audience probably wouldn't have approved much of censorship--though the Latin speaking clergy would have demanded it I'm sure.  :D
I generally try to stick pretty close to the source material for various reasons (e.g. nostalgia, insufficienct fluency in Japanese, etc.), so as a rule of thumb, if my source was censored, so is my translation; if my source was not censored, neither is my translation. I can do graphics hacking if I have to, but while I appreciate the results of other people's graphics hacks, making my own just doesn't have the same appeal as translating does.

As for the ancient Romans, well, they were no strangers to censorship ("censor" itself is a Latin word), but I think a bigger problem there would be the lack of hardware and compatible power sources to play the translation on :P.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 09, 2018, 02:31:57 pm
Thank you for looking at this!

While inserting the lines you quoted in my Atlas.txt are easy enough, if I am to undertand correctly, even with the COUNTER variable and AUTOCMD added, there is still a possibility of seeing these kinds of issues pop up. This leads me to the second fix:

if you see $BFD6 or $BFD7 come up, you can knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6 respectively to compensate. It means you lose 1 or 2 bytes of script space, but that's probably easier to deal with than making the ASM changes required to fix the bug in the original game.

I don't exactly understand the above instructions. Could you explain to me a little more clearly what you mean by "knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6"? First, how do I see when $BFD6 or $BFD7 come up and then how do I actually knock them back to the other addresses.

And finally, my attempt to rally you to the cause of graphics work went exactly as expected. I can't be blamed for trying though!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on December 09, 2018, 03:24:59 pm
Hey guys, with me finishing DW1, it's time for me to throw my hat into the text editing ring as well. I've been following this thread and I should be all caught up as far as having all the stuff I need. Big thanks to you abw for your help :)

abw, I noticed your table says you haven't figured out those control codes, but there's one in particular that's bugging me. In the python script posted on the first script there's a standalone byte, F2, which is a special "s" byte. It's used in the "piece<s> of gold" dictionary entry. I would take a good guess that this is the same as the EF byte in DW1 that adds or omits an S based of the number value used in the string. I'd hope to use this to change the battle message "[name][’ ]s HP is reduced by [number]." into "[name] takes [number] point(s) of damage". Is there a way I can update the table to use this standalone byte?
Title: Re: General NES Hacking Questions
Post by: abw on December 09, 2018, 04:44:32 pm
I don't exactly understand the above instructions. Could you explain to me a little more clearly what you mean by "knock the #AUTOCMD($17FE7, *) addresses back to $17FE5 or $17FE6"? First, how do I see when $BFD6 or $BFD7 come up and then how do I actually knock them back to the other addresses.
Oh, sorry, I meant you'll need to check the values of the pointers in the pointer table at ROM 0xB762-0xB7C1 (based on your current script size, 0xB7BE is the most likely culprit) following an insert to see if any of the values are $BFD6 or $BFD7 (a.k.a. D6 BF or D7 BF in little endian), and if so, then adjust the AUTOCMD commands in the insert script to fire when the insert point reaches $17FE5 or $17FE6 instead of $17FE7.

Basically, we need to make sure the game never tries to read script data at or after RAM $BFD7 / ROM 0x17FE7, and since the game unconditionally reads the first two bytes of the string based on the pointer value, pointer values of $BFD6 or $BFD7 are bad.

And finally, my attempt to rally you to the cause of graphics work went exactly as expected. I can't be blamed for trying though!
Based on your work from DW1, I think you'll be able to manage the graphics just fine without me :P.

Hey guys, with me finishing DW1, it's time for me to throw my hat into the text editing ring as well. I've been following this thread and I should be all caught up as far as having all the stuff I need. Big thanks to you abw for your help :)
Welcome to the party! (There's music for that in this game!) I figured you'd show up here sooner or later ;).

abw, I noticed your table says you haven't figured out those control codes, but there's one in particular that's bugging me.
I also assume the F2 code in DW2 works the same way as EF in DW1, so if you wanted to re-jig the dictionary to make that code available separately as opposed to being usable only as part of the "piece[(s)] of gold" entry, feel free. The process I described a couple of posts ago (https://www.romhacking.net/forum/index.php?topic=27053.msg367159#msg367159) should work just as well for this dictionary update as for "King" -> "Queen".

The codes I haven't bothered to check out are the F9, FD, and FF codes. Based on its usage in the script, F9 is probably some sort of item ID, though it'll take some research to figure out if and how it differs from F7. Some light experimentation suggested that FD might be useless, and FF looks like it might be some kind of conditional end token, but again, I haven't really looked at them to find out, so those are just guesses.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 09, 2018, 05:06:50 pm
Based on your work from DW1, I think you'll be able to manage the graphics just fine without me :P.

Hah. I'm glad my DW1 work *looks* impressive but truthfully all of that was my very first foray into hacking and more a result of elbow grease than actual skill. Most of it was simply spamming tiles from the famicom version all over the tile grid with a ton of duplication. The in game engine is still having the sprites point left, right, up, down etc but all of those directional tiles were simply replaced with the front facing famicom tiles so that they would display appropriately. The issues were when the US game would use mirrored images of tiles and that would put me in a spot where the famicom sprites would inevitably get screwed up. I would have to do little tricks to make the sprites more symmetrical like change the old wizard guy's head so his cap no longer leans to one side, or remove the guards spear because his body had to by symmetrical, all to work around the fact I couldn't disable the mirroring or point the code to different tiles.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 23, 2018, 04:19:50 pm
@abw

Since I'm pretty much done with my DW2 retranslation (perpetual rephrasings aside), I thought I would go back and give the first Dragon Warrior a bit of the old in out (or should I say "out in") with abcde. I figured I would let you and choppasmith have some time to bring your DW2 projects forward before I start assaulting you guys with requests for help on running extract and insert routines on DW3.

I got the table file that Choppasmith used, (byte format as opposed to the bit format one you sent me for DW2--I assume they both will work) and was pleasantly reminded to save it in UTF-8 per your readme. For the Cartographer.txt, I got the text pointer data from data crystal. I removed the auto jump instructions as I figured that would be DW2 specific. Gave it a shot and I got a ton of the same error message in command prompt: substr outside of string at C:\Rom Editing/abcde/Table/Table.pm line 234. It did spit out a very limited cartographer out file which I'll post a bit of immediately below. No in game text was captured in the output.

//POINTER #16 @ $8032 - STRING #16 @ $16FFE

#W16($8032)

// current address: $16FFE


//POINTER #17 @ $8034 - STRING #17 @ $1769B

#W16($8034)

// current address: $1769B


//POINTER #18 @ $8036 - STRING #18 @ $17A75

#W16($8036)

// current address: $17A75

This is what my Cartographer.text looks like below: (FYI I don't really need the comments for existing script turned on. I'm really only refining and expanding what I already wrote while I consult outside of game resources for the Japanese text.)

#GAME NAME:      Dragon Warrior I - Edit.nes

#BLOCK NAME:      Main Script
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $8012
#POINTER TABLE STOP:   $8037
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   16
#ATLAS PTRS:      Yes
#BASE POINTER:      $C010
#TABLE:         dw1_script.tbl
#COMMENTS:      No
#END BLOCK

I'm not sure if anything else needed to be changed. I figure pointer size would be the same. I'm not sure if DW1 also used 16 strings per pointer. Let me know if you think something is wrong here or if something needs to be tweaked with the table file for whatever reason. (attached below)

https://www.dropbox.com/s/h2uxum9jurl0jcb/dw1_script.tbl?dl=0
Title: Re: General NES Hacking Questions
Post by: abw on December 30, 2018, 04:12:13 pm
Try it again with "#BASE POINTER: $10" instead - the issue here is that a pointer value of e.g. $8028 + a base value of $C010 = $14038, but the entire ROM (including header) is only $14010 bytes, so we end up trying to read beyond the end of the file, which doesn't work so well. The other thing that might trip you up a bit is that the final pointer only has 10 strings instead of 16, so for a cleaner dump you'll want to split the main script in to two blocks:
Code: [Select]
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8012
#POINTER TABLE STOP: $8035
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 16
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

// only 10 strings for the final pointer
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8036
#POINTER TABLE STOP: $8037
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 10
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 30, 2018, 11:08:16 pm
Try it again with "#BASE POINTER: $10" instead - the issue here is that a pointer value of e.g. $8028 + a base value of $C010 = $14038, but the entire ROM (including header) is only $14010 bytes, so we end up trying to read beyond the end of the file, which doesn't work so well. The other thing that might trip you up a bit is that the final pointer only has 10 strings instead of 16, so for a cleaner dump you'll want to split the main script in to two blocks:
Code: [Select]
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8012
#POINTER TABLE STOP: $8035
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 16
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

// only 10 strings for the final pointer
#BLOCK NAME: Main Script
#TYPE: NORMAL
#METHOD: POINTER_RELATIVE
#POINTER ENDIAN: LITTLE
#POINTER TABLE START: $8036
#POINTER TABLE STOP: $8037
#POINTER SIZE: $02
#POINTER SPACE: $00
#STRINGS PER POINTER: 10
#ATLAS PTRS: Yes
#BASE POINTER: $10
#TABLE: dw1_script.tbl
#COMMENTS: No
#END BLOCK

@abw, thank you as always for that and the explanation.

Unfortunately however the results were not successful. The output file contains a little bit of the script in a mashed together format with no delineation as I had with DW2. After the small section of script there is this massive block of information in byte format. Here's a copy if you want to take a glance. I'm scratching my head as to what would be causing this result.

https://www.dropbox.com/s/4tvfmtab1upy3vu/Cartographer_out.txt?dl=0

my command line instruction for output is basically the same as the one you recommended for DW2. Should any of that change in the case of this game?

perl abcde.pl -m bin2text -cm abcde::Cartographer "Dragon Warrior I - Edit.nes" Cartographer.txt Cartographer_out -s
Title: Re: General NES Hacking Questions
Post by: abw on December 31, 2018, 01:06:04 am
Ah, yes, that table file doesn't mark any of the table entries as end tokens, and with no end tokens there's nothing to say when to stop dumping, so each string keeps going until the end of the ROM :P. It also doesn't contain any newline markers, so even after fixing the end tokens you'd still end up with all 16 of each pointer's strings on one line, which didn't feel right to me. Here's the table file I've been using; feel free to adjust the \n linebreaks to match your desired format.
Code: [Select]
00=0
01=1
02=2
03=3
04=4
05=5
06=6
07=7
08=8
09=9
0A=a
0B=b
0C=c
0D=d
0E=e
0F=f
10=g
11=h
12=i
13=j
14=k
15=l
16=m
17=n
18=o
19=p
1A=q
1B=r
1C=s
1D=t
1E=u
1F=v
20=w
21=x
22=y
23=z
24=A
25=B
26=C
27=D
28=E
29=F
2A=G
2B=H
2C=I
2D=J
2E=K
2F=L
30=M
31=N
32=O
33=P
34=Q
35=R
36=S
37=T
38=U
39=V
3A=W
3B=X
3C=Y
3D=Z
3E=‟
3F=”
# 40 and 53 are visually identical, but 40 is used as a right single quotation mark while 53 is used as an apostrophe
40=’
41=*
42=[triangle-r]
43=[triangle-d]
44=:
45=[..]

47=.
48=,
49=-

4B=?
4C=!
4D=;
4E=)
4F=(
50=‘
# 51 is visually identical to 50, but does not trigger "speech" sound effects or auto-indentation
51=[‘; no sound, no indent]
52=[.’]
# 53 is the apostrophe version of 40
53='
54=[ ’]

5F=

F0=[ Point(s)]
F1=[(n )enemy name]
F3=[experience Point(s)]
F4=[enemy name]
F5=[number]
F6=[spell name]
F7=[item name]
F8=[HERO]
FB=[wait]
# FC is the end token used in the main script
/FC=[end]\n\n
FD=[line]\n
# FF is the end token used in the item/monster/spell lists
/FF=[/name]\n
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on December 31, 2018, 01:45:13 am
This worked perfectly. I'll have to study the format you used here for when I get through this revision and eventually move on to doing the same for DW3. I really want to develop the skill level to do this on my own, but the thing I'm most foggy on is what kind of process you use to figure out all these opcodes. Would  you do an initial extraction with just letters and the line breaks and then use context clues to flesh out the opcodes?
Title: Re: General NES Hacking Questions
Post by: abw on December 31, 2018, 01:17:34 pm
This worked perfectly.
Hurray!

I really want to develop the skill level to do this on my own, but the thing I'm most foggy on is what kind of process you use to figure out all these opcodes.
Every game is a world unto itself, so the only (as far as I know, anyway) guaranteed way to figure out how and where the game stores its text is by getting that text to display in a debugging emulator and tracing the code from the point where it writes the text to screen all the way backwards until you find the code that read the text from ROM. You'll need to be at least somewhat familiar with the console's hardware, the emulator's debugging features, and whatever variety of ASM the console uses, and it's usually a pretty time-consuming process, but it will give you highly detailed knowledge of how the game handles its text. Frequently, though, there are indeed easier ways.

Would  you do an initial extraction with just letters and the line breaks and then use context clues to flesh out the opcodes?
For games with simple text encodings like Dragon Warriors I and III, yeah, that's probably the fastest way to go about it. Making a table file based on how the individual character tiles are laid out in the PPU and then dumping a block of text (if you already know where the text and/or pointers are) or even loading the ROM and table file in a hex editor and running a quick visual scan through the ROM for blocks that look like text will go a long way towards getting a usable script dump. After that, it's usually not too hard to guess what any other codes appearing in the dump are supposed to do, especially if you're already familiar with what the text is supposed to say (e.g. due to playing the game obsessively as a child) or can find the string in-game. For ones you aren't too sure about, experimenting by removing them from the string or adding them elsewhere and observing how the game's behaviour changes will often shed more light on their purpose, and if all else fails, you can start diving in to the ASM.

For games with slightly trickier text encodings like the main scripts of Dragon Warriors II and IV, even if you know how the individual characters are laid out in the PPU, the game doesn't store the text that way in ROM and you're never going to be able to just guess how the text actually is stored within any reasonable time frame (unless somebody else has already done a bunch of work for you, thus allowing you to make better guesses), so the simple methods aren't going to work there and tracing through the game's code is probably the best way of finding out what in the world it's doing under the hood.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on January 01, 2019, 07:42:07 pm
Every game is a world unto itself, so the only (as far as I know, anyway) guaranteed way to figure out how and where the game stores its text is by getting that text to display in a debugging emulator and tracing the code from the point where it writes the text to screen all the way backwards until you find the code that read the text from ROM. You'll need to be at least somewhat familiar with the console's hardware, the emulator's debugging features, and whatever variety of ASM the console uses, and it's usually a pretty time-consuming process, but it will give you highly detailed knowledge of how the game handles its text. Frequently, though, there are indeed easier ways.

This brings me to a question. Over the last couple weeks I've been making a concerted effort to learn more about the NES architecture, study tutorials on the basics of ASM, and have spent more time playing with the FCEUX debugger. The 6502 NES ASM tutorials usually focus on teaching the basics of building an NES game from the ground up, which is good (if overwhelming) information but doesn't necessarily hone in on what I really want to be learning, which is what you highlighted--using the debugger to follow the data trails back to the rom code that dictates them. I've spent a lot of time looking for videos or tutorials around debugging and haven't come up with much. The only substantial information I've encountered is what was included in the FCEUX files themselves. Do you happen to recall anything in particular that helped you learn the process of tracing data back to its source? I think this is what would help me most not only with resolving some of these script editing issues but it would also help me track down how tile address instructions are stored in the code for the graphics work I need to do.
Title: Re: General NES Hacking Questions
Post by: abw on January 02, 2019, 10:39:17 am
Do you happen to recall anything in particular that helped you learn the process of tracing data back to its source?
Nothing specifically, alas. At a high level, the process is fairly simple: start with something you know (e.g. some text currently being displayed), figure out how that something got to be where it is / where it came from, and then keep repeating until "where it came from" is the ROM. The details can sometimes get messy, but FCEUX provides a pretty good collection of debugging tools for helping you do all that figuring out.

The Dragon Warrior code seems to be very fond of copying data back and forth between multiple staging areas in cartridge RAM before writing to the PPU, so even if you've mastered the use of FCEUX's Debugger, using its Trace Logger is probably still going to be a quicker way to find out useful information for these particular games. I wrote up a detailed description of tracking down DW1's item list and the pointer to it using FCUEX's Trace Logger in the Dragon Warrior 1 Spanish Translation (http://www.romhacking.net/forum/index.php?topic=26135.msg358879#msg358879) thread; you might find that helpful if you haven't read it already.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on January 06, 2019, 10:45:39 pm
@abw,

So I started the process of reinserting the script changes as I go. My main motivation for doing so was that I was hoping to be able to track how many characters worth of data remained available, as that indicator was very helpful while working on II.

After my first insert of my DW1 script, the command prompt curiously gave me no indication of remaining data afterwards. Other than that, the insertion seems to be working within the game. I believe I have some data available since I eliminated a ton of blank character spaces in the existing script. Am I missing something that turns that on this time around? This is the data I added to the top of my Atlas file.

// Define, load, and activate a TABLE
#VAR(Table, TABLE)
#ADDTBL("dw1_script.tbl", Table)
#ACTIVETBL(Table)

// Jump to start of script
#JMP($8038)
#HDR($10)

I didn't include the auto commands for mid-string swaps/counter variables, etc that were used in my Atlast file for DW2

I suppose I can pull the game up for myself in a hex editor to gauge how I'm doing with space but that command prompt indicator I always got before was very helpful.
Title: Re: General NES Hacking Questions
Post by: abw on January 07, 2019, 09:34:43 pm
Am I missing something that turns that on this time around?
Yup - without an upper bound on the insert range, there's no good way to calculate how much available space remains. abcde will just keep on inserting data until either it runs out of data to insert or your computer runs out of disk space (though I haven't tried that last one myself) :P. Try changing the jump command to e.g. "#JMP($8038, $BCBF)" and you should get some better feedback.

On a side note, I just submitted an update for abcde. Nothing major, but there are a couple of goodies that will be helpful if you end up doing some menu work on these games.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on February 06, 2019, 07:47:17 pm
So after a lot of editing (and I still wouldn't say it's 100% done) I managed to edit my script dump with the mobile script. It's 80K, about 50% bigger than the original.

I had to edit it a few times to fix some errors, thankfully abcde makes it easy to find what the troubled line is.

I then get this:

(https://i.imgur.com/DbrUmm6.png)

So far so good?


But then I get some scrambled text:

(https://i.imgur.com/DrJGHVe.png)

Looks like I need to double check my dictionary values. I hope that's my only problem.

EDIT: Yep dictionary and table needed a tiny tweak

(https://i.imgur.com/GxaFTL2.png)

Though, I think there's something funny with the special table provided in this thread. The combined .' for dialog works fine but the standalone closing quote/apostrophe just brings up the openign single quote instead

(https://i.imgur.com/9vw9uVS.png)

Also, getting some scrambled text. I wonder if this is from being over the limit. There's no way to immediately tell as said on the last page.

(https://i.imgur.com/8yK2JXO.png)

EDIT 2: Yeah the Midenhall castle NPC dialog is near the end of the first bank. If I'm reading it right, would using the script on the previous page be a way of transferring dialog/pointers to the second bank?
Title: Re: General NES Hacking Questions
Post by: abw on February 06, 2019, 10:28:36 pm
So after a lot of editing (and I still wouldn't say it's 100% done) I managed to edit my script dump with the mobile script. It's 80K, about 50% bigger than the original.

I had to edit it a few times to fix some errors, thankfully abcde makes it easy to find what the troubled line is.
You might want to grab the latest version (0.0.3) of abcde - I made some improvements like letting you know exactly how far over your limit the script goes. For your massive script, DW2 comes with 3 whole empty banks, so at least you shouldn't have to expand the ROM this time. If you haven't come across it yet, the code for switching between script banks starts at 0x3FE15 / $0F:$FE05, so that seems like a good place to start rewriting ASM.

Though, I think there's something funny with the special table provided in this thread. The combined .' for dialog works fine but the standalone closing quote/apostrophe just brings up the openign single quote instead
Yeah, like I said, I hadn't put much effort into the table file at that point, and I've noticed a couple of mistakes in it. I've uploaded my current table file here (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) if you're interested, but if you've been modifying the dictionary, you'll also want to ensure that your dictionary does in fact contain the correct value.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on February 13, 2019, 02:42:50 pm
You might want to grab the latest version (0.0.3) of abcde - I made some improvements like letting you know exactly how far over your limit the script goes. For your massive script, DW2 comes with 3 whole empty banks, so at least you shouldn't have to expand the ROM this time. If you haven't come across it yet, the code for switching between script banks starts at 0x3FE15 / $0F:$FE05, so that seems like a good place to start rewriting ASM.
Yeah, like I said, I hadn't put much effort into the table file at that point, and I've noticed a couple of mistakes in it. I've uploaded my current table file here (https://drive.google.com/open?id=1MnB--MNEd4BG_IOxt4qZcwq3y6kigmMr) if you're interested, but if you've been modifying the dictionary, you'll also want to ensure that your dictionary does in fact contain the correct value.

Thanks for that, I've updated my table. I tried using the newest version of your abcde, I now get: "#JMP bounded by at $BE0F has -15738 ($-3D7A) space left" So I take it this means my script overwrites 15,738 bytes over the limit of the bank? Would cutting down my script temporarily be a good way to indicate where would be a good stopping point for the first bank? (I know I can't just go into the ROM and see where it overwrites data) BTW, maybe I'm missing something (and I'll be honest, I'm earnest in learning but a lot of this stuff still makes me go @_@) Why is it in the ROM Map, The Main Script Part 1 starts at B7C2 and Part 2 starts at 14010 but in the atlas script they're the other way around?
Title: Re: General NES Hacking Questions
Post by: abw on February 13, 2019, 06:38:46 pm
Thanks for that, I've updated my table. I tried using the newest version of your abcde, I now get: "#JMP bounded by at $BE0F has -15738 ($-3D7A) space left" So I take it this means my script overwrites 15,738 bytes over the limit of the bank? Would cutting down my script temporarily be a good way to indicate where would be a good stopping point for the first bank? (I know I can't just go into the ROM and see where it overwrites data)
If you want to get technical about it, the script space is already split between 2 banks, so that means you're 15,738 bytes over the total script space limit rather than the bank limit, but basically, yes. For finding out how much of your script fit into the first bank, you can check 0x3FA90 to see which pointer starts the next bank; if your new script is twice as big as the original, you'll probably have something like $17 there instead of $2E. Temporarily cutting down your script is probably the fastest way to find out how much fits into the original script space, though since it looks like you're going to need 3 banks anyway, you may wish to consider moving the script space from 0xB7C2-0xBE0F to one of the unused banks; some of DW2's bankswap routines are limited to swapping in one of the first 8 banks, so having lots of free space in bank 3 might come in handy if you plan on making other changes too.

BTW, maybe I'm missing something (and I'll be honest, I'm earnest in learning but a lot of this stuff still makes me go @_@) Why is it in the ROM Map, The Main Script Part 1 starts at B7C2 and Part 2 starts at 14010 but in the atlas script they're the other way around?
Oh, sorry, that was my fault - I was in "sort things by ROM address" mode :P. I've corrected the wiki now.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 01, 2019, 02:50:31 pm
Sorry for the late reply, just been finding no time to spare lately, but I'm still determined to learn and get this figured out.

So I found that I can fit up to "Pointer 34" in the original space (with 818 bytes left over which is plenty for possible edits and additions) leaving the text from Pointer 35-45 and the last little bit of "Script Part 2" left.

I also inserted just the script AFTER Pointer 34 into the original space and got 1401 bytes left. So by doing a little bit of math subtracting those numbers from 17,955 (the combined amount of bytes for script space according to the ROM Map) means the first part of my script that fits is 17,137 bytes and the remaining is 16,554 meaning my whole script is 33,691 bytes. So yeah despite different text file sizes, the actual script data is about double.

So I'm already a little confused here...

Firstly


For finding out how much of your script fit into the first bank, you can check 0x3FA90 to see which pointer starts the next bank; if your new script is twice as big as the original, you'll probably have something like $17 there instead of $2E.

So what am I looking at here (at 3FA90) exactly? It certainly says 2E. What does that signify?

And secondly, more a noobish question, but am I understanding correctly that the Wiki stating the game has 16 PRG-ROM Pages that are 16KB each referring to the banks? If I do switch banks for dialog, and yes I can see there are plenty of blank patches of data, is there an easy way to determine what bank is where aside from trying to divide the bytes of the rom by 16? I feel like I'm missing something glaringly obvious.

As far as doing anything else. The only thing I might do is what you and Chicken Knife talked about in moving the Monster Names. I had shortened the names just enough to fit, but having that extra little bit of space for having full names would be nice. Script is the priority for me though!

Anyway, once again, I appreciate the help and feel free to tell me just enough so I can figure it out while learning. Certainly don't want to come off as the type who just wants stuff done for them!
Title: Re: General NES Hacking Questions
Post by: abw on March 01, 2019, 06:52:28 pm
So what am I looking at here (at 3FA90) exactly? It certainly says 2E. What does that signify?
What you've got there is the key byte inside the routine for deciding which ROM bank needs to be swapped in to get the string the game wants to display:
Code: [Select]
; determine which ROM bank to load based on string index
; IN: A = low byte of string index, X = high byte of string index << 5 (don't ask me why), Y and C irrelevant
; OUT: no change to A or X, Y = index of ROM bank to swap in to read the string from, C not used, but does indicate whether we ended up choosing bank 5 (clear) or 2 (set)
; control flow target (from $FA68)
0x03FA7E|$0F:$FA6E:A0 05    LDY #$05 ; default to bank 5
0x03FA80|$0F:$FA70:48      PHA ; save the original string index in A
0x03FA81|$0F:$FA71:29 F0    AND #$F0 ; useless op
0x03FA83|$0F:$FA73:4A      LSR
0x03FA84|$0F:$FA74:4A      LSR
0x03FA85|$0F:$FA75:4A      LSR
0x03FA86|$0F:$FA76:4A      LSR
0x03FA87|$0F:$FA77:85 10    STA $10 ; 16 strings per pointer, so low byte of string index >> 4 = low nybble of pointer index
0x03FA89|$0F:$FA79:8A      TXA
0x03FA8A|$0F:$FA7A:29 E0    AND #$E0
0x03FA8C|$0F:$FA7C:4A      LSR ; A = high nybble of pointer index
0x03FA8D|$0F:$FA7D:05 10    ORA $10 ; glue the high and low nybbles of the pointer index together into a single byte
0x03FA8F|$0F:$FA7F:C9 2E    CMP #$2E
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead
; control flow target (from $FA81)
0x03FA95|$0F:$FA85:8C C6 60 STY $60C6
0x03FA98|$0F:$FA88:68      PLA ; restore the original string index in A
; control flow target (from $D12B)
; external bank control flow target (from $06:$9542, $06:$9550)
0x03FA99|$0F:$FA89:60      RTS
If you added the #INC commands I mentioned in an earlier post (http://www.romhacking.net/forum/index.php?topic=27053.msg367369#msg367369), that #$2E should have been updated to reflect the new dividing line between bank 5 and 2.

The good news is that it looks like you are pretty close to being able to fit your script into just 2 banks, and if you can manage that, then all you need to do is insert the second half of your script into one of the unused banks and update 0x3FA94 to the new bank number. Do you think you can update the dictionary to get an extra 3% compression?

And secondly, more a noobish question, but am I understanding correctly that the Wiki stating the game has 16 PRG-ROM Pages that are 16KB each referring to the banks?
Yup!

is there an easy way to determine what bank is where aside from trying to divide the bytes of the rom by 16? I feel like I'm missing something glaringly obvious.
Math is the way (especially when you're flipping between games with different bank sizes), but this shouldn't be something you need a calculator for. 16 KB = $4000, so (ignoring the $10 byte iNES header), $0000 - $3FFF is bank 0, $4000 - $7FFF is bank 1, $8000 - $BFFF is bank 2, $C000 - $FFFF is bank 3, and so on. So if you have an address like 0x3FA90, you can look at the 3F part and say something to yourself like "3 x 4 = 12, F is in bank 3, and 12 + 3 = 15" to know that 0x3FA90 is in ROM bank 15.

Anyway, once again, I appreciate the help and feel free to tell me just enough so I can figure it out while learning. Certainly don't want to come off as the type who just wants stuff done for them!
Heh, the first game was free, but you're going to have to work for the second one :D.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 02, 2019, 12:23:48 pm
Heh, the first game was free, but you're going to have to work for the second one :D.

Sounds fair :) I'll get this slowly but surely.

Okay, so, in updating my dictionary, I had to update the table because I was still unable to use apostrophes and single closing quotes. Part of my new dictionary table looks like this (taken from your latest table)

Code: [Select]
%1110110101='
%1110110110=-
%1110110111=’

I do the insert.bat and it stops at a line at Pointer 26

Code: [Select]
C:\Users\mog11\Downloads\abcde_v0_0_3>perl abcde.pl -m text2bin -cm abcde::Atlas "DRAGONW2.nes" Atlas.txt
unable to tokenize; best attempt failed at input position 160 at ^ indicator in
]'s coffin.[end-FC]‘'Tis more than a ma
]'s coffin.[end-FC]ΓÇÿ^
(does your table file contain a ""?) in text string starting at Atlas.txt line 1127!

And here's the line as it appears in my atlas.txt

Code: [Select]
puts it back in [name]'s coffin.[end-FC]

‘'Tis more than a man could ask to know such elation at so advanced an age!’[wait][line]

Really baffling what happened here. It's not like this is the first apostrophe in the text.

Also, while I updated the dictionary to save a few bytes (adding a space for "gold", "Goddess", and "key") I found that I missed a bunch of post-final boss NPC dialog that needed adding, so droping it 3% might be out unfortunately.
Title: Re: General NES Hacking Questions
Post by: abw on March 02, 2019, 06:13:33 pm
I do the insert.bat and it stops at a line at Pointer 26

Code: [Select]
C:\Users\mog11\Downloads\abcde_v0_0_3>perl abcde.pl -m text2bin -cm abcde::Atlas "DRAGONW2.nes" Atlas.txt
unable to tokenize; best attempt failed at input position 160 at ^ indicator in
]'s coffin.[end-FC]‘'Tis more than a ma
]'s coffin.[end-FC]ΓÇÿ^
(does your table file contain a ""?) in text string starting at Atlas.txt line 1127!

And here's the line as it appears in my atlas.txt

Code: [Select]
puts it back in [name]'s coffin.[end-FC]

‘'Tis more than a man could ask to know such elation at so advanced an age!’[wait][line]

Really baffling what happened here. It's not like this is the first apostrophe in the text.
Hmm, that does look odd. Based on that output, it looks like your system is defaulting to code page 437 (where "ΓÇÿ" = E2 80 98 = the UTF-8 encoding of U+2018 = "‘"), which means "┬ì" = C2 8D = the UTF-8 encoding of U+008D = Reverse Line Feed, but I have no idea how that managed to get in there. So, it looks like you've got an invisible character sitting between the "‘" and the "'" for some reason (you should be able to verify that hypothesis by viewing your insert script with a hex editor); try deleting that section and re-type the "‘'".

Also, here's a tip from abcde's readme: if you do "chcp 65001" in cmd before running abcde (e.g. at the start of a .bat file), you should get abcde's UTF-8 output showing up as UTF-8 characters instead of whatever code page your system defaults to... though in a case like this involving a non-printable control code, having gobbledygook is probably more helpful in tracking down the cause of the error!

Also, while I updated the dictionary to save a few bytes (adding a space for "gold", "Goddess", and "key") I found that I missed a bunch of post-final boss NPC dialog that needed adding, so droping it 3% might be out unfortunately.
Ah, if you have more text to add anyway, then yeah, trying to fit everything into 2 banks might be more work than just using 3 banks. So much for only needing a 1-byte hack :P.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 05, 2019, 11:14:58 am
Okay figured out the weird little script bug (you were right, there was some weird hex between the opening quote and the apostrophe). And with the recent dialog added 34,221 bytes.

Just to be sure. Your abcde script enters the script in both areas, right? I just want to make sure my math is right. The space remaining that I get when inserting part of it is the total amount between the two sections? Not just one? Because it seems like the best free space to use as a third bank is 1CC30 which is over 13K.

Old Script Part 1: 3FD7 Length (16, 343)
Old Script Part 2: 64E Length (1,614)
Total: 17,957

And to use a third bank I'm basically going to have to duplicate this part of the code:

Code: [Select]
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead

And change the pointer index and bank so that it's something like pointer index < #$22 (Pointer 34) then have something for Pointer 35-45 and then have a line for code 2E (46) equal and greater in value. Is that right?
Title: Re: General NES Hacking Questions
Post by: abw on March 05, 2019, 05:18:32 pm
Okay figured out the weird little script bug (you were right, there was some weird hex between the opening quote and the apostrophe).
Hurray!

Just to be sure. Your abcde script enters the script in both areas, right? I just want to make sure my math is right. The space remaining that I get when inserting part of it is the total amount between the two sections? Not just one?
Yup, yup, and yup.

Because it seems like the best free space to use as a third bank is 1CC30 which is over 13K.
I'd probably go with one (or two if you need another one) of the completely unused banks starting at 0x30010, 0x34010, or 0x38010 for a full 16k each, but if you can fit all your script into the free space starting around 0x1CC30, then great.

And to use a third bank I'm basically going to have to duplicate this part of the code:

Code: [Select]
0x03FA91|$0F:$FA81:90 02    BCC $FA85 ; if the pointer index < #$2E, keep the bank 5 default
0x03FA93|$0F:$FA83:A0 02    LDY #$02 ; if the pointer index >= #$2E, load bank 2 instead

And change the pointer index and bank so that it's something like pointer index < #$22 (Pointer 34) then have something for Pointer 35-45 and then have a line for code 2E (46) equal and greater in value. Is that right?
Basically, yeah. You'll just need to find a home for that code, but bank F is already pretty full. There should be enough unused space for your needs at 0x3FFA7 - 0x3FFC9 (0x3FFCA is used!), though. It also possible there's other code elsewhere that might need to be updated too; I haven't done that analysis for you this time either ;).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 10, 2019, 02:36:55 pm
Oh Happy Day! Feeling overwhelmed by the idea of having to move important code around, I took another crack at editing the dictionary, swapping out entries with about 15 or so uses with ones much higher and I got my script down to 32,378 bytes! And I can totally trim the fat in my script if need be. That should be much easier! What a relief!

So, if I keep bank 5 as default and go with your suggestion of using the space at 30010 (thanks btw, I didn't really think of looking that far, I didn't think there'd be full unused banks) which would be bank... 10, right? That would change the value of 3FA94 to 0A.

Now I just need to add the bit from your earlier post so it can split the script properly. Edit: yes the JMP is updated to reflect the dumping to the new bank.


Code: [Select]
// add this near the top of the insert script:
#VAR(pointerNum, COUNTER) // create a COUNTER variable named pointerNum
#CREATECTR(pointerNum, 8, 0) // pointerNum is an 8-bit value initialized to 0
#AUTOCMD($17FE7, #WLB(pointerNum, $3FA90)) // update the code that controls which pointer starts the next bank

// and then after every #W16 line in the insert script, add:
#INC(pointerNum, 1)

// Jump to start of script
#JMP($14010)
#HDR($C010)
// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
#AUTOCMD($17FE7, #HDR($10))
#AUTOCMD($17FE7, #JMP($30010, $3400F))

Edit: yes the JMP is updated to reflect the dumping to the new bank. I seem to be close. Once trying with my full script (as opposed to pieces to determine length and where the best split would be) I get "#JMP bounded by at $3400F has -1223 ($-4C7) space left" so it seems maybe it's still a bit too big? Did I make a miscalculation somewhere? Either way, 1K to trim down should be easy and inconsequential.
Title: Re: General NES Hacking Questions
Post by: abw on March 10, 2019, 05:05:58 pm
Oh Happy Day! Feeling overwhelmed by the idea of having to move important code around, I took another crack at editing the dictionary, swapping out entries with about 15 or so uses with ones much higher and I got my script down to 32,378 bytes! And I can totally trim the fat in my script if need be. That should be much easier! What a relief!
Nice!

So, if I keep bank 5 as default and go with your suggestion of using the space at 30010 (thanks btw, I didn't really think of looking that far, I didn't think there'd be full unused banks) which would be bank... 10, right? That would change the value of 3FA94 to 0A.
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

If you're still having space issues, don't forget about the single-character dictionary entries - if you're e.g. using "p" more than "w", you could kick "w" into the 10-bit range and move "p" into the 5-bit range to get better compression. For English text, you can probably also replace the "q" entry with "qu".
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 10, 2019, 05:24:29 pm
Nice!
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

Well, as said a few posts ago, I split my script into two parts to determine how much of it would fit. I took the bytes remaining as reported by abcde and subtracted from the total space available and did it again for the second part then combined the values which is how I got the 32,378 bytes. I had done the same thing before and I was getting over 34,000 bytes, so even if my math is somehow off it's still a significant decrease.

Nice!
Remember how we expanded the DW1 ROM by adding 4 new banks just before the final fixed bank? Enix did basically the same thing (except in hardware) when porting the game to English; they had to add 8 banks, but didn't end up using 3 of those.

In bank 5, you've got $17FE7 - $14010 = $3FD7 bytes, and bank 12 ($0C) gives you another $4000 bytes for a total of $7FD7 = 32,727 bytes, which should leave you with about 350 bytes to spare if you managed to get your script down to 32,378 bytes. Is that 32,378 bytes the script size after being converted to DW2's 5/10-bit encoding, or is that the file size when encoded as UTF-8?

If you're still having space issues, don't forget about the single-character dictionary entries - if you're e.g. using "p" more than "w", you could kick "w" into the 10-bit range and move "p" into the 5-bit range to get better compression. For English text, you can probably also replace the "q" entry with "qu".
Yeah that could work. I would assume, in layman's terms, that the first set of entries use less data than the 4 "C" tables? Or does each table use extra bits of data the further you go down? (Edit : forget that I'm dumb)I honestly still don't quite understand how those work, I initially thought it was like a special table where'd you use two bytes to load a dictionary value, and pretty much every word/letter from the dictionary wound be preceded by the corresponding value and then stuff from the first table fills in the rest with no dictionary value needed.

Edit : Just read the Wiki, I guess it works like I said above but in bits, not bytes?
Title: Re: General NES Hacking Questions
Post by: abw on March 10, 2019, 06:12:36 pm
Yeah that could work. I would assume, in layman's terms, that the first set of entries use less data than the 4 "C" tables? Or does each table use extra bits of data the further you go down? (Edit : forget that I'm dumb)I honestly still don't quite understand how those work, I initially thought it was like a special table where'd you use two bytes to load a dictionary value, and pretty much every word/letter from the dictionary wound be preceded by the corresponding value and then stuff from the first table fills in the rest with no dictionary value needed.

Edit : Just read the Wiki, I guess it works like I said above but in bits, not bytes?
Basically, yeah. I had this typed up before I saw your edit, so I'll post it anyway just in case it's still helpful:
--
Sort of, except as far as the game's concerned, the individual tokens are 5 bits each, not 8, and 4 of the entries from the first 5-bit table are used to switch to the corresponding "C" table. You could split that single table file into multiple table files if you wanted to; I just prefer having all the entries in a single table.

You can tell how much space each entry takes up by looking at its left-hand side. In a hexadecimal table, "80" takes up one byte (or 2 nybbles or 8 bits, depending on how you want to think of it), "80FF" takes up two bytes (/4 nybbles/16 bits), and so on; in a binary table, "11011" takes up 5 bits, "1110101111" takes up 10 bits, and so on. This will be an even bigger issue if you decide to tackle DW4, since it uses a Huffman encoding where the binary representation of individual characters ranges between 3 bits (e.g. for "e") and 18 bits (e.g. for "8").
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 11, 2019, 03:18:44 pm
Basically, yeah. I had this typed up before I saw your edit, so I'll post it anyway just in case it's still helpful:
--
Sort of, except as far as the game's concerned, the individual tokens are 5 bits each, not 8, and 4 of the entries from the first 5-bit table are used to switch to the corresponding "C" table. You could split that single table file into multiple table files if you wanted to; I just prefer having all the entries in a single table.

You can tell how much space each entry takes up by looking at its left-hand side. In a hexadecimal table, "80" takes up one byte (or 2 nybbles or 8 bits, depending on how you want to think of it), "80FF" takes up two bytes (/4 nybbles/16 bits), and so on; in a binary table, "11011" takes up 5 bits, "1110101111" takes up 10 bits, and so on. This will be an even bigger issue if you decide to tackle DW4, since it uses a Huffman encoding where the binary representation of individual characters ranges between 3 bits (e.g. for "e") and 18 bits (e.g. for "8").

Thanks, once I saw that the programmer's calculator had a binary converter, it just clicked.

Anyway, just from doing the above suggestion I'm down another 400 bytes! And just after some needed editing of the intro I'm now down to 760 bytes over the limit. I was really loose and liberal with my initial script (like gratuitous line breaks, pauses, and spaces), so it should be easy to trim the fat without sacrificing much. Although when I tried dumping with the new second address, I'd get horrible garbage dialog. The intro is only the 8th pointer in bank 5, so I'm not too sure what happened there. I know I have to update the ROM with the new second bank, but I'm not sure why dialog in the beginning wouldn't work properly. Did I miss something (other than updating the second bank?)

A couple other weird issues

Still having trouble with closing single quotes for some reason.

(https://i.imgur.com/w2bUVKS.png)

I double checked the ROM and I do indeed have $66 in the dictionary. I also have an entry with "!’" but that has Apostrophes ($67) work fine though

Also

(https://i.imgur.com/VrR1E2T.png)

Not sure what's going on here. There's a couple of windows like this with the oddly premature uncalled for line break. I'm assuming the window counts how many bits are used per line as opposed to individual letters/spaces/characters? Any suggestions? Or is it just a weird game programming/abcde limitation?
Title: Re: General NES Hacking Questions
Post by: abw on March 11, 2019, 08:03:16 pm
Hurray! The proper choice of dictionary entries makes a huge difference to the compression ratio. Just make sure you leave the first 5 end tokens and the 4 "C" table switches alone (since the game cares about those) and fill out the rest of the main 5-bit table with your 23 best tokens.

As for your issues, it's kind of hard to say for sure without seeing the files. If dumping the inserted data didn't work but the game still displays it correctly, then something is wrong with your dump script. For the closing single quotes, my next guess would be an error in the dictionary lengths, though a length error would also mess up all the dictionary entries following the one with the wrong length. The random line breaks is definitely strange too, and not an issue I've come across myself; I've definitely inserted longer strings than that without getting inappropriate line breaks. I haven't looked at the code for it, but just based on observation, DW2's line wrapping algorithm seems pretty solid. Any chance it always happens around the words "What" or "is"? If so, that could indicate that you've got a line break instead of a space in one of the dictionary entries - just a guess!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on March 16, 2019, 06:27:57 pm
I got it! I had used Pointer Tables to extract, edit, and insert the dictionary entries. Yeah, I know it's clunky, but easy for me. It was changing blank spaces (5F) to [no voice] (59). Once I changed it in the ROM it was fine. Similar problem with the closing single quotes and I fixed that too. I was hoping that this could've shrunk the script size, but, alas, it didn't. Oh well, more editing work for me!

Congrats on your Latin translation release, abw! I had looked at the Read Me and you said one of your improvements was editing the main party status windows to show (was this battle or field or both?) full character names instead of just showing the first four letters.

When working on DW1 menus, the thought of doing that for later entries struck me as a "would be nice if possible". You mentioned it was more than a one byte hack. Were there other entries for the window you had to change besides length? Or is it like monster/item/spell/etc names where you had to find that value that affected display length? Btw, I know it'd be easy to tell the magic byte that affects monster name length like DW1, but you did explain how you did it and I have every intention to try myself as part of my learning.
Title: Re: General NES Hacking Questions
Post by: abw on March 17, 2019, 05:53:07 pm
I got it!
Congrats!

Congrats on your Latin translation release, abw! I had looked at the Read Me and you said one of your improvements was editing the main party status windows to show (was this battle or field or both?) full character names instead of just showing the first four letters.

When working on DW1 menus, the thought of doing that for later entries struck me as a "would be nice if possible". You mentioned it was more than a one byte hack. Were there other entries for the window you had to change besides length? Or is it like monster/item/spell/etc names where you had to find that value that affected display length?
Thanks! If you check the screenshots, you'll see that full character names are displayed both in and out of battle; I think I made changes for displaying full names in a total of 37 different menus, which typically also involved widening and repositioning them and sometimes also involved making updates to the menu wiping process; the WEAPON/ARMOR/SHILD/HELMET menus were particularly irritating to deal with in that respect. For the code changes, the basic problem is that DW2 stores the first 4 characters of each hero's name in one spot and the last 4 characters in a completely different spot (probably due to the English version's extra 4 characters being bolted on to the original 4 characters present in the Japanese version), so you can't just find the code that says "read 4 bytes from X" and update one byte to say "read 8 bytes from X", you need to add more code (which needs more space) to read 4 bytes from X and glue them together with 4 bytes from Y.

For the later games in the series, the mini status menu was rearranged into 1 column per hero instead of 1 row per hero, so widening the columns probably wouldn't work out very well due to lack of screen space if everybody has a long name, but you could switch the layout back to rows if you wanted. It's just a question of how much work would be involved.

Btw, I know it'd be easy to tell the magic byte that affects monster name length like DW1, but you did explain how you did it and I have every intention to try myself as part of my learning.
Trace logger is your friend there too!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 16, 2019, 03:08:52 pm
Basck from vacation and ready to tackle this!

Okay, I'm having a strange pointer issue (at least I think it's Pointer related). I got my script (with around 19 bytes to spare!)

When the game loads the intro, instead of loading the line

"This is the royal castle of the kingdom of Moonbrooke." which is Pointer 7, String 14 it loads instead, "‘We shall await thy return most eagerly!’" Which is a couple lines up at String 12. Granted I removed the "Hold Reset" dialog to save space but I double checked and I don't see any missing end-FC marks I might've missed that would cause the issues. What's weird is, in the next pointer, the first two lines load just fine but everything after is mixed up.

Did I mess up the formatting of my atlas.txt somehow? It's right here
http://www.mediafire.com/file/e7kqqhf720plzpf/atlas.txt/file

(note because of the trouble Chicken Knife had with his script mentioned earlier in this thread, I opted to extract the script with comments:off so it's just the new dialog)

I know I read your post, abw, a couple pages back about the ROM flaw that pointers could get mixed up, but I'm not sure I understand what you mean by checking them. In DW1, I could load up the ROM in Windhex, look at the start of the string, the address, then compare it with the pointer, but I'm not sure how I can do that with DW2's compression.
Title: Re: General NES Hacking Questions
Post by: darthvaderx on April 16, 2019, 05:26:13 pm
Is it possible to run Lua Script in Mesen in the same way as in FCEUX? I've tried to run Metroid in every way but none worked.
Title: Re: General NES Hacking Questions
Post by: abw on April 16, 2019, 09:56:29 pm
When the game loads the intro, instead of loading the line

"This is the royal castle of the kingdom of Moonbrooke." which is Pointer 7, String 14 it loads instead, "‘We shall await thy return most eagerly!’" Which is a couple lines up at String 12.
I'm not seeing anything immediately obvious that would cause that kind of error, and a quick test with a version of the original table file tweaked to include your new characters gets me the expected text displayed during the entire intro sequence, so I suspect your problem lies elsewhere. Getting an earlier string from the same pointer makes me think the game might be counting more end tokens than you want it to... what does your table file look like? If you altered any of the first 5 entries, that might explain this behaviour.

I know I read your post, abw, a couple pages back about the ROM flaw that pointers could get mixed up, but I'm not sure I understand what you mean by checking them.
Basically you just pop open the ROM in a hex editor, look at 0xB772-0xB7D0, and check whether any of those pointers point to $BFD6 or $BFD7. If they do, then you'll need to move the auto-jump point (0x17FE7) back a byte or two in order to force the affected pointer to point into the second script bank instead of letting it read non-script data from the first script bank and make a mess of your text.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 18, 2019, 04:42:11 pm
I'm not seeing anything immediately obvious that would cause that kind of error, and a quick test with a version of the original table file tweaked to include your new characters gets me the expected text displayed during the entire intro sequence, so I suspect your problem lies elsewhere. Getting an earlier string from the same pointer makes me think the game might be counting more end tokens than you want it to... what does your table file look like? If you altered any of the first 5 entries, that might explain this behaviour.
Basically you just pop open the ROM in a hex editor, look at 0xB772-0xB7D0, and check whether any of those pointers point to $BFD6 or $BFD7. If they do, then you'll need to move the auto-jump point (0x17FE7) back a byte or two in order to force the affected pointer to point into the second script bank instead of letting it read non-script data from the first script bank and make a mess of your text.

Okay that went better than I hoped!  :laugh:

So, when you suggested the problem (thanks for that, it's easy for me to overlook little things like that), it was indeed something I did with my table/dictionary. I thought I'd swap the standalone [FF] with the standalone Period that was further down. Once I swapped it back, the intro text displayed fine again. Lesson learned. Adding a note in my copy of the table: DO NOT MESS WITH THE FIRST 5 VALUES! Only problem was I went from just being under the limit by about 20 bytes, to being OVER the limit by about 60. So I had to scour the dictionary again to see if there was some way to get the size back down. After adding a space to the comma entry and replacing the unused ; by realizing I had a lot of words and phrases that ended in "eth" and boom that was enough to put me WELL under with about 140 bytes to spare. WOW! I'm ecstatic! I should be able to restore the little bit I had to cut (mostly consists of long gratuitous laughs, pauses, and a post-final boss NPC line).

Anyway when I asked about changing the second text bank at 3FA94, I figured having the second block at $3400F would make it Bank 10 (0A) but when you replied with an explanation of how the bank swapping work, you mentioned Bank 12 and I'm not sure if you were just giving an unrelated example, subtly correcting me, or just made a typo. I'm getting some messed up text by a couple of the Midenhall Castle NPCs using either value (either blank windows or a sparse random letter) so i figured I'll have to look at those pointers, but I want to make sure I'm not doing this with the wrong bank value.

EDIT: These are my values between b772-B7D0

16 90 73 92 7D 92 87 92 91 92 9B 92 A5 92 AF 92 B9 92 02 94 19 95 3E 96 12 98 37 99 12 9C 62 9E 35 A1 0A A4 CE A7 AB AB 5E AF 68 AF 72 AF 7C AF 86 AF 42 B4 37 BB 43 02 B1 07 93 0E 4A 13 FC 17 7D 1B 88 1E 1C 22 54 27 A0 2D 9D 31 16 35 0A 39 DF B4 EB A0 4D BB 17 F4 F9 C3 E2 57 C0 E7 8F 9D

So, nothing wrong there, then?

Edit 2: So am I understating right that my problem is probably between Pointer 32 and 35 which is pointer B7A2 and B7A8 respectively? Since B7A8 points to the new bank? Pointer 32 has a lot of the Midenhall dialog, some of which is displayed properly. Sorry to say, I'm still puzzled here.
Title: Re: General NES Hacking Questions
Post by: abw on April 19, 2019, 12:01:43 am
Okay that went better than I hoped!  :laugh:
[...]
Adding a note in my copy of the table: DO NOT MESS WITH THE FIRST 5 VALUES!
Hurray! To be fair, I did actually mention that a few posts back (http://www.romhacking.net/forum/index.php?topic=27053.msg372393#msg372393), but I'm glad you've got it working now!

Anyway when I asked about changing the second text bank at 3FA94, I figured having the second block at $3400F would make it Bank 10 (0A) but when you replied with an explanation of how the bank swapping work, you mentioned Bank 12 and I'm not sure if you were just giving an unrelated example, subtly correcting me, or just made a typo.
Heh, that one I managed to write down correctly for a change :P. Banks $0C (starting at 0x030010), $0D (0x034010), and $0E (0x038010) are DW2's 3 empty banks.

I'm getting some messed up text by a couple of the Midenhall Castle NPCs using either value (either blank windows or a sparse random letter) so i figured I'll have to look at those pointers, but I want to make sure I'm not doing this with the wrong bank value.
Yup, that makes sense. The main script's pointer table actually starts at 0xB762, so you're missing the first 8 pointers, but they're probably fine so I'm not too concerned about that. The pointer table also stops at 0xB7C1, so the other 16 bytes you included here are just the start of the original second script bank's text and aren't really relevant. If you look at the rest of the pointer values, you'll see they keep increasing from $9016 at 0xB772 up to $BB37 at 0xB7A6 and then reset (that'll be the first pointer following the auto-jump) to $0243 at 0xB7A8 and start increasing again up to $390A at 0xB7C0, so those pointer values are all off by $8000 and giving you who knows what from system RAM instead of the data you want from cartridge ROM (as an added bonus, the $2002 PPU register is sensitive to reads, so if you're really lucky, you might also get some other graphical mayhem happening :D). Checking your insert script, I see that you've updated where to jump to when the insertion point reaches 0x17FE7 but didn't update the corresponding header value, which is throwing the pointer value calculations off. Try changing that to #AUTOCMD($17FE7, #HDR($28010)) instead and you should get the right pointer values.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 19, 2019, 12:05:26 pm
It occurred to me just before reading your reply that it was probably the pointer values being off for the new bank (you did post that "formula" for pointer values a while back). Had no idea that was what the HDR command in the script was for. Lesson learned for the future, but yep, that did it, it's working beautifully. Just got to start testing and tweaking and then doing all the menu stuff. Man that was nuts but that's a huge hurdle cleared for me thank you so much! :beer:

Edit: Chicken Knife gave me his atlas.txt so I could compare translations and I realized you have the "pointer formula" right in there. Doh! XP
Title: Re: General NES Hacking Questions
Post by: abw on April 20, 2019, 12:51:11 pm
Lesson learned for the future, but yep, that did it, it's working beautifully. Just got to start testing and tweaking and then doing all the menu stuff. Man that was nuts but that's a huge hurdle cleared for me thank you so much! :beer:
Sweet! Let's hope the testing and tweaking goes well too :).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 22, 2019, 01:54:37 pm
Okay sorry to post in this thread so soon, but I'm genuinely stumped by the plural rules (I foolishly thought it was a bunch of suffixes in a row, kinda like the dictionary)  . With new monster names need a couple of tweaks not covered by the current plural rules

New names

Mummy Boy > Boys
Hunter Mech > Mechs
Madusa > Madusae
Dragonfry > No change
Gigantes > No change (though I'm not 100% sure if a plural is even needed here, I don't think Gigantes are encountered in groups)
Cyclops > No change (actually I just read here (https://www.grammarphobia.com/blog/2013/08/cyclops.html) that Square-Enix goofed and that the correct plural for a cyclops is "Cyclopes", though I suppose if need be, Cyclopses isn't entirely wrong and off limits)
Technically Man o' War is listed as "Men o' War" as its plural but I figure with the way the system counts from the end and adds to the end Man o' Wars would be just fine

The good news is that the new names also free up the need for a few rules

ch > ches (only the Hunter Mech is used and that's a case where it only needs the s added)
dead > no change
ngo > no change
f > ves (unused anyway)
i > ies
Mouse/mouse > Mice/mice (they're rats now)
rus>rii
sh>shes


I mean I get that there's essentially two parts of that code. One looks at the end letters and then points to other part that removes/adds letters at the end. Even for something like editing the "ngo" if I wanted to change it to "fry" I only see the "n" and "g" to change, I don't see the o

Code: [Select]
; -o pluralization handler: -ngo -> -ngo, -o -> -os
0x01C845|$07:$8835:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
0x01C848|$07:$8838:C9 10    CMP #$10 ; "g"
0x01C84A|$07:$883A:D0 E7    BNE $8823 ; if not -go, append "s"
0x01C84C|$07:$883C:BD EF 60 LDA $60EF,X ; read third-last letter of monster name
0x01C84F|$07:$883F:C9 17    CMP #$17 ; "n"
0x01C851|$07:$8841:D0 E0    BNE $8823 ; if not -ngo, append "s"
0x01C853|$07:$8843:F0 EB    BEQ $8830 ; if -ngo, plural = singular

I guess what really confuses me are the if/if not suffix lines. How do I change THOSE?

Edit: Also doing Menus (no trouble understanding those :) ) but are you sure those pointers you listed on the wiki are right, abw? I'm getting some really broken up strings like

Nevermind, I misread the text start value as 7656 instead of 76E6 which was throwing me off.
Title: Re: General NES Hacking Questions
Post by: abw on April 22, 2019, 08:20:23 pm
Okay sorry to post in this thread so soon, but I'm genuinely stumped by the plural rules (I foolishly thought it was a bunch of suffixes in a row, kinda like the dictionary)  .
Nope, the pluralization rules are implemented as actual code, not just a table of suffixes or similar, so the bad news is you're going to have to roll up your sleeves and dig in to it a bit in order to make your changes. On the other hand, since you're going to be making changes anyway, you're free to make all the changes you want - you've got all the space from 0x01C801 to 0x01C8F3 to do whatever you want, and if that isn't enough space, most of the rest of the bank is empty. When this code starts running, the singular monster name has already been laid out in RAM at $6119 for you, and by the end, you just need to write the plural form to RAM starting at $60F1 and SEC before you RTS. You can use the debugger to set a breakpoint at the start and step through the code to watch exactly what it does.

For -ngo specifically, it works like this:

$87FC: LDA $60F1,X    ; read the final letter of the monster name into A (in this case that's "o", a.k.a. #$18; I think this is the part you missed)
$87FF: CMP #$18       ; among other things, this sets the Z (zero) processor flag based on whether A is #$18 ("o") or not
$8801: BEQ $8835      ; since Z is set, BEQ follows the branch

; at this point we know the last letter was "o", so we'll check the second-last letter
$8835: LDA $60F0,X    ; read the second-last letter the of monster name into A (in this case that's "g", a.k.a. #$10)
$8838: CMP #$10       ; as before, this sets the Z flag based on whether A is #$10 ("g") or not
$883A: BNE $8823      ; since A is #$10, Z is not set and we don't take this branch

; at this point we know the last two letters were "go", so we'll check the third-last letter
$883C: LDA $60EF,X    ; read the third-last letter of the monster name (in this case that's "n", a.k.a. #$17)
$883F: CMP #$17       ; as before, this sets the Z flag based on whether A is #$17 ("n") or not
$8841: BNE $8823      ; since A is #$17, Z is not set and we don't take this branch
$8843: BEQ $8830      ; ... which means we do take this branch
$8830: SEC            ; the calling code cares about whether C is set or not
$8831: RTS            ; and at this point we're done, not having changed a single byte of the monster name
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 22, 2019, 10:51:19 pm
Nope, the pluralization rules are implemented as actual code, not just a table of suffixes or similar, so the bad news is you're going to have to roll up your sleeves and dig in to it a bit in order to make your changes. On the other hand, since you're going to be making changes anyway, you're free to make all the changes you want - you've got all the space from 0x01C801 to 0x01C8F3 to do whatever you want, and if that isn't enough space, most of the rest of the bank is empty. When this code starts running, the singular monster name has already been laid out in RAM at $6119 for you, and by the end, you just need to write the plural form to RAM starting at $60F1 and SEC before you RTS. You can use the debugger to set a breakpoint at the start and step through the code to watch exactly what it does.

For -ngo specifically, it works like this:

$87FC: LDA $60F1,X    ; read the final letter of the monster name into A (in this case that's "o", a.k.a. #$18; I think this is the part you missed)
$87FF: CMP #$18       ; among other things, this sets the Z (zero) processor flag based on whether A is #$18 ("o") or not
$8801: BEQ $8835      ; since Z is set, BEQ follows the branch

; at this point we know the last letter was "o", so we'll check the second-last letter
$8835: LDA $60F0,X    ; read the second-last letter the of monster name into A (in this case that's "g", a.k.a. #$10)
$8838: CMP #$10       ; as before, this sets the Z flag based on whether A is #$10 ("g") or not
$883A: BNE $8823      ; since A is #$10, Z is not set and we don't take this branch

; at this point we know the last two letters were "go", so we'll check the third-last letter
$883C: LDA $60EF,X    ; read the third-last letter of the monster name (in this case that's "n", a.k.a. #$17)
$883F: CMP #$17       ; as before, this sets the Z flag based on whether A is #$17 ("n") or not
$8841: BNE $8823      ; since A is #$17, Z is not set and we don't take this branch
$8843: BEQ $8830      ; ... which means we do take this branch
$8830: SEC            ; the calling code cares about whether C is set or not
$8831: RTS            ; and at this point we're done, not having changed a single byte of the monster name


Thanks that explains it a little better. I now have an idea of what to do, I just don't know the commands. I realize fixing "Mechs" would be the easiest thing, I would just have to remove the line that checked the second to last letter for a c $8875-8878

But I looked at your notes on the Wiki and sort of brainstormed what I WANT to do.

Fixing S rules for Gigantes, Magus
Code: [Select]
; read second-last letter of monster name
; "u"
; if not -us, append "es"
; read third-last letter of monster name
; "g"
; if not -gus, append "es"
; read second-last letter of monster name
; "e"
; if not -es, append "es"
; read third-last letter of monster name
; "t"
; if -tes, don't change $8830
; "i"
; replace -us with -i
; Go to $8830
; replace final letter with "i"
; append "es"


Fixing Mummy Boy and Dragonfry
Code: [Select]
; "i"
; read second-last letter of monster name
; "o"
; if "-oy", append "s"
; "r"
; if "-ry", don't change ($8830)

Fixing Madusa

Code: [Select]
; read second-last letter of monster name
; "s"
; if not -sa, append "s"
; read third-last letter of monster name
; "u"
; "e"append e
; Go to $8830

It's something like that right? Again I get the programming gist of it, but I'm not sure what hex notation does what.

On a different note, thankfully window editing has gone quite well. I can see what you're talking about a few posts ago with the US "addition" of extra names. By expanding the Battle Command window, I realized I could display full names at the top of the window now, but strangely any unused character spaces still "cover" the top bar of the window. That's a real shame, otherwise it's neat! I'm thinking of keeping the change.

Edit: So as brought to my attention by laser lambert, apparently the special [(s)] (F2) doesn't work the same way as DW1. If used for "points", whenever there's 1, it just cuts off the string (experience) or doesn't display the whole window at all (points of damage in battle). Don't want to be a bother (I really really don't I swear! ^^'') But even if I used the Trace Logger/Debugger to find the routine I wouldn't know what to look for, I'm mostly just curious, but is there any way to fix that?
Title: Re: General NES Hacking Questions
Post by: abw on April 24, 2019, 12:06:32 am
When I was doing this for Latin, I took my list of singular monster names (I excluded the unique bosses since they never need plural forms), sorted it right-to-left so that I had all the same suffixes grouped together, wrote out the correct plural form for each name, did some extra Latin-specific stuff you won't need to deal with for English, and then wrote out the singular -> plural transformation rules I needed. Once I had that, writing the code was fairly straightforward; it's a little bit long and boring, but the individual sections are pretty simple. This is an oversimplification (particularly for CMP, which does multiple other things simultaneously), but for the purposes of this exercise, you can treat CMP #$hex as "check if A is #$hex", BEQ $addr as "if it is, go to $addr", and BNE $addr as "if it isn't, go to $addr".

As with script editing, I recommend writing code as code rather than typing things in to a hex editor, so you'll want to grab an assembler (anything that works for 65816 should also work for 6502, so pick your favourite SNES or NES assembler; I was actually using Asar (http://www.romhacking.net/utilities/964/), but I've been meaning to give xkas-plus (http://www.romhacking.net/forum/index.php?topic=19640.0) a try), and if you haven't done any ASM before, you'll also want some reference material (I like the handy instruction descriptions in chapter 18 of Programming the 65816 (http://www.romhacking.net/documents/423/), but feel free to consult other resources if that one doesn't suit your style).

Also, if you get the chance to be clever, take it! E.g. if you only have one monster name that ends in "r" and you're writing the code to handle names that end in "r", you don't have to check that the 2nd-last letter is "a" or the 3rd-last letter is "W" or anything else, you can immediately just change the 9th-last letter to "e" to get "Man o' War" pluralized to "Men o' War".

By expanding the Battle Command window, I realized I could display full names at the top of the window now, but strangely any unused character spaces still "cover" the top bar of the window. That's a real shame, otherwise it's neat! I'm thinking of keeping the change.
Check out the screenshots from my translation - I also added some code for changing trailing spaces in hero names to top borders when the names are displayed as part of a border :P.

Edit: So as brought to my attention by laser lambert, apparently the special [(s)] (F2) doesn't work the same way as DW1. If used for "points", whenever there's 1, it just cuts off the string (experience) or doesn't display the whole window at all (points of damage in battle). Don't want to be a bother (I really really don't I swear! ^^'') But even if I used the Trace Logger/Debugger to find the routine I wouldn't know what to look for, I'm mostly just curious, but is there any way to fix that?
Much like the [name] control code, [(s)] just tests the 16-bit value at $8F; the code for that is actually right before the pluralization code:
Code: [Select]
; data -> code
; if $8F-$90 == #$0001, print "s" + [end-FA] to $60F1 and SEC, else print [end-FA] and CLC
; indirect control flow target
; from $02:$BE37 via $8006
0x01C7E8|$07:$87D8:A5 90    LDA $90
0x01C7EA|$07:$87DA:D0 05    BNE $87E1 ; if $90 > 0, add "s"
0x01C7EC|$07:$87DC:A4 8F    LDY $8F
0x01C7EE|$07:$87DE:88      DEY
0x01C7EF|$07:$87DF:F0 0C    BEQ $87ED ; if $90 == 0 and $8F - 1 == 0 (i.e. $8F == 1), do not add "s"
; control flow target (from $87DA)
0x01C7F1|$07:$87E1:A9 1C    LDA #$1C ; "s"
0x01C7F3|$07:$87E3:8D F1 60 STA $60F1
0x01C7F6|$07:$87E6:A9 FA    LDA #$FA ; [end-FA]
0x01C7F8|$07:$87E8:8D F2 60 STA $60F2
0x01C7FB|$07:$87EB:38      SEC
0x01C7FC|$07:$87EC:60      RTS

; control flow target (from $87DF)
0x01C7FD|$07:$87ED:A9 FA    LDA #$FA ; [end-FA]
0x01C7FF|$07:$87EF:18      CLC
0x01C800|$07:$87F0:60      RTS
The [number] control code, which is used for both experience and gold gains, also uses $8F-$90, so I would have expected [(s)] to work for both of those situations.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 24, 2019, 06:41:42 am
When I was doing this for Latin, I took my list of singular monster names (I excluded the unique bosses since they never need plural forms), sorted it right-to-left so that I had all the same suffixes grouped together, wrote out the correct plural form for each name, did some extra Latin-specific stuff you won't need to deal with for English, and then wrote out the singular -> plural transformation rules I needed. Once I had that, writing the code was fairly straightforward; it's a little bit long and boring, but the individual sections are pretty simple. This is an oversimplification (particularly for CMP, which does multiple other things simultaneously), but for the purposes of this exercise, you can treat CMP #$hex as "check if A is #$hex", BEQ $addr as "if it is, go to $addr", and BNE $addr as "if it isn't, go to $addr".

As with script editing, I recommend writing code as code rather than typing things in to a hex editor, so you'll want to grab an assembler (anything that works for 65816 should also work for 6502, so pick your favourite SNES or NES assembler; I was actually using Asar (http://www.romhacking.net/utilities/964/), but I've been meaning to give xkas-plus (http://www.romhacking.net/forum/index.php?topic=19640.0) a try), and if you haven't done any ASM before, you'll also want some reference material (I like the handy instruction descriptions in chapter 18 of Programming the 65816 (http://www.romhacking.net/documents/423/), but feel free to consult other resources if that one doesn't suit your style).

Also, if you get the chance to be clever, take it! E.g. if you only have one monster name that ends in "r" and you're writing the code to handle names that end in "r", you don't have to check that the 2nd-last letter is "a" or the 3rd-last letter is "W" or anything else, you can immediately just change the 9th-last letter to "e" to get "Man o' War" pluralized to "Men o' War".

Thanks. I was worried my post came across too "Wahh I don't wanna! Just do it for me please." which wasn't what I intended at all. But that's exactly the info I was hoping for!

Quote
Check out the screenshots from my translation - I also added some code for changing trailing spaces in hero names to top borders when the names are displayed as part of a border :P.

Ah, you sly dog! Guess I'll have to download your translation and see what you did.

Speaking of. I was earnestly going to try to find monster and spell name length and used your explanation in that previous DW1 thread. But looking it over it confused me more than I thought it would.

In your example you said I could look up HEAL by setting a Read Breakpoint at $B5E6. The thing is looking back at the rom, I can't see what B5E6 is supposed to be. The pointer value was something similar but different like B8B6 (sorry away from my computer right now, something like that).

Am I missing a real obvious conversion or something? I thought the magic RAM to ROM conversion number was 3FF0?

Quote
Much like the [name] control code, [(s)] just tests the 16-bit value at $8F; the code for that is actually right before the pluralization code:
Code: [Select]
; data -> code
; if $8F-$90 == #$0001, print "s" + [end-FA] to $60F1 and SEC, else print [end-FA] and CLC
; indirect control flow target
; from $02:$BE37 via $8006
0x01C7E8|$07:$87D8:A5 90    LDA $90
0x01C7EA|$07:$87DA:D0 05    BNE $87E1 ; if $90 > 0, add "s"
0x01C7EC|$07:$87DC:A4 8F    LDY $8F
0x01C7EE|$07:$87DE:88      DEY
0x01C7EF|$07:$87DF:F0 0C    BEQ $87ED ; if $90 == 0 and $8F - 1 == 0 (i.e. $8F == 1), do not add "s"
; control flow target (from $87DA)
0x01C7F1|$07:$87E1:A9 1C    LDA #$1C ; "s"
0x01C7F3|$07:$87E3:8D F1 60 STA $60F1
0x01C7F6|$07:$87E6:A9 FA    LDA #$FA ; [end-FA]
0x01C7F8|$07:$87E8:8D F2 60 STA $60F2
0x01C7FB|$07:$87EB:38      SEC
0x01C7FC|$07:$87EC:60      RTS

; control flow target (from $87DF)
0x01C7FD|$07:$87ED:A9 FA    LDA #$FA ; [end-FA]
0x01C7FF|$07:$87EF:18      CLC
0x01C800|$07:$87F0:60      RTS
The [number] control code, which is used for both experience and gold gains, also uses $8F-$90, so I would have expected [(s)] to work for both of those situations.

Thanks again for this. How strange but I guess that's why the table only uses it for Gold and nothing else, maybe the devs just couldn't get it working properly? In a way I don't find it with fretting over. I could just tweak the script and then change my "point(s)" dictionary entry and be able to save MORE space that way.
Title: Re: General NES Hacking Questions
Post by: abw on April 24, 2019, 08:09:25 pm
Thanks. I was worried my post came across too "Wahh I don't wanna! Just do it for me please." which wasn't what I intended at all. But that's exactly the info I was hoping for!
We aim to please ;D.

Ah, you sly dog! Guess I'll have to download your translation and see what you did.
You won't be able to completely copy my code since I cannibalized the free space I created by shortening "ADVENTURE LOG" to "VOLUMEN", but the ASM for handling menu control codes $98 - $9F starts at 0x3ED8A; the original game ran the same code for all of $9A - $9F, so I stole $9B - $9F for the "names in border" code.

Speaking of. I was earnestly going to try to find monster and spell name length and used your explanation in that previous DW1 thread. But looking it over it confused me more than I thought it would.

In your example you said I could look up HEAL by setting a Read Breakpoint at $B5E6. The thing is looking back at the rom, I can't see what B5E6 is supposed to be. The pointer value was something similar but different like B8B6 (sorry away from my computer right now, something like that).

Am I missing a real obvious conversion or something? I thought the magic RAM to ROM conversion number was 3FF0?
Sorry, part of the problem here is that I can't type properly :'(. I've updated that post to use the correct breakpoint of $BE56 as shown in the debugger snapshot instead of $B5E6; after that, it goes back to the relationship between RAM and ROM addresses: the read breakpoint uses RAM address $BE56, which for DW1 corresponds to a ROM offset of 0x3E66 (not the address you're looking for), 0x7E66 (pick me because I say "HEAL"!), or 0xBE66 (also not the address you're looking for), or theoretically 0xFE66 (but this is reserved for the fixed bank, so it's also not the address you're looking for). Does it make any more sense like this?
DW1 ROM Offset     DW1 RAM Address   DW1 RAM to ROM Conversion Number
$0010 - $400F      $8000 - $BFFF     -$7FF0
$4010 - $800F      $8000 - $BFFF     -$3FF0
$8010 - $C00F      $8000 - $BFFF     +$10
$C010 - $1000F     $C000 - $FFFF     +$4010
Thanks again for this. How strange but I guess that's why the table only uses it for Gold and nothing else, maybe the devs just couldn't get it working properly? In a way I don't find it with fretting over. I could just tweak the script and then change my "point(s)" dictionary entry and be able to save MORE space that way.
Wait a minute... Am I missing something, or is getting 1 piece of gold in this game actually impossible? Slimes are worth 2 gold, unique saleable items go for 2 gold, and even buying an Antidote Herb with a Golden Card still gets you a discount of 2 gold...

Hmm, yup, that F2 is just plain broken when $8F-$90 is #$0001. Instead of
Code: [Select]
LDA #$FA
CLC
RTS
which as you've seen results in the rest of the string getting cut off, it should be doing
Code: [Select]
LDA #$FA
STA $60F1
SEC
RTS
So the good news is: it's not you, it's the game :P. Replacing the CLC/RTS with BNE $87E3 should be enough to fix that up.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 25, 2019, 02:47:18 pm
Okay, really embarrassing question but I can't seem to open the file in either program you recommended. Asar seems to ONLY accept SNES roms and xkas plus seems to only accept dissembled code. Since you've said you've been using the former, I feel like I'm missing something incredibly obvious.

Edit: Also, I thought I was getting your explanation of Read Breakpoints. I thought to find Monster Length I'd place a Read Breakpoint for Slime which the Pointer points to as memory at B718 (to avoid possible confusion I'll just say that with Chicken Knife sharing his atlas script I thought I'd use the work you guys did in moving the Item and Monster names). I figured it'd have to be CPU memory because PPU memory doesn't let me put in B718 (I assume 3ff0 is the maximum?) And yet when I try to go in to a battle with a Slime, the Debugger doesn't trip anything. And yes I made sure it's enabled before you ask :p It's an easy thing to miss, so I wouldn't blame you if you asked.
Title: Re: General NES Hacking Questions
Post by: abw on April 25, 2019, 05:43:04 pm
Okay, really embarrassing question but I can't seem to open the file in either program you recommended. Asar seems to ONLY accept SNES roms and xkas plus seems to only accept dissembled code. Since you've said you've been using the former, I feel like I'm missing something incredibly obvious.

Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:
Quote from: test.asm
; Example NES 6502 ASM file: writes a small infinite loop.
; Put this file in the same directory as asar and execute it with e.g.
;   copy /Y nul test.bin
;   asar -nocheck test.asm test.bin
; After that, test.bin should contain 64 KB of #$00 followed by "A9 00 4C 00 80"

norom   ; stop Asar from trying to apply SNES memory mapping to this NES code
org $10010   ; set the ROM file insertion point to 0x10010
base $8000   ; set the starting RAM address to $8000

loop:
   LDA #$00
   JMP loop

Edit: Also, I thought I was getting your explanation of Read Breakpoints. I thought to find Monster Length I'd place a Read Breakpoint for Slime which the Pointer points to as memory at B718 (to avoid possible confusion I'll just say that with Chicken Knife sharing his atlas script I thought I'd use the work you guys did in moving the Item and Monster names). I figured it'd have to be CPU memory because PPU memory doesn't let me put in B718 (I assume 3ff0 is the maximum?) And yet when I try to go in to a battle with a Slime, the Debugger doesn't trip anything. And yes I made sure it's enabled before you ask :p It's an easy thing to miss, so I wouldn't blame you if you asked.
Yeah, unless you're specifically looking for graphics stuff, a CPU breakpoint is probably what you want. The original monster list was at 0x1B728 a.k.a. $06:$B718, but I moved my monster list to 0x1D050, a.k.a. $07:$9040, so if you're looking for Slime, that's where it'll be.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on April 27, 2019, 07:40:19 pm
Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:Yeah, unless you're specifically looking for graphics stuff, a CPU breakpoint is probably what you want. The original monster list was at 0x1B728 a.k.a. $06:$B718, but I moved my monster list to 0x1D050, a.k.a. $07:$9040, so if you're looking for Slime, that's where it'll be.

I guess I figured what was there at the pointer would still work. Thanks.

Well I got the game to stop and saw
 0F:F47B:8D A0 60  STA $60A0 = #$0B

And thanks to laserlambert's testing I know that the line 1 of monster names gets cut off at 11 letters, so I figure that's GOTTA be it right? Am I right in thinking maybe this ISN'T a hardcoded value but something that's loaded in memory? I also know that the second line for monsters is limited to 9 letters which is at least 1 letter short for my new monster names and even when trying to make a breakpoint based on the Monster Line 2 pointer, I got a break but couldn't find anythign resembling a 9 letter limit. I even went BACK to DW1 and tried to recreate the steps you did in finding the length limit of HEAL. made a breakpoint and had it stop when casting and found

01:A868:AE E2 64  LDX $64E2 = #$0F

Note this is from my hack, so i figured this would have to be the new 15 letter limit for spells. If so, why does this use LDX? And how did you turn that into the ROM address of $77E9? Or am I just not looking at the right thing at all?
Title: Re: General NES Hacking Questions
Post by: abw on April 28, 2019, 09:21:17 am
This sounds okay as far as it goes, but you haven't hit ROM yet, so you need to keep following the trail a bit further. Once you find the value you're looking for being read from somewhere in the $8000 - $FFFF range, then you can stop and convert the RAM address to a ROM address (/ get FCEUX to do it for you if you want).

0F:F47B:8D A0 60  STA $60A0 = #$0B
This shows that the game is about to store whatever the current value of A is to $60A0 (which was #$0B just before that instruction executed), so you'll need to find out where $60A0 became #$0B in the first place.

01:A868:AE E2 64  LDX $64E2 = #$0F
Similarly, this shows that the game is loading X with the value of $64E2, which happens to be #$0F; you'll need to find out how $64E2 became #$0F.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on May 04, 2019, 06:40:23 pm
Asar assumes a SNES ROM and memory model by default, but you can get it to work with NES ROMs by flipping a couple of switches. Since it doesn't come with much in the way of examples, give this a try:

Sorry, I did this and ran it, it made the bin file but I'm still getting the "Not an SNES ROM Error" You might have to dumb it down even more for me. (9_6)

This sounds okay as far as it goes, but you haven't hit ROM yet, so you need to keep following the trail a bit further. Once you find the value you're looking for being read from somewhere in the $8000 - $FFFF range, then you can stop and convert the RAM address to a ROM address (/ get FCEUX to do it for you if you want).
This shows that the game is about to store whatever the current value of A is to $60A0 (which was #$0B just before that instruction executed), so you'll need to find out where $60A0 became #$0B in the first place.
Similarly, this shows that the game is loading X with the value of $64E2, which happens to be #$0F; you'll need to find out how $64E2 became #$0F.

Okay so I used trace logger to log the data from the world map to the start of a battle to the breakpoint. Still not seeing what I want. I tried to put an Execution Breakpoint on 60A0, but I don't think I'm doing it right. When doubleclicking in the debugger it just gives me "K==#00" as the "condition" and when I try to change it I get an invalid condition.

Really sorry about this. It's hard not to feel like that guy who was trying to translate DW1 into Spanish. It's frustrating to feel so clueless, but this is uncharted territory for me and I'm determined to pick up SOMETHING new for later games.

On another note, the Assembly guide you posted is really handy, thanks! It's nice to know what all those 3 letter terms mean.
Title: Re: General NES Hacking Questions
Post by: abw on May 15, 2019, 06:06:35 pm
Sorry for the delay in responding, I've been offline for the past couple of weeks!

Sorry, I did this and ran it, it made the bin file but I'm still getting the "Not an SNES ROM Error" You might have to dumb it down even more for me. (9_6)
I'm not sure how much further down I can go :P. Go to the directory containing Asar, copy the sample ASM I provided into a new file named test.asm, make an empty file named test.bin, and then open a command prompt in that directory and run "asar -nocheck test.asm test.bin". Works like a charm for me.

Okay so I used trace logger to log the data from the world map to the start of a battle to the breakpoint. Still not seeing what I want. I tried to put an Execution Breakpoint on 60A0, but I don't think I'm doing it right. When doubleclicking in the debugger it just gives me "K==#00" as the "condition" and when I try to change it I get an invalid condition.
For $60A0, you'd want a write breakpoint since you're looking for places where the game writes #$0B to $60A0. Execute breakpoints fire when the code at the address you set the breakpoint for gets executed (e.g. the F47B in "0F:F47B:8D A0 60  STA $60A0 = #$0B") and read/write breakpoints fire when the address you set the breakpoint for gets modified by some code (e.g. 60A0 is being written to in "0F:F47B:8D A0 60  STA $60A0 = #$0B").

With the trace log, you don't necessarily need to set any breakpoints; they just help to reduce the size of the log file you need to look through. If I do the same thing as you with an unaltered ROM, making a trace log from the world map to the start of a battle, it's a huge file but I can search it for $B718 (the start of the monster list) to get:
Code: [Select]
        $F422:A0 00     LDY #$00                                     A:18 X:00 Y:00 S:F7 P:nvUBdIzc
        $F424:AE A0 60  LDX $60A0 = #$0B                             A:18 X:00 Y:00 S:F7 P:nvUBdIZc
        $F427:B1 57     LDA ($57),Y @ $B718 = #$36                   A:18 X:0B Y:00 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $010A = #$5F                   A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:36 X:0B Y:00 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:36 X:0B Y:01 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:36 X:0A Y:01 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B719 = #$15                   A:36 X:0A Y:01 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0109 = #$5F                   A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:15 X:0A Y:01 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:15 X:0A Y:02 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:15 X:09 Y:02 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71A = #$12                   A:15 X:09 Y:02 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0108 = #$5F                   A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:12 X:09 Y:02 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:12 X:09 Y:03 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:12 X:08 Y:03 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71B = #$16                   A:12 X:08 Y:03 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0107 = #$5F                   A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:16 X:08 Y:03 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:16 X:08 Y:04 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:16 X:07 Y:04 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71C = #$0E                   A:16 X:07 Y:04 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F42D:9D FF 00  STA $00FF,X @ $0106 = #$5F                   A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F430:C8        INY                                          A:0E X:07 Y:04 S:F7 P:nvUBdIzc
        $F431:CA        DEX                                          A:0E X:07 Y:05 S:F7 P:nvUBdIzc
        $F432:D0 F3     BNE $F427                                    A:0E X:06 Y:05 S:F7 P:nvUBdIzc
        $F427:B1 57     LDA ($57),Y @ $B71D = #$FF                   A:0E X:06 Y:05 S:F7 P:nvUBdIzc
        $F429:C9 FF     CMP #$FF                                     A:FF X:06 Y:05 S:F7 P:NvUBdIzc
        $F42B:F0 07     BEQ $F434                                    A:FF X:06 Y:05 S:F7 P:nvUBdIZC
        $F434:60        RTS (from $F3FE) --------------------------- A:FF X:06 Y:05 S:F7 P:nvUBdIZC
which shows that the game is copying data from $B718-$B71D to $010A-$0106 (stored backwards) until it reads a #$FF (monster name end token) or X reaches #$00, and that X was set based on $60A0.

Spoiler:
As a side note, a little further down, you'll see the game copies the monster name from $00FF,X to $6119,Y, where it will eventually get used by the [name] control code:
Code: [Select]
      $FCE8:AE A0 60  LDX $60A0 = #$0B                             A:FF X:06 Y:00 S:F9 P:nvUBdIZc
      $FCEB:BD FF 00  LDA $00FF,X @ $010A = #$36                   A:FF X:0B Y:00 S:F9 P:nvUBdIzc
      $FCEE:99 19 61  STA $6119,Y @ $6119 = #$25                   A:36 X:0B Y:00 S:F9 P:nvUBdIzc
      $FCF1:C8        INY                                          A:36 X:0B Y:00 S:F9 P:nvUBdIzc
      $FCF2:CA        DEX                                          A:36 X:0B Y:01 S:F9 P:nvUBdIzc
      $FCF3:D0 F6     BNE $FCEB                                    A:36 X:0A Y:01 S:F9 P:nvUBdIzc
...

Searching backwards in the trace log for $60A0, the very first result is this:
Code: [Select]
     $FC92:A9 0B     LDA #$0B                                     A:00 X:00 Y:00 S:FA P:nvUBdIZc
     $FC94:8D A0 60  STA $60A0 = #$01                             A:0B X:00 Y:00 S:FA P:nvUBdIzc
So $60A0 got its value from A, and A got its value set based on $FC93 (the #$0B part of "LDA #$0B"), which unlike $60A0 comes from ROM i.e. 0x3FCA3. Ta-da!

With that as a guide, see if you can find where the maximum length of the second "line" of monster names in the main dialogue box is set (hint: it's #$09 and it's not too far away from where the maximum length of the first "line" is set), and then see if you can track down where the lengths for each of the two lines in the monster menu list get set (hint: same values as the dialogue box lengths, but set in a different area of the code; they'll still be in your trace log, though).

Really sorry about this. It's hard not to feel like that guy who was trying to translate DW1 into Spanish. It's frustrating to feel so clueless, but this is uncharted territory for me and I'm determined to pick up SOMETHING new for later games.

On another note, the Assembly guide you posted is really handy, thanks! It's nice to know what all those 3 letter terms mean.
Yeah, if you're not used to this kind of thing, it can take a while to really sink in. Just keep at it and you'll get the hang of it sooner or later!
Title: Re: General NES Hacking Questions
Post by: Choppasmith on May 18, 2019, 07:11:35 am
Hey! Glad to see your back and that you're okay! I was genuinely worried for a bit there that something bad might've happened that would've taken you out of the picture. While I'm sure I could've found help, you're a cool guy and it would've been a bummer to not be able to finish this while learning how to do it on my own. But anyway...

I'm not sure how much further down I can go :P. Go to the directory containing Asar, copy the sample ASM I provided into a new file named test.asm, make an empty file named test.bin, and then open a command prompt in that directory and run "asar -nocheck test.asm test.bin". Works like a charm for me.

I wonder if it's a Windows thing, are you on 10? When I try to run that very same command from the command line, it seems to work for a second but just takes me back to the command line with a modified test.bin and trying to run asar again just gives me the usual.


Searching backwards in the trace log for $60A0, the very first result is this:
Code: [Select]
     $FC92:A9 0B     LDA #$0B                                     A:00 X:00 Y:00 S:FA P:nvUBdIZc
     $FC94:8D A0 60  STA $60A0 = #$01                             A:0B X:00 Y:00 S:FA P:nvUBdIzc
So $60A0 got its value from A, and A got its value set based on $FC93 (the #$0B part of "LDA #$0B"), which unlike $60A0 comes from ROM i.e. 0x3FCA3. Ta-da!

With that as a guide, see if you can find where the maximum length of the second "line" of monster names in the main dialogue box is set (hint: it's #$09 and it's not too far away from where the maximum length of the first "line" is set), and then see if you can track down where the lengths for each of the two lines in the monster menu list get set (hint: same values as the dialogue box lengths, but set in a different area of the code; they'll still be in your trace log, though).

So on one hand I DID find the second line monster value of 9 at 3FCBF, though I'm not sure how you turned FC93 to 3FCA3. I mean yeah you added 30010 but where did THAT come from? It doesn't quite match up with what you were talking about RAM to ROM addresses on the last page.

And I made a honest effort, but I can't seem to find what you're talking about for the Monster List window. I DID find the subroutine FCE8 in my trace log and while trying to understand it still makes my eyes go @_@ I can understand enough that there's two sections concerning whether or not the Monster needs that second line printed in the window. And I can see that it loads the value as X as opposed to A in the main dialog window. But can't seem to find anything in my log about the value being stored in 60A0.
Title: Re: General NES Hacking Questions
Post by: abw on May 18, 2019, 12:57:54 pm
Hey! Glad to see your back and that you're okay! I was genuinely worried for a bit there that something bad might've happened that would've taken you out of the picture. While I'm sure I could've found help, you're a cool guy and it would've been a bummer to not be able to finish this while learning how to do it on my own. But anyway...
Yeah, every now and then I go offline for a couple of weeks for IRL stuff, though one time it was for an entire year!

I wonder if it's a Windows thing, are you on 10? When I try to run that very same command from the command line, it seems to work for a second but just takes me back to the command line with a modified test.bin and trying to run asar again just gives me the usual.
I'm actually on Windows 7 (cuz eww 8 and 10), but if you're getting a modified test.bin, then it sounds like Asar is working. Try changing test.asm and see if you get a different test.bin.

So on one hand I DID find the second line monster value of 9 at 3FCBF,
Nice job!

though I'm not sure how you turned FC93 to 3FCA3. I mean yeah you added 30010 but where did THAT come from? It doesn't quite match up with what you were talking about RAM to ROM addresses on the last page.
If the Trace Logger included bank number, $FC93 would show up as $0F:$FC93, and $0F * $4000 - $8000 - $01 * $4000 + $FC93 + $10 = $3FCA3 (i.e. <ROM bank number> * <bank size> - <base RAM-to-ROM offset> - <RAM bank number> * <bank size> + <RAM address> + <iNES header size>). Or find $FC93 in the Hex Editor -> right click -> Go Here In ROM File.

And I made a honest effort, but I can't seem to find what you're talking about for the Monster List window. I DID find the subroutine FCE8 in my trace log and while trying to understand it still makes my eyes go @_@
Ah, $FCE8's not so bad :P. When you're trying to wrap your head around a block of code, remember that the Debugger and Trace Logger give you two different views of the same thing; sometimes it's easier to understand what's going on when looking at one instead of the other. Here's the basic code you'll see in the Debugger:
Code: [Select]
0F:FCE8:AE A0 60 LDX $60A0
0F:FCEB:BD FF 00 LDA $00FF,X
0F:FCEE:99 19 61 STA $6119,Y
0F:FCF1:C8      INY
0F:FCF2:CA      DEX
0F:FCF3:D0 F6    BNE $FCEB
0F:FCF5:60      RTS

and here's a commented version:
Spoiler:
Code: [Select]
; copy $60A0 bytes of data from $00FF,X to $6119,Y
; X is used as a read index, Y as a write index
; data gets copied in reverse order
; IN:
; A/X/C = irrelevant
; Y = current write index
; OUT:
; A = last byte copied (but calling code doesn't care)
; X = 0
; Y = current write index; this is important since the calling code needs to remember the write index from the first segment when dealing with the second segment
; C = unchanged
; control flow target (from $FC9D, $FCBA)
0x03FCF8|$0F:$FCE8:AE A0 60 LDX $60A0 ; initialize the read index to the value of $60A0
; control flow target (from $FCF3)
0x03FCFB|$0F:$FCEB:BD FF 00 LDA $00FF,X ; read data from $00FF,X
0x03FCFE|$0F:$FCEE:99 19 61 STA $6119,Y ; write data to $6119,Y
0x03FD01|$0F:$FCF1:C8      INY ; increment write index
0x03FD02|$0F:$FCF2:CA      DEX ; decrement read index
0x03FD03|$0F:$FCF3:D0 F6    BNE $FCEB ; if the read index is not 0, loop back to $FCEB
0x03FD05|$0F:$FCF5:60      RTS ; otherwise the read index is 0, so we're done

I can understand enough that there's two sections concerning whether or not the Monster needs that second line printed in the window. And I can see that it loads the value as X as opposed to A in the main dialog window. But can't seem to find anything in my log about the value being stored in 60A0.
If you keep looking for reads on $B718, you should notice that the game scans through the monster name list a few times while starting a battle; the last one is for the monster list menu (you can also easily isolate this one by starting a trace log just before pressing FIGHT, as the monster menu gets redrawn after that point). At the spot where the breakpoint fires, you'll be in the same block of code as for the main dialogue window, but coming from a different place (the stack will show something like FA,F3,C5,EF,..., which means the last JSR before you got to this code ended at $F3FA [so started at $F3F8], and the JSR before that ended at $EFC5 [$EFC3]). Searching backwards for $60A0 from there should quickly get you to:
Code: [Select]
              $EFA9:A9 0B     LDA #$0B                                     A:00 X:18 Y:07 S:F1 P:nvUBdIZc
              $EFAB:8D A0 60  STA $60A0 = #$00                             A:0B X:18 Y:07 S:F1 P:nvUBdIzc
and searching forwards will eventually (after a long series of other uses for $60A0) get you the maximum length of the second line too (or if you look in the Debugger, the code for handling the second line of monster names is only a few lines of ASM away from the code for handling the first line).
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 13, 2019, 09:43:40 pm
Okay, so first of all, sorry for the delay. I mean yeah I work but a big reason is I just got nice CPU upgrade so I've been finally able to play DQXI among other things, and I think the last time I looked at this (about a week ago) my brain was like "Nope, not today..." but the Smash reveal made me go... "yeah I need to get back to this." If anything I'm just eager to get to 3 and finish the NES Erdrick trilogy at the very very least.

Anyway. Good news is I not only figured out the monster name menu length (though to your credit, you made it pretty easy in your last post) but I ALSO got the spell length for dialog.

(https://i.imgur.com/CC0s7l2.png)
(yeah I swapped out Heal for Holy Protection as a quick way to test)

I'm not sure if I want to mess with the menu length and keep it abbreviated like I did DW1 (because I know unlike DW1 not only the spell menus are done differently, but there are many more long spell names too). Otherwise that's all taken care of.

Bad news it, and I'm so sorry, but the ASM stuff is still stumping me. More, the asar usage than anything else.
I'm actually on Windows 7 (cuz eww 8 and 10), but if you're getting a modified test.bin, then it sounds like Asar is working. Try changing test.asm and see if you get a different test.bin.

Is the bin file supposed to be viewable in text form? Because opening it up in Notepad+ just gives me junk so I have no idea what to look for.
Title: Re: General NES Hacking Questions
Post by: abw on June 14, 2019, 09:21:41 am
Okay, so first of all, sorry for the delay. I mean yeah I work but a big reason is I just got nice CPU upgrade so I've been finally able to play DQXI among other things, and I think the last time I looked at this (about a week ago) my brain was like "Nope, not today..." but the Smash reveal made me go... "yeah I need to get back to this." If anything I'm just eager to get to 3 and finish the NES Erdrick trilogy at the very very least.
Heh, I hear DQXI is a leading cause of delay among DQ NES hackers ;).

Anyway. Good news is I not only figured out the monster name menu length (though to your credit, you made it pretty easy in your last post) but I ALSO got the spell length for dialog.
Congrats! It sounds like you must be getting close to finishing with this game - what's still left?

Bad news it, and I'm so sorry, but the ASM stuff is still stumping me. More, the asar usage than anything else.
Is the bin file supposed to be viewable in text form? Because opening it up in Notepad+ just gives me junk so I have no idea what to look for.
Just as viewable as any other ROM, i.e. not very :P. Like I said earlier (http://www.romhacking.net/forum/index.php?topic=27053.msg374493#msg374493), that sample ASM file should generate a file with 64 KB of zero bytes followed by the bytes "A9 00 4C 00 80", so open it up in a hex editor, scroll to the very bottom, and if you see those bytes, then it's working.

For real world usage, you'd want to adjust the org/base values to the ROM/RAM addresses you want to write to, replace the useless infinite loop ASM I concocted for the sample with whatever code you actually want to insert, and then run it against a real ROM instead of an empty file.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 21, 2019, 09:45:23 pm
Heh, I hear DQXI is a leading cause of delay among DQ NES hackers ;).

The funny thing about playing XI while working on II is spotting the references. The Puff Puff girl in Gondolia says the same thing as the girl in Lianport/Rippleport word for word from the mobile version. That was a real A-ha moment for me and I'm hoping doing these script ports more people will be able to see that.

Quote
Congrats! It sounds like you must be getting close to finishing with this game - what's still left?

Honestly outside fixing the buggy  plural s and of course the monster name ASM, just need to edit the uncompressed Prologue text and get the new graphics from Chicken Knife's hack I'll be done. I have the new menus ready to go for insertion.

Speaking of A-Ha moment, I finally get how asar works. Man I feel dumb. I was expecting some kind of fancy interface like PS2Dis or something where you can just edit the lines of code as you go.

Am I right in thinking I could just copy the Pluralization rules from the Wiki into the test.asm (replace the loop code) file but change org to $01C805 (code where it starts to check monster count) and base to $4000 (?) and then just "revise" the code to my liking and then run "asar -nocheck test.asm (DW2 rom)" Is that what you do?
Title: Re: General NES Hacking Questions
Post by: abw on June 22, 2019, 04:46:25 pm
Speaking of A-Ha moment, I finally get how asar works. Man I feel dumb. I was expecting some kind of fancy interface like PS2Dis or something where you can just edit the lines of code as you go.
Ah, yeah, it's not that fancy, alas. Quite possibly there are better assembler options out there; Asar is just the one I'm used to and it hasn't yet irritated me enough to go looking for alternatives.

Am I right in thinking I could just copy the Pluralization rules from the Wiki into the test.asm (replace the loop code) file but change org to $01C805 (code where it starts to check monster count) and base to $4000 (?) and then just "revise" the code to my liking and then run "asar -nocheck test.asm (DW2 rom)" Is that what you do?
Sort of. You'll want to set the base value (RAM address) to $87F5 and then trim out the non-ASM ROM address, RAM address, and assembled code bytes from the wiki, leaving just the opcodes and data bytes (and probably the comments, because why not?). Unless you can manage to keep the byte counts between control flow targets identical to the original code or enjoy manually counting bytes and updating lots of pointers every time you make a change, you will also want to convert the control flow addresses into labels and use those, so it'll look something like this:
Code: [Select]
LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$18 ; "o"
BEQ o_handler ; -ngo -> -ngo, -o -> -os
CMP #$0F ; "f"
BEQ f_handler ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ y_handler ; -y -> -ies
CMP #$12 ; "i"
BEQ i_handler ; -i -> -ies
CMP #$1C ; "s"
BEQ s_handler ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ h_handler ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ n_handler ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0E ; "e"
BEQ e_handler ; -mouse -> -mice, -Mouse -> -Mice, -e -> es
CMP #$0D ; "d"
BEQ d_handler ; -dead -> -dead, -d -> -ds
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
done:
SEC
RTS

o_handler:
...

Keep in mind that other unrelated code starts at $88E4, so make sure not to overwrite that; fortunately bank 7 is mostly empty, so if you need more space, you can JMP $8C13 and keep going from there.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 25, 2019, 01:31:06 pm
Okay, so, here it is!

NOTE: I'm keeping addresses and bytes here for referential purposes. I'm well aware from your last post that these should be removed for insertion.



Code: [Select]
LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$0F ; "f"
BEQ $885E ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ $8874 ; -y -> -ies
CMP #$12 ; "i"
BEQ $8863 ; -i -> -ies
CMP #$1C ; "s"
BEQ $8835 ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ $886B ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ $889C ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ e_handler ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ d_handler ; Man O' War -> Men O' War
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
done:
SEC
RTS

; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
$8835:  LDA $60F0,X ; read second-last letter of monster name
$8838:  CMP #$0E ; "e"
$883A:  BEQ $8830 ; if -es, singular=plural
$883C:  CMP #$0A ; "a"
$883F:  BEQ $8830 ; if -as, singular=plural
$8841:  CMP #$19 ; "p"
$8843:  BEQ $8830 ; if -ps, singular=plural
$8845:  LDA $60EF,X ; read third-last letter of monster name
$8848:  CMP #$10 ; "g"
$884A:  BEQ $8855   ; Shortcut here, if g_s (Magus) replace us with i
$884C:  BNE $885D ; If not either Cyclops or Atlas (or Magus), add -es

; append "es" to monster name
$884E:E8      INX
$884F:A9 0E    LDA #$0E ; "e"
$8851:9D F1 60 STA $60F1,X ; append "e" to monster name
$8854:D0 BE    BNE $8823 ; append "s" to monster name; note that this branch is always taken



$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS

; -f pluralization handler: -f -> -ves (no need to have this, but if there's space, might keep it for possible hacks)
$885E:A9 1F    LDA #$1F ; "v"
$8860:4C 6C 88 JMP $886C ; replace "f" with "v" then append "es"


; -i pluralization handler: -i -> -ies (same as above)
$8863:A9 12    LDA #$12 ; "i"
; unused control flow target (from $8867)
$8865:9D F1 60 STA $60F1,X ; replace final letter with "i"
$8868:4C 5D 88 JMP $884E ; append "es"

; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
$886B:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
$886E:C9 1C    CMP #$1C ; "s"
$8870:F0 E0    BEQ $885D ; if -sh, append "es"
$8872:D0 A4    BNE $8823 ; else, append "s"

; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
$8874 LDA $60F0,X ; read second-last letter of monster name
$8877 CMP #$18 ; "o"
$8879 BEQ $8823 ; if -oy, append "s"
$887B CMP #$18 ; "r"
$887D BEQ $8830       ; if -ry, no change
$887F BNE $886C       ; otherwise replace y with -ies

; -a pluralization handler: needed for madusa -> madusae
$8881 LDA $60F0,X ; read second-last letter of monster name
$8884 CMP #$1C ; "s"
$8886 BEQ             ; if -sa add "e"
$8888 BNE $8823       ; if not -sa append "s"

;adding "e"
$888A INX
$888B LDA #$0E ; "e"
$888D STA $60F1,X ; append "e" to monster name
$8890 JMP 8829

; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?

; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
$889C:BD F0 60 LDA $60F0,X ; read second-last letter of monster name
$889F:C9 0A    CMP #$0A ; "a"
$88A1:D0 9D    BNE $8823 ; if not -an, append "s"
$88A3:BD EF 60 LDA $60EF,X ; read third-last letter of monster name
$88A6:C9 16    CMP #$16 ; "m"
$88A8:F0 04    BEQ $8891 ; -man -> -men
$88AA:C9 30    CMP #$30 ; "M"
$88AC:D0 92    BNE $8823 ; if not -Man, append "s"
; control flow target (from $888B)
$88AE:A9 0E    LDA #$0E ; "e"
$88B0:9D F0 60 STA $60F0,X ; replace second-last letter of monster name
; control flow target (from $88DF)
$88B3:38      SEC
$88B4:60      RTS

Not neading the -dead or -mouse codes freed up a bunch of space. So much so, I thought I'd just keep some of the extra codes I didn't necessarily need like f-ves or i-ies in case someone wants to use my script port as a base for a hack that needs new monster names.

I just had a couple of confused points. I'm not quite sure how the replace text code works so I'm a little stumped at changing the old -us to -ii code to just -us to -i (Magus to Magi)

Code: [Select]
$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS

and I just wanted to run my Man O' War to Men O' War code by you

Code: [Select]
; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
Title: Re: General NES Hacking Questions
Post by: abw on June 25, 2019, 08:39:22 pm
I just had a couple of confused points. I'm not quite sure how the replace text code works so I'm a little stumped at changing the old -us to -ii code to just -us to -i (Magus to Magi)

Code: [Select]
$8856:  STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
$8859:  STA $60F1,X
$885C:  SEC
$885D: RTS
There's nothing magic going on here; try setting an execute breakpoint at $8853, getting into a fight with some Magic Vampirii (or temporarily rename e.g. Slime to Slrus and get into a fight with some of those), and step through the code, watching what happens around $60F1 in the Hex Editor.

Does this make it any clearer?
Spoiler:
Code: [Select]
0x01C863|$07:$8853:A9 12    LDA #$12 ; "i"
0x01C865|$07:$8855:9D F0 60 STA $60F0,X ; replace the second-last letter ("u") with "i"
0x01C868|$07:$8858:9D F1 60 STA $60F1,X ; replace the last letter ("s") with "i"
0x01C86B|$07:$885B:38      SEC ; let calling code know to read a #$FA-terminated string from $60F1 instead of a single byte from A
0x01C86C|$07:$885C:60      RTS

and I just wanted to run my Man O' War to Men O' War code by you

Code: [Select]
; -r pluralization handler: needed for man o' war
$8893 LDA $60E9,X     ; read ninth from end letter of monster name
$8895 CMP #$0A ; "a"
$8897 BNE $8823       ; if not "a" append "s"
$8899 STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
Close, but as it's written that code will still end pluralizing Man O' War to Man O' War, since it takes the ninth-last letter and writes it to the ninth-last letter; try doing "LDA #$0E" before the "STA $60E9,X".

Once you've finished labelling the control flow targets, try assembling and inserting it! Your code is definitely shorter than the original (and way shorter than mine for Latin!), so this probably won't be an issue, but just in case, keep in mind that all those branches use a 1-byte signed displacement from the end of the operand, which basically means if you need to move farther than -127/+128 bytes, you'll have to use a JMP instead. Assuming that works, it's time to test it out - here's hoping the game doesn't crash on you :D.
Title: Re: General NES Hacking Questions
Post by: Choppasmith on June 26, 2019, 09:42:21 pm
There's nothing magic going on here; try setting an execute breakpoint at $8853, getting into a fight with some Magic Vampirii (or temporarily rename e.g. Slime to Slrus and get into a fight with some of those), and step through the code, watching what happens around $60F1 in the Hex Editor.

Does this make it any clearer?
Spoiler:
Code: [Select]
0x01C863|$07:$8853:A9 12    LDA #$12 ; "i"
0x01C865|$07:$8855:9D F0 60 STA $60F0,X ; replace the second-last letter ("u") with "i"
0x01C868|$07:$8858:9D F1 60 STA $60F1,X ; replace the last letter ("s") with "i"
0x01C86B|$07:$885B:38      SEC ; let calling code know to read a #$FA-terminated string from $60F1 instead of a single byte from A
0x01C86C|$07:$885C:60      RTS
Close, but as it's written that code will still end pluralizing Man O' War to Man O' War, since it takes the ninth-last letter and writes it to the ninth-last letter; try doing "LDA #$0E" before the "STA $60E9,X".

Once you've finished labelling the control flow targets, try assembling and inserting it! Your code is definitely shorter than the original (and way shorter than mine for Latin!), so this probably won't be an issue, but just in case, keep in mind that all those branches use a 1-byte signed displacement from the end of the operand, which basically means if you need to move farther than -127/+128 bytes, you'll have to use a JMP instead. Assuming that works, it's time to test it out - here's hoping the game doesn't crash on you :D.

First of all, really smacking my forehead at the obvious lack of a LDA command for some of these. I think I forgot to copy that line over for the Wiki but also, it's really easy for me to overlook something obvious.  :crazy:

Not sure what I did, but my first time seemed to work okay aside from a couple of mistakes, but now after checking my Jump/Branch values I seem to have made it worse. A lot of my test names tend to show one or two letters like "Three y appeared" instead of two "Iron Dragonfry appeared"

(https://i.imgur.com/7AxRlHp.png)

I was able to parse your example asm script above, filling in the byte values appropriately, but I did change stuff like done: and add_s: assuming those were meant to be comments. Did I mess somethign up there?

This is my current script as it's being inserted (also yeah I should have known base meant was a RAM address. Here I thought it was some kind of hardware value that needed to be changed to something signifying NES hardware instead of SNES hardware. Live and learn.

Code: [Select]
norom   ; stop Asar from trying to apply SNES memory mapping to this NES code
org $01C805   ; set the ROM file insertion point to 0x10010
base $87F5   ; set the starting RAM address to $8000

LDY $8F ; number of monsters in the current group
DEY
BEQ $882C ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$0F ; "f"
BEQ $8856 ; -f -> -ves (not used)
CMP #$22 ; "y"
BEQ $886C ; -y -> -ies
CMP #$12 ; "i"
BEQ $885B ; -i -> -ies
CMP #$1C ; "s"
BEQ $882E ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ $8863 ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ $8899 ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ $887A ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ $888C ; Man O' War -> Men O' War
; control flow target (from $883A, $8841, $8863, $8884, $888F, $889D, $88E1)
; append "s" to monster name
; default pluralization if not handled above
; add_s:
INX
LDA #$1C ; "s"
STA $60F1,X ; append "s" to monster name
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
; control flow target (from $87F8, $8843)
; done:
SEC
RTS

; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
LDA $60F0,X ; read second-last letter of monster name
CMP #$0E ; "e"
BEQ $882C ; if -es, singular=plural
CMP #$0A ; "a"
BEQ $882C ; if -as, singular=plural
CMP #$19 ; "p"
BEQ $882C ; if -ps, singular=plural
LDA $60EF,X ; read third-last letter of monster name
CMP #$10 ; "g"
BEQ $884E   ; Shortcut here, if g_s (Magus) replace us with i
BNE $8846 ; If not either Cyclops or Atlas (or Magus), add -es

; append "es" to monster name
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
BNE $881F ; append "s" to monster name; note that this branch is always taken

; replace us with i
LDA #$12    ; "i"
STA $60F0,X ; replace -us with -ii (how do I change this to replace -us to -i?)
SEC
RTS

; -f pluralization handler: -f -> -ves (no need to have this, but if there's space, might keep it for possible hacks)
LDA #$1F ; "v"
JMP $885D ; replace "f" with "v" then append "es"


; -i pluralization handler: -i -> -ies (same as above)
LDA #$12 ; "i"
; unused control flow target (from $8867)
STA $60F1,X ; replace final letter with "i"
JMP $8846 ; append "es"

; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ $8846 ; if -sh, append "es"
BNE $881F ; else, append "s"

; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
LDA $60F0,X ; read second-last letter of monster name
CMP #$18 ; "o"
BEQ $881F ; if -oy, append "s"
CMP #$1B ; "r"
BEQ $882C       ; if -ry, no change
BNE $885D      ; otherwise replace y with -ies

; -a pluralization handler: needed for madusa -> madusae
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ $8883   ; if -sa add "e"
BNE $881F       ; if not -sa append "s"

;adding "e"
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
JMP $8825

; -r pluralization handler: needed for man o' war
LDA $60E9,X     ; read ninth from end letter of monster name
CMP #$0A       ; "a"
BNE $881F       ; if not "a" append "s"
LDA #$0E        ; "e"
STA $60E9,X     ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?

; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
LDA $60F0,X ; read second-last letter of monster name
CMP #$0A ; "a"
BNE $881F ; if not -an, append "s"
LDA $60EF,X ; read third-last letter of monster name
CMP #$16 ; "m"
BEQ $88AB   ; -man -> -men
CMP #$30 ; "M"
BNE $881F ; if not -Man, append "s"
; control flow target (from $888B)
LDA #$0E ; "e"
STA $60F0,X ; replace second-last letter of monster name
; control flow target (from $88DF)
SEC
RTS
Title: Re: General NES Hacking Questions
Post by: abw on June 27, 2019, 04:56:07 pm
First of all, really smacking my forehead at the obvious lack of a LDA command for some of these. I think I forgot to copy that line over for the Wiki but also, it's really easy for me to overlook something obvious.  :crazy:
Yeah, you'll probably make all sorts of little mistakes on your first few attempts. It's especially bad if you're like me and have a bad habit of thinking one thing but typing another :P.

Not sure what I did, but my first time seemed to work okay aside from a couple of mistakes, but now after checking my Jump/Branch values I seem to have made it worse. A lot of my test names tend to show one or two letters like "Three y appeared" instead of two "Iron Dragonfry appeared"
When writing ASM, labels are your friends, they get sad if you exclude them :P. It's basically the same idea as pointers when inserting a script: let the computer handle all the boring tedious address calculations for you; that's what computers are good at. Additionally, if you check the assembled code, branches are probably not getting assembled the way you think they are. E.g. that "BEQ $886C" actually ends up as "F0 6C", i.e. branch ahead $2C bytes, or "BEQ $8873", which happens to be right in the middle of "CMP #$1B", which is almost never a good idea, especially since 1B isn't a valid 6502 opcode. Labels don't have that problem.

Try something like this instead (not really tested):
Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $01C805 ; set the ROM file insertion point
base $87F5 ; set the starting RAM address

LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$22 ; "y"
BEQ _y ; -y -> -ies
CMP #$12 ; "i"
BEQ _i ; -i -> -ies
CMP #$1C ; "s"
BEQ _s ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ _h ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ _n ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ _a ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ _r ; Man O' War -> Men O' War
; append "s" to monster name
; default pluralization if not handled above
append_s:
INX
LDA #$1C ; "s"
set_final_letter:
STA $60F1,X
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
done:
SEC
RTS

_s:
; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
LDA $60F0,X ; read second-last letter of monster name
CMP #$0E ; "e"
BEQ done ; if -es, singular=plural
CMP #$0A ; "a"
BEQ done ; if -as, singular=plural
CMP #$19 ; "p"
BEQ done ; if -ps, singular=plural
LDA $60EF,X ; read third-last letter of monster name
CMP #$10 ; "g"
BEQ _us ; Shortcut here, if g_s (Magus) replace us with i
; If not either Cyclops or Atlas (or Magus), add -es
append_es:
; append "es" to monster name
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
BNE append_s ; append "s" to monster name; note that this branch is always taken

_us:
; replace "us" with "i"
DEX ; shorten string by 1 letter
LDA #$12 ; "i"
BNE set_final_letter

_i:
; -i pluralization handler: -i -> -ies
LDA #$12 ; "i"
STA $60F1,X ; replace final letter with "i"
BNE append_es ; append "es"

_h:
; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ append_es ; if -sh, append "es"
BNE append_s ; else, append "s"

_y:
; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
LDA $60F0,X ; read second-last letter of monster name
CMP #$18 ; "o"
BEQ append_s ; if -oy, append "s"
CMP #$1B ; "r"
BEQ done ; if -ry, no change
BNE _i ; otherwise replace y with -ies

_a:
; -a pluralization handler: needed for madusa -> madusae
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ append_e ; if -sa add "e"
BNE append_s ; if not -sa append "s"

append_e:
; adding "e"
INX
LDA #$0E ; "e"
BNE set_final_letter

_r:
; -r pluralization handler: needed for man o' war
LDA $60E9,X ; read ninth from end letter of monster name
CMP #$0A ; "a"
BNE append_s ; if not "a" append "s"
LDA #$0E ; "e"
STA $60E9,X ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
BNE done

_n:
; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
LDA $60F0,X ; read second-last letter of monster name
CMP #$0A ; "a"
BNE append_s ; if not -an, append "s"
LDA $60EF,X ; read third-last letter of monster name
CMP #$16 ; "m"
BEQ update_e ; -man -> -men
CMP #$30 ; "M"
BNE append_s ; if not -Man, append "s"
update_e:
LDA #$0E ; "e"
STA $60F0,X ; replace second-last letter of monster name
SEC
RTS
Title: Re: General NES Hacking Questions
Post by: Choppasmith on July 05, 2019, 09:35:04 pm
Yeah, you'll probably make all sorts of little mistakes on your first few attempts. It's especially bad if you're like me and have a bad habit of thinking one thing but typing another :P.
When writing ASM, labels are your friends, they get sad if you exclude them :P. It's basically the same idea as pointers when inserting a script: let the computer handle all the boring tedious address calculations for you; that's what computers are good at. Additionally, if you check the assembled code, branches are probably not getting assembled the way you think they are. E.g. that "BEQ $886C" actually ends up as "F0 6C", i.e. branch ahead $2C bytes, or "BEQ $8873", which happens to be right in the middle of "CMP #$1B", which is almost never a good idea, especially since 1B isn't a valid 6502 opcode. Labels don't have that problem.

Try something like this instead (not really tested):
Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $01C805 ; set the ROM file insertion point
base $87F5 ; set the starting RAM address

LDY $8F ; number of monsters in the current group
DEY
BEQ done ; if only 1 monster, then no need to pluralize, so we're done
DEX ; back up to [end-FA]
DEX ; back up to final letter of monster name
LDA $60F1,X ; read final letter of monster name
CMP #$22 ; "y"
BEQ _y ; -y -> -ies
CMP #$12 ; "i"
BEQ _i ; -i -> -ies
CMP #$1C ; "s"
BEQ _s ; -rus -> -rii, -s -> -ses
CMP #$11 ; "h"
BEQ _h ; -ch -> -ches, -sh -> -shes, -h -> -hs
CMP #$17 ; "n"
BEQ _n ; -man -> -men, -Man -> -Men, -n -> -ns
CMP #$0A ; "a"
BEQ _a ; Madusa -> Madusae
CMP #$1B ; "r"
BEQ _r ; Man O' War -> Men O' War
; append "s" to monster name
; default pluralization if not handled above
append_s:
INX
LDA #$1C ; "s"
set_final_letter:
STA $60F1,X
INX
LDA #$FA ; [end-FA]
STA $60F1,X ; append [end-FA] to monster name
INX
done:
SEC
RTS

_s:
; -s pluralization handler: Cyclops, Gigantes, and Atlas don't change, Magus changes to Magi otherwise add es
LDA $60F0,X ; read second-last letter of monster name
CMP #$0E ; "e"
BEQ done ; if -es, singular=plural
CMP #$0A ; "a"
BEQ done ; if -as, singular=plural
CMP #$19 ; "p"
BEQ done ; if -ps, singular=plural
LDA $60EF,X ; read third-last letter of monster name
CMP #$10 ; "g"
BEQ _us ; Shortcut here, if g_s (Magus) replace us with i
; If not either Cyclops or Atlas (or Magus), add -es
append_es:
; append "es" to monster name
INX
LDA #$0E ; "e"
STA $60F1,X ; append "e" to monster name
BNE append_s ; append "s" to monster name; note that this branch is always taken

_us:
; replace "us" with "i"
DEX ; shorten string by 1 letter
LDA #$12 ; "i"
BNE set_final_letter

_i:
; -i pluralization handler: -i -> -ies
LDA #$12 ; "i"
STA $60F1,X ; replace final letter with "i"
BNE append_es ; append "es"

_h:
; -h pluralization handler: -ch -> -ches, -sh -> -shes, -h -> -hs (no longer need ch, keeps Mech as Mechs)
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ append_es ; if -sh, append "es"
BNE append_s ; else, append "s"

_y:
; -y pluralization handler: like -i, except needs exceptions for boy-boys and dragonfry-no change
LDA $60F0,X ; read second-last letter of monster name
CMP #$18 ; "o"
BEQ append_s ; if -oy, append "s"
CMP #$1B ; "r"
BEQ done ; if -ry, no change
BNE _i ; otherwise replace y with -ies

_a:
; -a pluralization handler: needed for madusa -> madusae
LDA $60F0,X ; read second-last letter of monster name
CMP #$1C ; "s"
BEQ append_e ; if -sa add "e"
BNE append_s ; if not -sa append "s"

append_e:
; adding "e"
INX
LDA #$0E ; "e"
BNE set_final_letter

_r:
; -r pluralization handler: needed for man o' war
LDA $60E9,X ; read ninth from end letter of monster name
CMP #$0A ; "a"
BNE append_s ; if not "a" append "s"
LDA #$0E ; "e"
STA $60E9,X ; replace "a" with "e" so that Man o' War becomes "Men o' War" <-IS THIS RIGHT?
BNE done

_n:
; -n pluralization handler: -man -> -men, -Man -> -Men, -n -> -ns
LDA $60F0,X ; read second-last letter of monster name
CMP #$0A ; "a"
BNE append_s ; if not -an, append "s"
LDA $60EF,X ; read third-last letter of monster name
CMP #$16 ; "m"
BEQ update_e ; -man -> -men
CMP #$30 ; "M"
BNE append_s ; if not -Man, append "s"
update_e:
LDA #$0E ; "e"
STA $60F0,X ; replace second-last letter of monster name
SEC
RTS

So.. yeah. I think when I saw your post earlier in the thread about addresses in ASM I read it as "Oh hey, there are ways to make branches in ASM so you don't have to count bytes. You could, but you don't have to" kinda like Pointers. Looking at the code now, I see If/Then branches don't actually use addresses but some kind of relative jump value (and that comment you made about needing JMP commands over a certain amount of bytes makes more sense now too). Only JMP commands do. Lesson learned. Anyway, the new code works great so far. Just have to test out stuff like Man/Men o' War.

Changing tracks though. I was working on the Menu script and instead of dealing with Pointer Tables and updating the Pointers manually like I did with DW1, I thought I'd take a crack at making a Cartographer script so that I could just update and insert as much as I want with no fuss, and I seem to be hung up on the extraction part. It's the #base pointer: that I think is messing me up.

Code: [Select]
#GAME NAME:      DragonW2.nes

#BLOCK NAME:      Menus
#TYPE:         NORMAL
#METHOD:      POINTER_RELATIVE
#POINTER ENDIAN:   LITTLE
#POINTER TABLE START:   $7652
#POINTER TABLE STOP:   $76E5
#POINTER SIZE:      $02
#POINTER SPACE:      $00
#STRINGS PER POINTER:   01
#ATLAS PTRS:      Yes
#BASE POINTER:      $16368
#TABLE:         dw2menu.tbl
#COMMENTS:      No
#END BLOCK

I tried following the examples (initially I thought, "Oh, Base Pointer is Text Address - Pointer Address, right?") but I can't seem to get it to extract right (cartographer just hangs as it generates a huge block of text that I don't think is anywhere close to what I want)

Speaking of menus, I just want to bring up something you mentioned a while back

Quote
You won't be able to completely copy my code since I cannibalized the free space I created by shortening "ADVENTURE LOG" to "VOLUMEN", but the ASM for handling menu control codes $98 - $9F starts at 0x3ED8A; the original game ran the same code for all of $9A - $9F, so I stole $9B - $9F for the "names in border" code.

So thanks to my menu editing I too was able to cut down my usage of the ADVENTURE LOG macro (going with " Log", with a space, and was still able to get what I wanted into the menus with plenty of space to spare. But poking around your Latin menus to see what you did. There's resizing going on obviously, but it looks like you ADDED windows (namely a couple extra battle windows). What's going on there?

I mean having the full names in the battle status window would be awesome, but I'd be happy if I could just add the full names to battle menus minus extra spaces. But is the code you mentioned above JUST for the trailing spaces on the window bar or does it cover that too?
Title: Re: General NES Hacking Questions
Post by: abw on July 06, 2019, 11:19:58 am
I think when I saw your post earlier in the thread about addresses in ASM I read it as "Oh hey, there are ways to make branches in ASM so you don't have to count bytes. You could, but you don't have to" kinda like Pointers. Looking at the code now, I see If/Then branches don't actually use addresses but some kind of relative jump value (and that comment you made about needing JMP commands over a certain amount of bytes makes more sense now too).
Yeah, all the branch ops use a signed displacement, which means they target a certain number of bytes away from the current PC value (e.g. +5 bytes, -22 bytes, etc.), rather than an absolute address like the jumps do (e.g. $885C).

So, yes, you could, but you don't have to, and if you do, then you need to write the addresses differently.

I was working on the Menu script and instead of dealing with Pointer Tables and updating the Pointers manually like I did with DW1, I thought I'd take a crack at making a Cartographer script so that I could just update and insert as much as I want with no fuss, and I seem to be hung up on the extraction part. It's the #base pointer: that I think is messing me up.
[...]
I tried following the examples (initially I thought, "Oh, Base Pointer is Text Address - Pointer Address, right?") but I can't seem to get it to extract right (cartographer just hangs as it generates a huge block of text that I don't think is anywhere close to what I want)
BASE POINTER is how much you need to add to the pointer value to get the text ROM address. The pointer value is a RAM address, though. So taking that first pointer at 0x7652 as an example, its value is D6 B6 = $B6D6 and it's in ROM bank 1, so that corresponds to ROM address 0x76E6, and $B6D6 + (-$3FF0) = $76E6, so -$3FF0 is the BASE POINTER value you want. What you're currently getting with that $16368 is 0x21A3E, which is in the middle of the title screen graphics :D.

Even without that issue, though, the problem I had with dumping the menus was that it was hard to tell where the menu data stopped. Unlike most other strings in the game, the menus don't have end tokens or anything that says exactly how many bytes long they are; after processing the menu header that sets up things like the window's height, width, and line spacing, the game just keeps on reading data until the menu is full, taking into account some variable-width control codes and non-uniform line-wrapping behaviour. If you're using the latest version of abcde, it comes with enough upgrades to get a decent dump of these menus and includes the Cartographer and table files I was using, so unless you're really determined to figure it out on your own, it's probably easier to reuse what I had.

So thanks to my menu editing I too was able to cut down my usage of the ADVENTURE LOG macro (going with " Log", with a space, and was still able to get what I wanted into the menus with plenty of space to spare. But poking around your Latin menus to see what you did. There's resizing going on obviously, but it looks like you ADDED windows (namely a couple extra battle windows). What's going on there?

I mean having the full names in the battle status window would be awesome, but I'd be happy if I could just add the full names to battle menus minus extra spaces. But is the code you mentioned above JUST for the trailing spaces on the window bar or does it cover that too?
Nope, there aren't any new windows, in battle or otherwise. Which ones look like they're new? I did move some of the windows around both in ROM as a sanity aid (I got tired of scrolling all over the place while editing related menus that were originally spread far apart) and on screen for various reasons. The code I added (https://drive.google.com/open?id=1d5miZn-VBNnmTc7B9RvIFjj2PgN1FNtu) for printing hero names with spaces replaced by top borders prints the full 8 byte names, but if you wanted to only display 4 byte names, you would only have to delete a dozen of so lines of code.
Title: Re: General NES Hacking Questions
Post by: gamingcat02261991 on July 07, 2019, 11:05:07 pm
How do you edit palettes and sprites for SMB2 (or SMUSA for the Japanese)?
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on July 08, 2019, 03:48:41 pm
How do you edit palettes and sprites for SMB2 (or SMUSA for the Japanese)?
This thread has mostly consisted of Dragon Quest 1 / 2 technical discussion but considering the thread title your question is perfectly on topic haha.

When I started with sprite editing, I used a combination of Tile Layer Pro and YYCHR software. I'd recommend watching some youtube videos that walk you through the basics. There are several good ones. FYI, one detail that took me awhile to figure out is the importance of hitting the plus and minus keys if the sprite graphics don't immediately show up clearly in the editors.
Title: Re: General NES Hacking Questions
Post by: abw on July 08, 2019, 09:10:11 pm
This thread has mostly consisted of Dragon Quest 1 / 2 technical discussion but considering the thread title your question is perfectly on topic haha.
Yeah, given that the first 96 posts in this thread are all about Dragon Warrior 1 and 2 (mostly about 2), it might be worth changing the thread title to reflect that :P.
Title: Re: General NES Hacking Questions
Post by: Chicken Knife on July 08, 2019, 09:32:06 pm
Yeah, given that the first 96 posts in this thread are all about Dragon Warrior 1 and 2 (mostly about 2), it might be worth changing the thread title to reflect that :P.
How revoltingly...sensible of a suggestion. There we are.

PS: Don't be surprised when I change the subject further to Dragon Warrior 1, 2 & 3  ;)
Title: Re: General NES Hacking Questions
Post by: abw on July 09, 2019, 08:46:19 pm
How revoltingly...sensible of a suggestion. There we are.
I try to come up with one of those every now and then. :laugh:

PS: Don't be surprised when I change the subject further to Dragon Warrior 1, 2 & 3  ;)
I await the day! :thumbsup:
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on August 24, 2019, 12:11:24 pm
@abw

Ok, the day has arrived that I'm seriously trying to get extraction and insertion rolling. I would have loved to have gotten further than step 1 on my own. :P

The first challenge I'm facing is the fact no one seems to have posted the start and end addresses of the dialogue pointers for DW3 like was done for the first two games. It's easy enough to see the actual storage of the dialogue based on loading the posted table file into a hex editor but I'm not having much success in my attempts to trace back to the block of pointers based on my formerly useful methods. I have a feeling you're going to tell me that debugging is the best way to determine them and... I need to get better at that. Also, I'd imagine this would have to be broken up into a part 1 and part 2 based on the previous extractions? I think that had to do with the final strings being a different length but I'm not 100 percent clear.

**EDIT I just want to clarify, I'm not waiting for you to find the pointer table for me. I'm going to continue to be poking and prodding.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on August 24, 2019, 06:09:02 pm
The first challenge I'm facing is the fact no one seems to have posted the start and end addresses of the dialogue pointers for DW3 like was done for the first two games. It's easy enough to see the actual storage of the dialogue based on loading the posted table file into a hex editor but I'm not having much success in my attempts to trace back to the block of pointers based on my formerly useful methods. I have a feeling you're going to tell me that debugging is the best way to determine them and... I need to get better at that. Also, I'd imagine this would have to be broken up into a part 1 and part 2 based on the previous extractions? I think that had to do with the final strings being a different length but I'm not 100 percent clear.
Debugging is always a way, but there's nothing wrong with trying an educated guess or two first. You already know from the first two games that they only store a pointer to the first string in a block of 16 strings, so maybe DW3 does the same. If you take the start of the text you can read easily at 0x40010, count ahead 16 strings to 0x40190, convert those to RAM addresses $8000 and $8180, and then search the ROM for 00808081, you should get a hit at 0x28080. Et voilà!

DW3's text is approximately twice the size of DW2's, and it's not compressed, so it sprawls over 6 banks and not every bank ends with a full 16 strings, so you'll need to split the extraction into several parts, but other than being a bit dull and repetitive, it shouldn't pose any problems.

Hmm, the Text section of the DW3 ROM map on the wiki does look woefully inadequate... let's see if we can pump it up a little ;).
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on August 24, 2019, 07:00:11 pm
Debugging is always a way, but there's nothing wrong with trying an educated guess or two first. You already know from the first two games that they only store a pointer to the first string in a block of 16 strings, so maybe DW3 does the same. If you take the start of the text you can read easily at 0x40010, count ahead 16 strings to 0x40190, convert those to RAM addresses $8000 and $8180, and then search the ROM for 00808081, you should get a hit at 0x28080. Et voilà!

DW3's text is approximately twice the size of DW2's, and it's not compressed, so it sprawls over 6 banks and not every bank ends with a full 16 strings, so you'll need to split the extraction into several parts, but other than being a bit dull and repetitive, it shouldn't pose any problems.

Hmm, the Text section of the DW3 ROM map on the wiki does look woefully inadequate... let's see if we can pump it up a little ;).
The problem with my attempts to search for the pointer was getting way too many hits. Combining two addresses being pointed to and then searching is one hell of a great idea. This is what I pay you for! :laugh:

Doing some updates to the ROM information sounds like a good idea. I'll make sure to do that.

As far as counting the strings manually at the ends of rom banks and setting up separate sections for each, that sounds doable!

One more thing I'm foggy on though. I'd say that our script increased in size somewhere between 5-10 percent. Seeing the massive chunk of blank space after the existing text made this seem like not much of a problem. But how big of a problem would it be if the text creeps into another rom bank?

Also, to your comment of DW2 and DW3's text size being roughly equivalent, that's actually shocking to hear. DW3 not only has more towns but has a day / night cycle that would seem to almost double the text. The translating / writing process definitely *felt* like it took 3x as long.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on August 26, 2019, 01:52:05 pm
Speaking of translating and inserting, I'm ready to start inserting my changes and testing DQ2.

Abw, I downloaded abcde v4 and saw your setup and tables for all the various things (seriously WOW). Unless I missed something, when extracting, the atlas.txt didn't add the table changes (easy to add myself) or the JMP commands for jumping to various parts of the ROM (a little trickier). Was abcde SUPPOSED to add those? (I did sort of Frankenstein my whole atlas.txt adding the other parts from the new version (credits, menus, etc) to my original which was just the script plus item and monster names). I'm wondering if I missed something somewhere and should just rip from scratch, or if I just have to add those JMP commands. Thanks.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 01, 2019, 01:42:57 pm
Combining two addresses being pointed to and then searching is one hell of a great idea. This is what I pay you for! :laugh:
*cha-ching* :P.

Doing some updates to the ROM information sounds like a good idea. I'll make sure to do that.
That was actually a (very) roundabout way of hinting that I had just fleshed out the text section a little bit. Definitely add anything I missed, though!

I'd say that our script increased in size somewhere between 5-10 percent. Seeing the massive chunk of blank space after the existing text made this seem like not much of a problem. But how big of a problem would it be if the text creeps into another rom bank?
Overflowing any of the ROM banks is a problem, but hopefully not one that will be difficult to overcome. The existing text is uncompressed, so if you're only over by 5-10%, adding a simple compression like DTE (which usually gets you 30-40% compression) should be more than enough. The existing text is also spread over multiple banks and there is a large chunk of free space in the final text bank, so probably there's a tiny data table somewhere that controls which bank gets loaded for each string, and you could modify that to shift the text around between banks or even extend it to use one (or more) of the existing empty banks.

Also, to your comment of DW2 and DW3's text size being roughly equivalent, that's actually shocking to hear. DW3 not only has more towns but has a day / night cycle that would seem to almost double the text. The translating / writing process definitely *felt* like it took 3x as long.
DW3 definitely has much more text than DW2. It depends exactly what you count and how, but 3x sounds about right. That "twice" was clearly a typo for "thrice" and not at all me just throwing out a number based on some vague recollection from a script size comparison I did months ago ;).

Abw, I downloaded abcde v4 and saw your setup and tables for all the various things (seriously WOW).
Heh, possibly my setup is overkill, but I like having the non-text data embedded in the script show up all nice and pretty in my script dumps; it saves me from having to go back to look up what the data is for every time I'm editing things, plus I can do all of its editing/inserting as part of the same process for editing/inserting the script.

Unless I missed something, when extracting, the atlas.txt didn't add the table changes (easy to add myself) or the JMP commands for jumping to various parts of the ROM (a little trickier). Was abcde SUPPOSED to add those?
Yeah, I've been of two minds about this. When you're not making any changes to the text engine, it would be nice to have more of the insert commands generated for you by the extract process, but when you are making engine changes, having insert commands based on the old engine is kind of annoying. Hmm, maybe I'll just add an option for that and let the end user decide what they want. For now, though, abcde does NOT add everything you'll want in your insert script, but usually it only takes a couple of minutes to add the missing parts yourself.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on September 11, 2019, 11:26:39 pm
So, I was running into some problems

First more of a program usage question abw, when trying to use abcde with all those tables I get

Code: [Select]
table 'C:\Users\mog11\Downloads\abcde_v0_0_4\menu_params.tbl' contains a control code table switch parameter to non-existant table 'heights' at C:\Users\mog11\Downloads\abcde_v0_0_4/abcde/Table/Table.pm line 82. at Atlas.txt line 0!

C:\Users\mog11\Downloads\abcde_v0_0_4>pause
Press any key to continue . . .

They're all in the same folder so I'm not sure what I'm missing here.

Also, for the asar usage, in your coding suggestions I see an added table file with inputting text with you noting "you can input these manually if you don't want to bother with a table $36,$18,$16,$0E,$FA ;" I thought I'd try this and just entered those bytes on a line and asar said it didn't understand the command. Is it db or something else? Also, just curious is the warnpc: address I see in some asm files is that a way to warn you if your code goes past a certain point?

Okay, so even worse news for me is that somehow or another my "base" rom (pre-script insert but had the new dictionary and item/monster name values changed) became corrupt somehow so I had to start over. I changed the dictionary and also from my notes changed the second script bank from 02 to 0C (bank 12) via x03FA92 and I'm getting garbled text (it also freezes once you get past the Prologue when starting a new game) I even cut out the extra stuff from the atlas.txt to focus just on the script, and I have no idea what happened.

This still looks correct, right?

Quote
// Define, load, and activate a TABLE
#VAR(Table, TABLE)
#ADDTBL("dw2_script_NEW.tbl", Table)
#ACTIVETBL(Table)

// add this near the top of the insert script:
#VAR(pointerNum, COUNTER) // create a COUNTER variable named pointerNum
#CREATECTR(pointerNum, 8, 0) // pointerNum is an 8-bit value initialized to 0
#AUTOCMD($17FE7, #WLB(pointerNum, $3FA90)) // update the code that controls which pointer starts the next bank

// Jump to start of script
#JMP($14010)
#HDR($C010)

// auto-commands for when DW2 does a mid-string bankswap and resets its read address:
#AUTOCMD($17FE7, #HDR($28010))
#AUTOCMD($17FE7, #JMP($30010, $3400F))

Really at a loss here.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 12, 2019, 07:35:30 pm
First more of a program usage question abw, when trying to use abcde with all those tables I get

Code: [Select]
table 'C:\Users\mog11\Downloads\abcde_v0_0_4\menu_params.tbl' contains a control code table switch parameter to non-existant table 'heights' at C:\Users\mog11\Downloads\abcde_v0_0_4/abcde/Table/Table.pm line 82. at Atlas.txt line 0!

They're all in the same folder so I'm not sure what I'm missing here.
What command were you running? The Atlas.bat script included in the examples folder does include all the required tables and works for me. It sounds like maybe you didn't include heights.tbl on the command line or inside your Atlas.txt file.

Also, for the asar usage, in your coding suggestions I see an added table file with inputting text with you noting "you can input these manually if you don't want to bother with a table $36,$18,$16,$0E,$FA ;" I thought I'd try this and just entered those bytes on a line and asar said it didn't understand the command. Is it db or something else? Also, just curious is the warnpc: address I see in some asm files is that a way to warn you if your code goes past a certain point?
Yup, the full line would be something like "db $36,$18,$16,$0E,$FA". I did say that I hadn't tested it :P. The xkas/Asar documentation describes what warnpc is supposed to do, though I vaguely recall having issues at some point with it not actually working. Might be worth giving it a test to confirm its behaviour.

Okay, so even worse news for me is that somehow or another my "base" rom (pre-script insert but had the new dictionary and item/monster name values changed) became corrupt somehow so I had to start over. I changed the dictionary and also from my notes changed the second script bank from 02 to 0C (bank 12) via x03FA92 and I'm getting garbled text (it also freezes once you get past the Prologue when starting a new game)
And this is why having a fully scripted insert process backed by source file version control is a Good Idea™ :P. For the crash, try modifying 0x3FA94 instead - 0x3FA92 is the +$02 in "BCC +$02", so changing that to "BCC +$0C" would probably have disastrous results.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on September 17, 2019, 12:10:40 am
So 1. I was using the old bat file I was using for my initial script inserts. Didn't realize you had to add ALL the tables to the command line. Lesson learned.

2. After a good 3 HOURS of trial and error, I found that it was indeed my dictionary text causing problems. I still don't know what exactly but since I was using the DW2 menu table to edit it, there must've been a random wrong byte getting swapped in somehow. To avoid future problems I just made a separate dictionary table and all is well again.

3.
Quote
And this is why having a fully scripted insert process backed by source file version control is a Good Idea™ :P. For the crash, try modifying 0x3FA94 instead - 0x3FA92 is the +$02 in "BCC +$02", so changing that to "BCC +$0C" would probably have disastrous results.

In my defense, having the same values right next to each other is sure easy to change the wrong one.

4. Now I can start messing around with those battle routines and I'm not having much luck so far. But so far with enemies I'm actually getting stuff like "(Hero name) appears!" for single enemies. Takign your suggestions from other posts, I did manage to combine a couple.

Code: [Select]
norom

org $1150C
base $94FC

LDA #$52
STA $A8 ; initialize per-group string ID to use in multi-group battles
LDA #$00
STA $A7 ; initialize total # of empty groups processed
process_group_string:
JSR $9EEE ; given an index (in A) into the array of structures at $0663, set $B5-$B6 to the address of the corresponding item inside that structure
LDY #$09
LDA ($B5),Y
STA $8F ; number of monsters in this group
BEQ done_display_string ; 0 monsters => no string to print
INC $A8 ; string ID to use for this group
LDY #$00
LDA ($B5),Y
STA $0161 ; current monster ID
LDX #$00
JSR $9CD6 ; write monster name in A (+ monster number within its group in X, if > 0) to $6119
LDA $60D8 ; total number of non-empty enemy groups in the current battle
CMP #$01
BEQ display_string ; if there's only 1 group, then use string ID #$0001 (change the text to be appropriate)
LDA $A8 ; otherwise use the per-group string ID (also change those texts to be appropriate)
display_string:
JSR $9CCA ; for A < #$60, display string ID specified by A; for A >= #$60, display string ID specified by A + #$A0
done_display_string:
INC $A7
LDA $A7
CMP #$04
BCC process_group_string
LDA $60D8 ; total number of non-empty enemy groups in the current battle
BNE next_section
LDA #$02 ; String ID #$0002: But it wasn't real.[end-FC]
JSR $9CCA ; for A < #$60, display string ID specified by A; for A >= #$60, display string ID specified by A + #$A0
LDA #$FD
STA $98 ; outcome of last fight?
JMP $9685
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP

next_section:

Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $00BF00 ; set the ROM file insertion point
base $8EF0 ; set the starting RAM address

LDA $0161 ; monster ID for the current group
CMP #$4E ; bosses have IDs >= #$4E (so does the "Enemies" monster, but that's not a monster ID you can encounter)
BCS no_change
SEC ; not strictly necessary since we got here by BCS and nothing has changed C
RTS
LDY $8F    ; number of monsters in group
DEY        ; count from 0 instead of 1
BEQ one     ; 0 => only one monster => handle "A" vs "An"
LDX #$00    ; read index
LDY #$00    ; write index
loop:
LDA some,X ; Monster Counts text
STA $60F1,Y ; start of text variable buffer
INY       
INX       
CMP #$FA    ; [end-FA]
BNE loop  ; if not end token, keep copying
done:
SEC        ; SEC to trigger read of [end-FA]-terminated string from $60F1, CLC to use A
RTS       
some:
db $36,$18,$16,$0E,$FA ;"Some" not using a table here
one:
; at this point we know Y = 0
LDA #$24 ; "A"
STA $60F1,Y ; start of text variable buffer
LDA $6119 ; first letter of monster name
CMP #$24 ; "A"
BEQ an
CMP #$28 ; "E"
BEQ an
CMP #$2C ; "I"
BEQ an
CMP #$32 ; "O"
BEQ an
CMP #$38 ; "U"
BNE no_change
an:
LDA #$17 ; "n"
INY
STA $60F1,Y ; start of text variable buffer
no_change:
LDA #$FA ; [end-FA]
STA $60F1,Y ; start of text variable buffer
BNE done

Also I meant to ask you if this "s" fix looked right to you

Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $01C7E8 ; set the ROM file insertion point
base $B7D8  ; set the starting RAM address

; data -> code
; if $8F-$90 == #$0001, print "s" + [end-FA] to $60F1 and SEC, else print [end-FA] and CLC
; indirect control flow target
; from $02:$BE37 via $8006
    LDA $90
    BNE s ; if $90 > 0, add "s"
    LDY $8F
    DEY
  BEQ end ; if $90 == 0 and $8F - 1 == 0 (i.e. $8F == 1), do not add "s"

s:
    LDA #$1C ; "s"

fix:
STA $60F1
    LDA #$FA ; [end-FA]
STA $60F2
    SEC
    RTS

end:
  LDA #$FA ; [end-FA]
    BNE fix
As I recall it's those last 2 bytes that needed fixing and you suggested a quick branch as an easy fix that would fit into the existing space. I just wanted to make sure I have this right.

But moving priorities a bit.

I was looking at your Latin translation so I could apply your border names code. I extracted the Latin script with Cartographer and strangely noticed that with mine, the original, the pointer and strings count up normally from 0 while yours jumps straight to 30 after 0 and seems to combine a bunch of those strings together. Did you maybe shift some pointers around or is it just Cartographer being weird?
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 17, 2019, 09:00:11 pm
So 1. I was using the old bat file I was using for my initial script inserts. Didn't realize you had to add ALL the tables to the command line. Lesson learned.
You have to tell abcde where your table files are somehow; you can do that on the command line or inside the command files, but it's not going to go around opening random files and trying to use them as tables. A table's ID and file name can be completely different, so guessing at the file name based on the ID doesn't really work (and in v0.0.5 I re-added a pre-release idea where you can have multiple tables inside the same file, so it really doesn't work there).

2. After a good 3 HOURS of trial and error, I found that it was indeed my dictionary text causing problems. I still don't know what exactly but since I was using the DW2 menu table to edit it, there must've been a random wrong byte getting swapped in somehow. To avoid future problems I just made a separate dictionary table and all is well again.
The menus definitely use a different table - it's very similar to the table used for most other things, but not quite the same. Spaces in particular can be $81 in the menus, but putting a $81 in the dictionary might result in weirdness happening.

3. In my defense, having the same values right next to each other is sure easy to change the wrong one.
Yup, there are plenty of pitfalls in this business - I'm just saying that having a scripted insert process is one way of avoiding some of them :P.

4. Also I meant to ask you if this "s" fix looked right to you
Yeah, I think that was the smallest modification that would fix the F2 control code. For the 0xBF00 code, you can shuffle things around a little bit to save a couple of bytes:
Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $00BF00 ; set the ROM file insertion point
base $8EF0 ; set the starting RAM address

LDY $8F ; number of monsters in group
DEY ; count from 0 instead of 1
BEQ one ; 0 => only one monster => handle unique monsters and "A" vs "An"
LDX #$00 ; read index
LDY #$00 ; write index
loop:
LDA some,X ; Monster Counts text
STA $60F1,Y ; start of text variable buffer
INY
INX
CMP #$FA ; [end-FA]
BNE loop ; if not end token, keep copying
done:
SEC ; SEC to trigger read of [end-FA]-terminated string from $60F1, CLC to use A
RTS
some:
db $36,$18,$16,$0E,$FA ;"Some" not using a table here
one:
; at this point we know Y = 0
LDA $0161 ; monster ID for the current group
CMP #$4E ; bosses have IDs >= #$4E (so does the "Enemies" monster, but that's not a monster ID you can encounter)
BCS no_change
LDA #$24 ; "A"
STA $60F1,Y ; start of text variable buffer
LDA $6119 ; first letter of monster name
CMP #$24 ; "A"
BEQ an
CMP #$28 ; "E"
BEQ an
CMP #$2C ; "I"
BEQ an
CMP #$32 ; "O"
BEQ an
CMP #$38 ; "U"
BNE no_change
an:
LDA #$17 ; "n"
INY
STA $60F1,Y ; start of text variable buffer
no_change:
LDA #$FA ; [end-FA]
STA $60F1,Y ; start of text variable buffer
BNE done

I was looking at your Latin translation so I could apply your border names code. I extracted the Latin script with Cartographer and strangely noticed that with mine, the original, the pointer and strings count up normally from 0 while yours jumps straight to 30 after 0 and seems to combine a bunch of those strings together. Did you maybe shift some pointers around or is it just Cartographer being weird?
I think I mentioned it earlier, but I did reorder the menus to make them easier for me to work with (e.g. storing similar menus side by side), so what you're seeing sounds right.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on September 20, 2019, 12:08:37 am
@abw,

After much dragging of feet with DQ3, I finally got down to figuring out the start and end addresses of the pointers related to each of the six rom banks, along with segregating the final pointer for each bank and counting the number of strings for it.

Something unexpected came up where there is some text data appearing at the end of the bank $15 text that doesn't have an EF end token following it. It starts at 0x05616F and ends at 0x056197. I'm thinking this has a different end token, EE? Should I count that text as an additional string in my cartographer file?

As I go through the initial text in bank $10--which has a ton of in game command related text, those end tokens would seem to be EE as well. I suppose I would just need my table file to tell abcde that both EE and EF are end tokens, right?

Now the main thing I need to do before extracting is to make sure my table file is optimal. I'm also a little foggy on the concept of the base pointer. Would I use a base pointer of $10 for all the segments of the DQ3 script or would I need to make any adjustments to that?
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 21, 2019, 11:09:20 am
Something unexpected came up where there is some text data appearing at the end of the bank $15 text that doesn't have an EF end token following it. It starts at 0x05616F and ends at 0x056197. I'm thinking this has a different end token, EE? Should I count that text as an additional string in my cartographer file?

As I go through the initial text in bank $10--which has a ton of in game command related text, those end tokens would seem to be EE as well. I suppose I would just need my table file to tell abcde that both EE and EF are end tokens, right?
It looks like I have all of EE, EF, FE, and FF listed as end tokens; FE and FF get used outside of the main script (spell/monster/item names etc.). For the main script, it's the same basic idea as in DW2 - EF is a full string end token and EE is a sub-string end token for when the game needs to chop up a longer string into smaller sections, generally to supply different values for variable control codes that get used multiple times in the full string. So yes, mark them all as end tokens!

I'm also a little foggy on the concept of the base pointer. Would I use a base pointer of $10 for all the segments of the DQ3 script or would I need to make any adjustments to that?
BASE POINTER is just how much you need to add to the pointer value (a RAM address) in order to get to the string in the ROM file (including any header). So if your pointer at 0x28080 has a value of $8000 and the corresponding string starts at 0x40010, you need a BASE POINTER value of 0x40010 - $8000 = 0x38010. Later on, when your pointer at 0x280B4 also has a value of $8000 but the corresponding string starts at 0x44010 instead, you need to update the BASE POINTER value to 0x44010 - $8000 = 0x3C010. Typically, updating the BASE POINTER value goes hand-in-hand with extracting data from a new ROM bank, but games are free to do whatever kind of craziness their programmers came up with, so that's not a hard and fast rule.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on September 23, 2019, 11:27:11 pm
 :woot!: :woot!: :woot!:

Just got the main script fully dumped--All 11 sections of it. (11 and not 12 because bank $11's final pointer conveniently addressed 16 strings.

I then went through a rather pleasurable exercise of tweaking my table file's opcode representation to present the grammar in as natural a style as possible for writing purposes. As I think about this, I should probably get the spell / item / monster lists included now as part of the extraction as to make their insertion easier down the road. I'll start working on that promptly, and will also continue to scour the document for any unidentified opcodes. It seems like it's easiest to fix them now in the dump stage of the process.

Once I get those items done, I suppose I'll be adding in the mostly finished script line by line. Since the compression you mentioned before is totally new territory for me, should I be doing anything differently with my formatting or simply prepare the insertion script the same way as I did for DQ1 and 2? After that I will be reaching out to you for help with adding code for the compression. Also, this is not a concern right now, but I believe I'll be setting up a separate boundary for each $4000 rom bank in the atlas file, correct?

September 24, 2019, 08:05:48 am - (Auto Merged - Double Posts are not allowed before 7 days.)
Ok, another thing. The game seems to have an opcode for his/her: $B0. Since there is a ton of misgendering in the original script when you pick a female hero (getting called son, boy, etc) we went for a more gender neutral approach (child, etc). I haven't yet ventured into the realm of creating new opcodes but I assume the idea of making one like son/daughter or he/she would pose quite a challenge. But again, it's not something I'm too worried about since we are relatively happy with our current gender neutral language.

September 24, 2019, 09:35:37 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
**UPDATE

It would appear that $B6 is an opcode for son/daughter. It's amazing that these things exist in the game but the game doesn't use them 9/10 of the time  :laugh:
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 24, 2019, 09:42:44 pm
:woot!: :woot!: :woot!:

Just got the main script fully dumped--All 11 sections of it. (11 and not 12 because bank $11's final pointer conveniently addressed 16 strings.
Congratulations!

Since the compression you mentioned before is totally new territory for me, should I be doing anything differently with my formatting or simply prepare the insertion script the same way as I did for DQ1 and 2? After that I will be reaching out to you for help with adding code for the compression. Also, this is not a concern right now, but I believe I'll be setting up a separate boundary for each $4000 rom bank in the atlas file, correct?
No, the insert script formatting is pretty much the same either way. Bank boundaries will take a little extra work, but not too much; basically I just set up per-bank insertion ranges and then update the ROM -> RAM address calculation offset, so something like:
Quote
#JMP($40010, $43FE7)
#HDR($38010)
// bank $10 script here

#JMP($44010, $47FE7)
#HDR($3C010)
// bank $11 script here

// etc.
As for fitting your larger script in, try to cram as much as you can into each bank and then see if the rest will fit in all the extra space in bank $15. If it does, then great; it looks like the code for controlling which bank to load for a given string starts at 0x03AE9F, so updating a couple of numbers there should be enough to make things work. If not, then you get to decide whether you want to add compression or whether you want to use up some of the other giant empty areas of space.

I haven't yet ventured into the realm of creating new opcodes but I assume the idea of making one like son/daughter or he/she would pose quite a challenge. But again, it's not something I'm too worried about since we are relatively happy with our current gender neutral language.
You mean like the B6=[son/daughter] or B2=[he/she] control codes that already exist? :D If you're happy with a gender-neutral script, that's fine, but the original game does come equipped with several control codes that display different text based on the hero's gender, so you have the option of using gender-specific language too.

For control codes, here's what I had (I think the A* ones were not used in the main script, but you can confirm that):
Quote
B0=[his/her]
B1=[himself/herself]
B2=[he/she]
B3=[him/her]
B4=[man/lady]
B5=[he's/she's]
B6=[son/daughter]
B7=[(s)-B7]
B8=[check if orb placed]

A0=[y/ies]
A1=[an/en]
A2=[ol/lls?]
A3=[i/ls?]
A4=[(es)-A4]
A5=[(s)-A5]
A6=[a/e]
A7=[no plural]

C0=[(s)-C0]
EB=[line]\n
ED=[sacrificial merchant's name]
/EE=[end-EE]\n\n
/EF=[end-EF]\n\n
F0=[class]
F2=[letter]
F3=[spell]
F4=[item]
F5=[name]
F7=[number]
F8=[number]
F9=[hero]
FD=[wait]
/FE=[end-FE]\n\n
/FF=[end-FF]\n\n
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on September 24, 2019, 10:10:18 pm
Just like what has frequently happend, I figured out that son/daughter is an opcode right before you told me. In fact I had cracked every single one on your list except for the A sequence which, yes, wasn't in the script. FYI you may have a problem with assigning both $F7 and $F8 the same designation: [number]. I used [value] for $F8.

I seem to recall you saying that you weren't exactly planning to do anything with DQ3. I'm very amused that you seem to have done all this charting out ahead of me simply for fun.  :laugh:

I've decided to just leave the spell/monster/item lists for the insertion, which is what I did last time for DQ2 according to my notes. That means it's time to start porting in the script. This will be lengthy but shouldn't be much of a headache. After that I'll start playing around with the bank loading code you pointed out. We'll almost certainly be discussing that further but I'll give it a crack.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on September 24, 2019, 11:45:13 pm
Just like what has frequently happend, I figured out that son/daughter is an opcode right before you told me. In fact I had cracked every single one on your list except for the A sequence which, yes, wasn't in the script.
See? You don't need me for this stuff at all anymore ;).

FYI you may have a problem with assigning both $F7 and $F8 the same designation: [number]. I used [value] for $F8.
Hmm, yup, good catch. I guess that must have been the point where I got sidetracked, since I had the other duplicate texts separated. Based on its usage in the script, F7 must be used for cardinal numbers (One, Two, Three, ...) and F8 for digits (1, 2, 3, ...).

I seem to recall you saying that you weren't exactly planning to do anything with DQ3. I'm very amused that you seem to have done all this charting out ahead of me simply for fun.  :laugh:
Don't you know by now that you can't believe everything you read on the internet? :P Translating just 1 and 2 feels wrong - either finish the Erdrick trilogy or the NES quadrilogy! But I got distracted by other things and didn't get much further than an initial script dump + replay (for refreshing my memory, checking some control codes I wasn't sure about, and generating a decent CDL file to use for disassembling).

I've decided to just leave the spell/monster/item lists for the insertion, which is what I did last time for DQ2 according to my notes. That means it's time to start porting in the script. This will be lengthy but shouldn't be much of a headache. After that I'll start playing around with the bank loading code you pointed out. We'll almost certainly be discussing that further but I'll give it a crack.
Sounds like a plan to me!
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: tvtoon on September 28, 2019, 10:03:50 pm
So I finally translated the biggest RAM mapping work I have done for these games, the DQ3 GBC version. I have been stocking this here for more than half a decade, always forget to post this massive stuff but when I saw this topic, flashes happened! I was going to do the NES and SNES versions too, but time is short plus it shares many traits with them.

The work is now available at Data Crystal, enjoy! (https://datacrystal.romhacking.net/wiki/Dragon_Quest_III_%28Game_Boy_Color%29:RAM_map) Off: it is about time to update those templates!
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Grimoire LD on September 29, 2019, 08:29:20 am
Spectacular! I was shocked at how little information there was on this legendary game and that about satisfied my curiosity for the most part (for most matters outside of battle). Great work!
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on October 07, 2019, 10:57:07 pm
So I finally translated the biggest RAM mapping work I have done for these games, the DQ3 GBC version. I have been stocking this here for more than half a decade, always forget to post this massive stuff but when I saw this topic, flashes happened! I was going to do the NES and SNES versions too, but time is short plus it shares many traits with them.

The work is now available at Data Crystal, enjoy! (https://datacrystal.romhacking.net/wiki/Dragon_Quest_III_%28Game_Boy_Color%29:RAM_map) Off: it is about time to update those templates!
Great work! We may find some value in this for the NES projects and it certainly looks like a nice win for Dragon Quest hacking community in general.

@abw

nejimakipiyo and I are quite far along in getting the Atlas file ready for DQ3 but we ran into a roadblock with miscellaneous text. We are having a difficult time verifying it against the Japanese. In some cases, it is dependent on rare items; in other cases my partner would have to do another playthrough; in yet other cases we have no idea what the pieces of text connect to in the game. Therefore we decided that we would need to do an extraction of the DQ3 Japanese rom's text.

We spent tonight building a kana table file and then were going to try to locate the beginning of the script along with relevant pointers. The first problem we hit was that I can't get Windhex32 to display the kana. I spent a fair amount of time investigating forums and some people seem to indicate that Windhex32EX is the version I need but that doesn't seem to be available anywhere. Anyway, I moved on to an idea I caught from a forum to use romaji instead of kana in the table file. It gets a little dicey with hiragana vs katakana vs small hiragana all translating to the same combinations of letters but we worked out a .tbl that seemed sensible. Once I loaded that, I didn't see any great swaths of text that I was expecting. The text seems to be limited to a couple short segments, and with nejimakipiyo looking over them, they appeared to be related to items. Now I'm getting DQ2 flashbacks of the compressed script, something that wouldn't surprise me considering the infamous storage issues they had with Japanese DQ3 resulting in a cut title screen and fanfare.

So to boil that all down to a couple questions: 1, any thoughts about my kana viewing issues in a hexeditor? 2, do you think my suspicions of compression are correct? And 3, if compression, could the solution be relatively simple like the bit based table file we created for DQ2? Going down this kana extraction path has created some real headaches but I think it's pretty important. I wouldn't feel great about knowing that several English lines were left alone and never vetted. I'm not exactly the "good enough is good enough" type, fortunately and unfortunately.

abw, thank you as always...
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 09, 2019, 08:41:44 pm
The work is now available at Data Crystal, enjoy! (https://datacrystal.romhacking.net/wiki/Dragon_Quest_III_%28Game_Boy_Color%29:RAM_map)
Great work! We may find some value in this for the NES projects and it certainly looks like a nice win for Dragon Quest hacking community in general.
+1 :cookie:! It'll be interesting to see how well the GBC RAM map matches up to the NES RAM map.

Therefore we decided that we would need to do an extraction of the DQ3 Japanese rom's text.
Wait, if you didn't have a script dump of either the English or Japanese text, what were you translating from?

1, any thoughts about my kana viewing issues in a hexeditor?
Saving your table file encoded in Shift-JIS, loading it in WindHex, and then enabling the "View Text Data As Unicode" menu option does the trick for me. If you're going with romaji and want/need to differentiate between the kana systems, one thing I've done in the past is to use e.g. lowercase for hiragana and uppercase for katakana.

2, do you think my suspicions of compression are correct? And 3, if compression, could the solution be relatively simple like the bit based table file we created for DQ2?
Yes, DQ3's text encoding appears to be quite similar to DQ4's from a structural point of view. It's a 6-bit encoding with hiragana <=> katakana switches on a $3C (a.k.a. %111100) and dictionary switches on $3D/3E/3F. The major obvious difference between the two encodings is the dictionary contents and some of the kana being different/reordered. For table file building, you can use the DQ4 example that ships with abcde as a basis; DQ3's 6-bit -> 8-bit hiragana lookup table starts at 0x3BB8A, its katakana lookup table starts at 0x3BBC6 (these two mostly follow the order of tiles in the PPU, but not exactly), and the 6 dictionary pointers are at 0x3BC74, pointing to $FE-terminated dictionary entries in ROM bank $10 (i.e. starting at 0x28B4B). Once you've got that sorted out, it looks like the main script pointer table starts at 0x3BC02.


In other news, I did a little bit of poking around the text engine code for DW3, and adding in DTE was pretty easy; the $80 - $AF byte range was apparently unused and had its own little code path, so I stole that for the new DTE entries, and $30 DTE entries is enough to compress the original script by over 27%, so if you're only 5% - 10% larger than the original, you should have plenty of room to spare. Assuming Choppasmith is eventually going to want to insert a script that's somewhere around 100% larger than the original, I also looked at the code that sets up the bank and pointer for a given string ID, and rewrote it to make using extra banks easy; in combination with the DTE compression, using 3 of the existing empty ROM banks should be enough space to hold double the original text.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on October 10, 2019, 03:32:16 pm
Quote
Wait, if you didn't have a script dump of either the English or Japanese text, what were you translating from?
Well, I said above that I had an English dump so we are definitely using that. :P It probably would have been more efficient for us to crack our heads getting the Japanese dump at the beginning of the process, but instead we obtained *most* of the Japanese script through various resources online and a Japanese playthrough. I have a bad habit of choosing the longer and less technically difficult path, which is quite foolish when here I am struggling with the technicalities of a Japanese extraction regardless.

Quote
Saving your table file encoded in Shift-JIS, loading it in WindHex, and then enabling the changing the settings to "View Text Data As Unicode" menu option does the trick for me. If you're going with romaji and want/need to differentiate between the kana systems, one thing I've done in the past is to use e.g. lowercase for hiragana and uppercase for katakana.
I ended up downloading the Notepad++ software after investigating Shift-JIS and kind of fell in love with it in general. As far as viewing kana in WindHex, I'm stuck on your instruction of "View Text Data As Unicode". I do see the option of "View Text Data As Japanese" under options, but I've scoured the tabs at the top several times and can't find the Unicode option.

Quote
Yes, DQ3's text encoding appears to be quite similar to DQ4's from a structural point of view. It's a 6-bit encoding with hiragana <=> katakana switches on a $3C (a.k.a. %111100) and dictionary switches on $3D/3E/3F.
Ok, the concepts of 3 table files and switches are making my head hurt. This reminds me of the numerous table files used for fixing the SHILD issue in DW2 but I'd be lying if I said I actually understood that incredible web of files you sent me. Now I'll try to comprehend this (since 3 is better than 20).

So if I understand correctly, every time the game alternates between displaying a word in hiragana vs katakana that a switch byte--$3C appears and causes the game to pull from a different character table. Does this $3C switch appear in the code that tells the game to display the text or does it appear in the text itself? I assume that my Cartographer instructions would have to load all the table files and then tell abcde to perform the same switches between tables based on the presence of that byte. If you could point out the instructions that do that in your DQ4 cartographer doc that would be helpful.

Quote
DQ3's 6-bit -> 8-bit hiragana lookup table starts at 0x3BB8A, its katakana lookup table starts at 0x3BBC6 (these two mostly follow the order of tiles in the PPU, but not exactly), and the 6 dictionary pointers are at 0x3BC74, pointing to $FE-terminated dictionary entries in ROM bank $10 (i.e. starting at 0x28B4B). Once you've got that sorted out, it looks like the main script pointer table starts at 0x3BC02.
This is another very confusing element here. In fact, I'm so confused I can hardly even articulate an appropriate question. I did see in the table files that the characters with diacritics used 8 bits instead of 6. Wouldn't they just be pairs of bytes at this point and could be saved in the table file as such? And therefore wouldn't they essentially be uncompressed and I could rely on the byte information showing in a PPU viewer?

Quote
so if you're only 5% - 10% larger than the original, you should have plenty of room to spare.
During my second big round of editing as I've been adding our script lines into the insertion file one at a time, I've found several opportunities to reduce redundancies and make for punchier language. Nothing was compromised, and in fact I strongly prefer dense, to the point writing (in spite of what my RHDN forum activity would probably indicate.) That all said, it would seem likely that I may not have a problem with text space. But we shall see. Dealing with that problem can't be any more daunting than these obstacles around extracting DQ3's Japanese script.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: filler on October 10, 2019, 06:57:30 pm
As far as viewing kana in WindHex, I'm stuck on your instruction of "View Text Data As Unicode". I do see the option of "View Text Data As Japanese" under options, but I've scoured the tabs at the top several times and can't find the Unicode option.

Those are both the same command, essentially telling WindHex to display bytes that appear in a table file as their corresponding Japanese characters. The reason for the change in wording is likely because "View Text as Unicode" was not accurate since it reads table files in S-JIS format and that setting simply renders the characters in Japanese. It's more accurate to say "View Text Data as Japanese" and Bongo` changed the wording in subsequent version(s).
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on October 10, 2019, 09:58:39 pm
Those are both the same command, essentially telling WindHex to display bytes that appear in a table file as their corresponding Japanese characters. The reason for the change in wording is likely because "View Text as Unicode" was not accurate since it reads table files in S-JIS format and that setting simply renders the characters in Japanese. It's more accurate to say "View Text Data as Japanese" and Bongo` changed the wording in subsequent version(s).
Thanks for the clarification. I did experiment with that and tried it again just now. I loaded the Japanese Dragon Quest 2 rom, loaded a hiragana table file that has been converted to SHIFT-JIS, and changed to the mode of View Text Data as Japanese.

It showed basically all the data as kanji characters, but I didn't see any of the hiragana. I used that same table file several months ago to do a text dump of the kana and it came out more or less correctly. So that all still doesn't solve the issue. hmm..
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 10, 2019, 10:19:38 pm
I have a bad habit of choosing the longer and less technically difficult path, which is quite foolish when here I am struggling with the technicalities of a Japanese extraction regardless.
Sometimes it can be very tricky indeed to tell whether the path that looks easy will actually end up being any faster than the path that looks difficult. That's a pain I'm sure most of us here can sympathize with ;).

Ok, the concepts of 3 table files and switches are making my head hurt.
Alright, how about an example? If you talk to the old man standing by the pool in the lower left corner of Aliahan castle at the start of a new game, he says:
Quote
とうぞくバコタの つくった
カギは かんたんなドアを
すべて あけたそうじゃ

That text starts partway through the byte at 0x2D3D2:

      E7 F8 BC DD 01 8A F1 8D 91 1E E3 FA F0 3D
C4 F1 9D 85 B4 FB 54 F3 73 00 F2 CE 8C DF D3 D2
D8 02 0F FA 5E 7E

Which in binary is:

                  11100111 11111000 10111100 11011101 00000001 10001010 11110001 10001101 10010001 00011110 11100011 11111010 11110000 00111101
11000100 11110001 10011101 10000101 10110100 11111011 01010100 11110011 01110011 00000000 11110010 11001110 10001100 11011111 11010011 11010010
11011000 00000010 00001111 11111010 01011110 01111110


More specifically, the old man's text starts at the second-last bit of 0x2D3D2, so if we ignore the first 6 bits (which are the end token for the previous string), we get

                        11 11111000 10111100 11011101 00000001 10001010 11110001 10001101 10010001 00011110 11100011 11111010 11110000 00111101
11000100 11110001 10011101 10000101 10110100 11111011 01010100 11110011 01110011 00000000 11110010 11001110 10001100 11011111 11010011 11010010
11011000 00000010 00001111 11111010 01011110 01111110


DQ4 uses 6-bit tokens instead of 8-bit, so considering that string of bits in groups of 6 gives us:

111111 100010 111100 110111 010000 000110 001010 111100 011000 110110 010001 000111 101110 001111 111010 111100
000011 110111 000100 111100 011001 110110 000101 101101 001111 101101 010100 111100 110111 001100 000000 111100
101100 111010 001100 110111 111101 001111 010010 110110 000000 001000 001111 111110 100101 111001 111110

which tokenizes as:

Table File   Token   Text/Effect
(hiragana)   111111   [switch to dictionary $3F for 1 token]
(dict_$3F)   100010   とう[add dakuten to next token]そく
(hiragana)   111100   [switch to katakana]
(katakana)   110111   [add dakuten to next token]
(katakana)   010000   ハ
(katakana)   000110   コ
(katakana)   001010   タ
(katakana)   111100   [switch to hiragana]
(hiragana)   011000   の
(hiragana)   110110   
(hiragana)   010001   つ
(hiragana)   000111   く
(hiragana)   101110   っ
(hiragana)   001111   た
(hiragana)   111010   [line]
(hiragana)   111100   [switch to katakana]
(katakana)   000011   カ
(katakana)   110111   [add dakuten to next token]
(katakana)   000100   キ
(katakana)   111100   [switch to hiragana]
(hiragana)   011001   は
(hiragana)   110110   
(hiragana)   000101   か
(hiragana)   101101   ん
(hiragana)   001111   た
(hiragana)   101101   ん
(hiragana)   010100   な
(hiragana)   111100   [switch to katakana]
(katakana)   110111   [add dakuten to next token]
(katakana)   001100   ト
(katakana)   000000   ア
(katakana)   111100   [switch to hiragana]
(hiragana)   101100   を
(hiragana)   111010   [line]
(hiragana)   001100   す
(hiragana)   110111   [add dakuten to next token]
(hiragana)   111101   [switch to dictionary $3D for 1 token]
(dict_$3D)   001111F   へ
(hiragana)   010010   て
(hiragana)   110110   
(hiragana)   000000   あ
(hiragana)   001000   け
(hiragana)   001111   た
(hiragana)   111110   [switch to dictionary $3E for 1 token]
(dict_$3E)   100101   そう[add dakuten to next token]しゃ
(hiragana)   111001   [end]

(those extra 6 bits at the end are the start of the next string)
So, every time the game reads a $3C (%111100), it toggles between hiragana and katakana and stays in the new table until the next time it reads a switch token; when it reads a $3D, $3E, or $3F (%111101/%111110/%111111), it switches to the corresponding dictionary for 1 token (or at least that's the high level effect; the actual ASM divides the bits up differently and treats the dictionary as 6 parts with 32 entries each rather than 3 parts with 64 entries each).

In the sample DQ4 table files, DQ4's table switches happen on the
Quote
!%111100=,<@katakana>:%111100
!%1111=,<@dictionary>:1
lines. You'll notice that I split the 12 bits for the dictionary switch + dictionary entry into 4 bits for switching and 8 bits (1 byte) for the entry; keeping them as 6 bits each will give you the exact same effect, so which form you prefer is entirely up to you.

This is another very confusing element here. In fact, I'm so confused I can hardly even articulate an appropriate question. I did see in the table files that the characters with diacritics used 8 bits instead of 6. Wouldn't they just be pairs of bytes at this point and could be saved in the table file as such? And therefore wouldn't they essentially be uncompressed and I could rely on the byte information showing in a PPU viewer?
DQ3's dictionary entries are a series of 8-bit values, so they get to use the full 8-bit range, but the hiragana and katakana entries are only 6-bit values and they get translated into 8-bit values via a pair of lookup tables, one for hiragana (e.g. hiragana %000000 translates to $0B, which is あ in the PPU viewer) and one for katakana (e.g. katakana %000000 translates to $3D, which is ア in the PPU viewer). You should be able to read the dictionary entries and 6-to-8-bit lookup tables with a byte-based table file, but the script itself is still encoded as a series of 6-bit tokens, so you won't be able to see that as easily.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on October 13, 2019, 01:40:17 am
Assuming Choppasmith is eventually going to want to insert a script that's somewhere around 100% larger than the original, I also looked at the code that sets up the bank and pointer for a given string ID, and rewrote it to make using extra banks easy; in combination with the DTE compression, using 3 of the existing empty ROM banks should be enough space to hold double the original text.

Thank you for this! From earlier posts I was looking at the amount of space going, with the first two games so far, I'll probably need DTE at least.

For control codes, here's what I had (I think the A* ones were not used in the main script, but you can confirm that):

I had heard about the mostly gender neutral, but infamous "boy" dialog in the Japanese version as well. This is great for me because the mobile script just gives the female hero a separate string, while there's some like below where they ADD dialog for Female Hero, there's a lot of duplicate lines.

Code: [Select]
My father was always telling me stories about the mighty hero Ortega.<10
And now here I am working in my dad's place, and there you are adventuring in yours.<10
It's true what they say, isn't it? Like father, like son!<41<63 Well, daughter in your case!<64<0F

I think it was something the Japanese script did to help correct said dialog problem in the original.

Also I just saw as I'm typing this there ARE cases where the gender is adjusted for strings. Weird

Code: [Select]
We expected no less of you, <41son<63daughter<64 of Ortega! We have witnessed the birth of a true hero!
So I finally translated the biggest RAM mapping work I have done for these games, the DQ3 GBC version. I have been stocking this here for more than half a decade, always forget to post this massive stuff but when I saw this topic, flashes happened! I was going to do the NES and SNES versions too, but time is short plus it shares many traits with them.

The work is now available at Data Crystal, enjoy! (https://datacrystal.romhacking.net/wiki/Dragon_Quest_III_%28Game_Boy_Color%29:RAM_map) Off: it is about time to update those templates!


Hey thanks for this! I still plan on tackling the GBC remakes down the line. So, the more info on III, the better!



So abw, sorry to bring up DW2, I really hope this is the last thing. But I allllmost got the code right for changing battle dialog to:

1: Ignore counting the last few bosses
2: Replace the multiple enemy counts with "Some" and "A/An"
3: Change the extra groups that appear to "And/AND some [monster]"

here's my code

Code: [Select]
norom ; stop Asar from trying to apply SNES memory mapping to this NES code
org $00BF00 ; set the ROM file insertion point
base $BEF0 ; set the starting RAM address

LDA $0161 ; monster ID for the current group
CMP #$4E ; bosses have IDs >= #$4E (so does the "Enemies" monster, but that's not a monster ID you can encounter)
BCS no_change
LDY $8F    ; number of monsters in group
DEY        ; count from 0 instead of 1
BEQ one     ; 0 => only one monster => handle "A" vs "An"
LDX #$00    ; read index
LDY #$00    ; write index
loop:
LDA some,X ; Monster Counts text
STA $60F1,Y ; start of text variable buffer
INY       
INX       
CMP #$FA    ; [end-FA]
BNE loop  ; if not end token, keep copying
done:
SEC        ; SEC to trigger read of [end-FA]-terminated string from $60F1, CLC to use A
RTS       
some:
db $36,$18,$16,$0E,$FA ;"Some" not using a table here
one:
; at this point we know Y = 0
LDA #$24 ; "A"
STA $60F1,Y ; start of text variable buffer
LDA $6119 ; first letter of monster name
CMP #$24 ; "A"
BEQ an
CMP #$28 ; "E"
BEQ an
CMP #$2C ; "I"
BEQ an
CMP #$32 ; "O"
BEQ an
CMP #$38 ; "U"
BNE no_change
an:
LDA #$17 ; "n"
INY
STA $60F1,Y ; start of text variable buffer
no_change:
LDA #$FA ; [end-FA]
STA $60F1,Y ; start of text variable buffer
BNE done
no_cardinal:
LDA #$FA ; [end-FA] the game will handle trimming the empty space from the
STA $60F1
BNE done

The thing that has me stumped is the A/an part. In battle if I get a single enemy with a consonant letter I get, for example _ Dracky where the underscore is an extra space the A should go. Yet getting a single Iron Ant gives me "A Iron Ant"

(to my credit, I was getting nothing but garbage before, so this has come a long way before this post)
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on October 14, 2019, 12:07:39 pm
@abw

Thank you for the very detailed breakdown of how the compressed script works. This was helpful but there are still some things I need to understand before I can do this.

First, I did figure out the issue with viewing kana in WindHEX. It's essentially what you all were saying. I needed to use a software like Notepad++ to change the coding from UTF-8 variants to Shift-JIS. Interestingly, when I change the encoding to the latter, it makes the Japanese characters unviewable to me in the table files so not a good situation for editing. Perhaps I need to download something for windows to allow my PC to read Shift-JIS? The kana become things like x82xAD, etc. When I take that same garbled table file, load it in windhex and view data in Japanese it actually works. Let me qualify that: it works in a non compressed Japanese roms like DQ2, but not compressed ones like DQ3 or DQ4. When I try to load your included hira table file (reencoded to Shift-JIS) for DQ4 into the game, it doesn't show me anything at all--probably because Windhex isn't equipped to view data in that 6 bit format. With that being the case, I don't know how I'm going to be able to locate certain things like start address of the script. I would imagine it starts at the beginning of a rom bank and if you told me what the address of the first script bank was, I could probably use the pointers to figure out how many banks hold text and get the rest of the info I need. Well, I might need the address of the final script pointer since you only gave me the first. :P

The other thing I need help with is understanding some aspects of your DQ4 example files. The cartographer.txt file you included has all the different sections for different parts of the script (all with mysteriously different string lengths) but the only table file that gets referenced is the hira.tbl. You explained how the switch bytes cause the game to switch between the two character sets and the dictionary but you didn't seem to explain how abcde is going to do the switching. Since I don't see any table switching instructions in the cartographer file, I also looked at the table files themselves and the only thing that might be the instructions are
!%111100=,<@katakana>:0
!%1111=,<@dictionary>:1
I'd think that these were the switching instructions if it wasn't for the fact that the table names aren't included here. But I do notice theres a little @katakana symbol at the top of the kata.tbl file. Is that the reference point that this instruction points at allowing the switching to happen during extraction?

Well, that's the extent of my questions for now. I truly wish I was better at figuring these kinds of things out on my own. I am certainly trying my hardest before coming back to you.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 14, 2019, 10:11:43 pm
The thing that has me stumped is the A/an part. In battle if I get a single enemy with a consonant letter I get, for example _ Dracky where the underscore is an extra space the A should go. Yet getting a single Iron Ant gives me "A Iron Ant"
Have you tried stepping through the code in a debugger to follow along with exactly what it's doing and where it's going wrong?
Spoiler:
It looks like the Y values aren't getting updated correctly between writes to $60F1,Y, so the last letter you write is getting clobbered by $FA; making sure there's an INY after each write should help.

I don't know how I'm going to be able to locate certain things like start address of the script.
If you want to play along, Trace Logger + Debugger tells all. It looks like the main part of the text engine code starts somewhere around $0E:$B9D1, so that would be a good place for an execute breakpoint. The bank specified by $A8 gets swapped in before reading from the script and the engine setup code sets $A8 to either #$10, #$11 (via INC $A8), or #$02 based on the string index, so those banks ought to match up with the script pointers.

You explained how the switch bytes cause the game to switch between the two character sets and the dictionary but you didn't seem to explain how abcde is going to do the switching.
No, that part gets covered in abcde's readme where I spend several hundred words and many examples explaining how table file switching works by default and various ways you can control specific details of the switching process in order to mirror your game's behaviour :P. The following excerpt is particularly relevant here:
Quote from: readme.txt
# matchType says how many matches to make in the new table before falling back to the current table:
# * 0 => keep going as long as you can;
# * X => make exactly X matches in the new table, where X is any positive decimal integer;
# * -1 => fall back right away;

I'd think that these were the switching instructions if it wasn't for the fact that the table names aren't included here.
You can also find where the table files get referenced by looking at the example insert batch file / shell script:
Quote from: Atlas.bat
perl ..\..\..\abcde.pl -m text2bin -t hira.tbl -t kata.tbl -t dict.tbl -cm abcde::Atlas Atlas.nes Atlas.txt
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on October 20, 2019, 01:31:02 am
Have you tried stepping through the code in a debugger to follow along with exactly what it's doing and where it's going wrong?
Spoiler:
It looks like the Y values aren't getting updated correctly between writes to $60F1,Y, so the last letter you write is getting clobbered by $FA; making sure there's an INY after each write should help.

So I was in fact able to figure out on my own that, yeah, something was going on with reading the first letter of the Monster name (somehow it was reading an M instead of an I in Iron Ant) But I had no idea it was because of a missing Increment (INY). Thank you so much. Am I correct in understanding that Y, in this case, is basically the "cursor" in reading and printing letters/strings and that's why you need to increment it after a printed letter?

Also, in the other thread when I brought up rewriting the dialog into present tense instead of past tense, namely for "[monster] appear/s" you suggested using the "Special S" but I quickly realized unlike nouns the usage of s in verbs is reversed. Unlike 1 Coin/10 Coins, you have 1 monster appears/3 monsters appear (I even thought, "Oh! Draws near! Wait, no, that's the same thing".) I just want to ask if there was something you were thinking of that I'm not thinking of otherwise I think it's a pretty negligible sacrifice to use the past tense "appeared". Sure am glad to see DW3 seems to use present tense "appear" and "appears". :)
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 20, 2019, 04:19:59 pm
Am I correct in understanding that Y, in this case, is basically the "cursor" in reading and printing letters/strings and that's why you need to increment it after a printed letter?
Yup, in that particular code, Y is used as the write index: "STA $60F1,Y" means to store the current value of A to the address $60F1 + Y, so when Y is 0, that $60F1, when Y is 1, that's $60F2, etc. If you open the Hex Editor, load the appropriate DW2 table file, and scroll down to $60F1, you can step through the code and watch the string being assembled there.

Also, in the other thread when I brought up rewriting the dialog into present tense instead of past tense, namely for "[monster] appear/s" you suggested using the "Special S" but I quickly realized unlike nouns the usage of s in verbs is reversed. Unlike 1 Coin/10 Coins, you have 1 monster appears/3 monsters appear (I even thought, "Oh! Draws near! Wait, no, that's the same thing".) I just want to ask if there was something you were thinking of that I'm not thinking of otherwise I think it's a pretty negligible sacrifice to use the past tense "appeared". Sure am glad to see DW3 seems to use present tense "appear" and "appears". :)
Hmm, yes, that is a good point. If you have enough space, you could add a new control code that does the opposite of what that $F2 does, i.e. prints "s" when $8F-$90 is 1 instead of not 1, and then use that in your script. But using "appeared" instead of "appear"/"appears" is also fine too.


In other news, I noticed on TCRF (https://tcrf.net/Dragon_Warrior_II_(NES)) that Cannock Castle also had a map change between the Japanese and English releases, so I figured I'd make a restoration patch for that too, and while I was at it, I figured I might as well also upload some patches for DW1 and DW2, so hopefully those will be approved shortly!
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on October 22, 2019, 12:02:59 am
In other news, I noticed on TCRF (https://tcrf.net/Dragon_Warrior_II_(NES)) that Cannock Castle also had a map change between the Japanese and English releases, so I figured I'd make a restoration patch for that too, and while I was at it, I figured I might as well also upload some patches for DW1 and DW2, so hopefully those will be approved shortly!

Yeah I saw that, it's really minor change that just makes you go "Why?" There's a lot more in DW3, but the Data Crystal entry says map graphics are essentially like text. I would assume they're done in rows so I hope it turns out to be easier to edit.



So, abw, I did a "fresh" rip of DW2 with abcde v5. Copied my script over and changed the appropriate variables and there's two weird things. While my previous Atlas.txt was kinda all over the place, my new one is more in line with the rip.

First: It ripped the Prologue and everything else fine, but when I try to re-insert it says it can't tolkenize spaces and points to the first space I have there (Left side of the first line). I realized I needed to add the Table line to make sure it uses the spacer.tbl file but the Atlas readme isn't too insightful on how to use different tables I have it as:

Code: [Select]
#VAR(nonScriptTBL, TABLE)
#ADDTBL("spacer.tbl", main) // or whatever you named your table file
#ACTIVETBL(main)

I'm sure it's something painfully obvious I'm missing. If I don't add the main part it says it tries to call a table that doesn't exist, and when it's added it says main table unknown. @_@

Second: As I said in a previous post, I incorporated your full name hack (thank you again for that :))and I ripped the script in the menus to see what you did. I see that the new value $9C is for names in battle with trailing spaces, but I'm having trouble with the new field menu party list. YOUR Script shows for the beginning solo Midenhall status like this

Code: [Select]
═NOMEN═════GR══SL══LM[border line]
<$5F>[Midenhall's short name]  [Midenhall's level]<$5F>[Midenhall's current HP]<$5F>[Midenhall's current MP][line]

I think something might be going wrong with the fact that NORMALLY $5F and $81 are both spaces. I guess I'm stumped because I expected some kind of new value instead of "Midenhall's short name". Does the $5F really do something different here needing a change to the menu_params.tbl?

EDIT: Oh! I see your patches have just been approved! Maybe I'll use the menu improvement patch instead of trying to reverse engineer it from your Latin translation!

EDIT2: Between your Latin version and seeing how your new menu improvement patch works, I'm getting really turned around from the pointer values jumping back and forth. Makes it super confusing to edit! I totally see what you did but I just need to know: If I rip that part of the script with abcde/cartographer, with it's random Pointer Jumps (going from pointer #73 to Pointer #8) and edit the menus will Atlas still insert it properly? I'm curious how you even went in and said "Okay $7694 is practically the same as this menu pointed to $7690 so just point to that one"?(within something like abcde, I get how you'd do it manually)

EDIT3: Not a question, just a minor observation, but I'm surprised with your battle menu edit, you didn't just move the command menu to the left a couple of spaces over so that the monster name window doesn't overlap with the command window. That's what I did *shrug*
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 22, 2019, 08:36:51 pm
Yeah I saw that, it's really minor change that just makes you go "Why?"
Yup, that's pretty much exactly what I said when I saw that too :P.

There's a lot more in DW3, but the Data Crystal entry says map graphics are essentially like text. I would assume they're done in rows so I hope it turns out to be easier to edit.
Let's hope so - editing DW2's maps is only fun in a masochistic sort of way :(.

I realized I needed to add the Table line to make sure it uses the spacer.tbl file but the Atlas readme isn't too insightful on how to use different tables I have it as:

Code: [Select]
#VAR(nonScriptTBL, TABLE)
#ADDTBL("spacer.tbl", main) // or whatever you named your table file
#ACTIVETBL(main)

I'm sure it's something painfully obvious I'm missing. If I don't add the main part it says it tries to call a table that doesn't exist, and when it's added it says main table unknown. @_@
The variable name you create with #VAR has to match up with the variable name you use in #ADDTBL and #ACTIVETBL. In my insert script, I just went with a plain old boring:
Code: [Select]
#VAR(Spacer, TABLE)
#ADDTBL("spacer.tbl", Spacer)
#ACTIVETBL(Spacer)
but you can use whatever names you want as long as you use them consistently.

I see that the new value $9C is for names in battle with trailing spaces
In the menu patch I uploaded separately, menu control code $9C now prints all 8 bytes of the current hero + 1's name with both $5F and $60 spaces replaced by top borders. The version I used in my translation is a bit different - it only replaces $60 spaces, which worked since I changed Cannock/Moonbrooke's names to use $60 instead of $5F (matching Midenhall, who also uses $60 spaces).

EDIT2: Between your Latin version and seeing how your new menu improvement patch works, I'm getting really turned around from the pointer values jumping back and forth. Makes it super confusing to edit! I totally see what you did but I just need to know: If I rip that part of the script with abcde/cartographer, with it's random Pointer Jumps (going from pointer #73 to Pointer #8) and edit the menus will Atlas still insert it properly? I'm curious how you even went in and said "Okay $7694 is practically the same as this menu pointed to $7690 so just point to that one"?(within something like abcde, I get how you'd do it manually)
I'm not sure what you're saying here - I did re-order how the menu data is stored, but the menu pointed at by $7694 is still the same menu in both the original and patched versions (I've dubbed it "Menu ID #$21: Mini status window, top, Midenhall + Cannock + Moonbrooke"), and it's still a separate menu from the menu pointed at by $7690 ("Menu ID #$1F: Mini status window, bottom, Midenhall + Cannock + Moonbrooke"), which is likewise also the same menu in both cases. If it bothers you that much you can re-sort the menus to be in the same order as their pointers; I just found the original ordering to be both counter-intuitive and counter-productive for editing purposes, so I changed it.

EDIT3: Not a question, just a minor observation, but I'm surprised with your battle menu edit, you didn't just move the command menu to the left a couple of spaces over so that the monster name window doesn't overlap with the command window. That's what I did *shrug*
After trying it both ways for a while, I eventually decided I liked the overlapping menu effect more. I was trying to keep everything within the bounds of the 24-tile main dialogue window, and mostly succeeded, except the 16-tile equipment list and 10-tile attack/defense power list didn't quite fit and moving the attack/defense power window just looked weird.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Swordmaster on October 23, 2019, 05:05:45 am
In other news, I noticed on TCRF (https://tcrf.net/Dragon_Warrior_II_(NES)) that Cannock Castle also had a map change between the Japanese and English releases, so I figured I'd make a restoration patch for that too, and while I was at it, I figured I might as well also upload some patches for DW1 and DW2, so hopefully those will be approved shortly!

Just wanted to clarify that the original location of Cannock/Samaltria was changed before even the Japanese release.  'Twas a casualty of last minute balancing before release.  And we all know how perfectly balanced the finished product is.   :laugh:
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on October 23, 2019, 03:27:27 pm

I'm not sure what you're saying here - I did re-order how the menu data is stored, but the menu pointed at by $7694 is still the same menu in both the original and patched versions (I've dubbed it "Menu ID #$21: Mini status window, top, Midenhall + Cannock + Moonbrooke"), and it's still a separate menu from the menu pointed at by $7690 ("Menu ID #$1F: Mini status window, bottom, Midenhall + Cannock + Moonbrooke"), which is likewise also the same menu in both cases. If it bothers you that much you can re-sort the menus to be in the same order as their pointers; I just found the original ordering to be both counter-intuitive and counter-productive for editing purposes, so I changed it.

Sorry, allow me to simplify:

-I ripped the menu dialog using the defualt example script provided

-This makes the output text jump to pointers randomly. Instead of Pointer 0:Text, Pointer 1:Text, Pointer 2:Text, etc, this causes the output file to Pointer 0:Text, Pointer 30: text, Pointer 31: Text, Pointer 9: Text, etc.

-I also noticed with the way the text was reordered a lot of the pointers would just display a lot of the same dialog

-This was tricky to figure out with your Latin translation because I would see stuff like CUI and go "Crap, which menu does this refer to?" After using Google Translate and playing the patched rom I was able to figure it out. 

What it really comes down to is: Since there's a separate patch for the English version now, I just need to change a couple letters like Armor->Armour, the Healer/Chruch Menu etc. Is it safe to edit and re-insert the text like this or am I going to need to take a different approach?
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on October 23, 2019, 10:09:16 pm
-I ripped the menu dialog using the defualt example script provided
Keep in mind the example script is for the original game, so if you're going to re-dump from the patched ROM, you'll need to adjust the #SCRIPT STOP to $7F3D otherwise the last few bytes of menu data will get cut off.

-This makes the output text jump to pointers randomly. Instead of Pointer 0:Text, Pointer 1:Text, Pointer 2:Text, etc, this causes the output file to Pointer 0:Text, Pointer 30: text, Pointer 31: Text, Pointer 9: Text, etc.
Yup. And? As long as the pointers keep pointing to the right thing, it mostly doesn't matter what order the things are arranged in (on the NES, anyway).

-I also noticed with the way the text was reordered a lot of the pointers would just display a lot of the same dialog
I'm not sure what you're seeing here - no two of the menu pointers point to the same string.

-This was tricky to figure out with your Latin translation because I would see stuff like CUI and go "Crap, which menu does this refer to?" After using Google Translate and playing the patched rom I was able to figure it out. 
Yeah, it took a little while to track down which menu was which, particularly with all those WHOM menus. If you check the commented disassembly (http://datacrystal.romhacking.net/wiki/Dragon_Warrior_II::ROM_map/ASM_bank_01), the menu pointer table at $01:$B642 is a handy reference for stuff like that.

What it really comes down to is: Since there's a separate patch for the English version now, I just need to change a couple letters like Armor->Armour, the Healer/Chruch Menu etc. Is it safe to edit and re-insert the text like this or am I going to need to take a different approach?
Editing and re-inserting should be fine... the only thing to watch out for is that the code for printing bordered names starts at $7F40, so if you need to use any of that space, you'll have to update the ASM file to move the border code somewhere else and then re-assemble it.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on November 16, 2019, 04:11:11 pm
Keep in mind the example script is for the original game, so if you're going to re-dump from the patched ROM, you'll need to adjust the #SCRIPT STOP to $7F3D otherwise the last few bytes of menu data will get cut off.
Yup. And? As long as the pointers keep pointing to the right thing, it mostly doesn't matter what order the things are arranged in (on the NES, anyway).
I'm not sure what you're seeing here - no two of the menu pointers point to the same string.
Yeah, it took a little while to track down which menu was which, particularly with all those WHOM menus. If you check the commented disassembly (http://datacrystal.romhacking.net/wiki/Dragon_Warrior_II::ROM_map/ASM_bank_01), the menu pointer table at $01:$B642 is a handy reference for stuff like that.
Editing and re-inserting should be fine... the only thing to watch out for is that the code for printing bordered names starts at $7F40, so if you need to use any of that space, you'll have to update the ASM file to move the border code somewhere else and then re-assemble it.

Thanks abw, I've procrastinated quite a bit, but it's coming together nicely now. I'm just having a weird issue with abcde/atlas. I fixed the table issue I mentioned above, but now it gives me the message

Code: [Select]
table 'C:\Users\mog11\Downloads\abcde_v0_0_5\eg\NES\Dragon Warrior II\spacer.tbl' contains a control code table switch parameter to non-existant table 'main' at C:\Users\mog11\Downloads\abcde_v0_0_5/abcde/Table/Table.pm line 111. at Atlas.txt line 0!
I thought I'd try to start fresh with a new unzipped copy of abcde, but there I get

Code: [Select]
attempt to write beyond $7F3D at C:\Users\mog11\Downloads\abcde_v0_0_5c/abcde/Atlas.pm line 476, <COMMAND_FILE> line 3088.
variable 'Table' has already been declared at Atlas.txt line 3088!

Line 3088 Is the Credits part of my Atlas.txt which is setup as
#VAR(Credits, TABLE)
#ADDTBL("credits_spacer.tbl", Credits)
#ACTIVETBL(Credits)

Another weird thing is that after inserting the new Prologue the game just skips it (Start new game > Fades to black > Moonbrooke cutscene) I figured maybe I have a a couple of lines too long. I thought it would be like doing the title screens in DW1. Room for 32 spaces with an even amount of spaces on either side of the text to center it as long as it adds up to 32. Could that not be the case here?
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on November 17, 2019, 01:58:49 pm
@abw

I thought I'd provide my own miserable update. The last we left off, you referred me to the table switching section of abcde's documentation. Over the last few weeks I've made numerous attempts to digest it. "It" being copied below:

Quote
# Table switch entries are new (and/or improved, depending on what you're already familiar with); this can get a bit complicated, so let's add some more table files and see some examples.
# The table switch entry format looks like this: !lhs=<label>,<@table ID>:matchType,<@table ID>:matchType,<@table ID>:matchType,...
# Where <label> is an optional label for the control code that, if provided, appears in text output,
# <@table ID>: is an optional table ID that will be used to continue matching; if you don't provide a table ID, one of two things will happen:
# * if you don't provide any value here, you get raw hexadecimal-encoded output;
# * if you provide the special value <binary> (note the lack of "@"), you get raw binary-encoded output;
# matchType says how many matches to make in the new table before falling back to the current table:
# * 0 => keep going as long as you can;
# * X => make exactly X matches in the new table, where X is any positive decimal integer;
# * -1 => fall back right away;
# * $hex or %bin => keep going until you match $hex or %bin again (the "$" or "%" are required here so that we can tell whether 10 means ten, sixteen, or two).
# Once the matching condition for the table that was switched into has been satisfied, translation will continue with the table that did the switching.
#
# For table entries that require exactly X matches, the default match counting behaviour is for each matched table entry in the new table to count as 1 match.
# However, different games can count matches in different ways, so abcde provides support for modifying the counting behaviour in a couple of ways:
# * you can change the number of matches a table entry counts as by suffixing the left-hand side of that entry with "<Y>" for any non-negative decimal integer Y; the table entry will then be counted as Y matches instead of 1 match;
# * you can do the same thing with matchType when using $hex or %bin;
# * matchType can be followed by a "+" to indicate that matches in the new table should also contribute towards the number of required matches in the current table.
In short, your explanation reads to me like something written from a programmer to a programmer, and I personally can't make heads of tales of it. I was able to follow everything above that section of the doc, probably due to A: being already knowledgeable on the topics, and B: the concepts being simpler in general. I feel like you need a whole separate set of instructions written for the laypeople (if laypeople ultimately have any business trying to take on these kinds of projects in the first place.)

What might be a lot more helpful to me is attempting to simply copy the examples you've provided from DQ4. I may try that in a few days with my next opportunity to be alone in a quiet house. I do want to acclimate myself to actually being able to read these kinds of instructions, but I feel like I need to get a hold of some books that provide a general introduction to programming before that is possible. It's a shame to feel this stuck when so much work has been done for the DQ3 re-translation. Maybe I return to the idea of grinding for all the 1/256 rare drops that we ultimately need the Japanese text from.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on November 24, 2019, 07:39:17 pm
Sorry for leaving you both hanging - I've been offline again for the past little while.

I fixed the table issue I mentioned above, but now it gives me the message

Code: [Select]
table 'C:\Users\mog11\Downloads\abcde_v0_0_5\eg\NES\Dragon Warrior II\spacer.tbl' contains a control code table switch parameter to non-existant table 'main' at C:\Users\mog11\Downloads\abcde_v0_0_5/abcde/Table/Table.pm line 111. at Atlas.txt line 0!
That means the 'spacer.tbl' file wants to be able to switch to the table whose ID is 'main', but none of the table files that got loaded have a 'main' ID (@main), so you're missing at least one table file.

I thought I'd try to start fresh with a new unzipped copy of abcde, but there I get

Code: [Select]
attempt to write beyond $7F3D at C:\Users\mog11\Downloads\abcde_v0_0_5c/abcde/Atlas.pm line 476, <COMMAND_FILE> line 3088.
variable 'Table' has already been declared at Atlas.txt line 3088!
That looks like a) whatever text you were trying to insert (if it was capped at $7F3D, it's probably the menu text) was too long and b) you tried to declare 'Table' as a variable name multiple times (i.e. you have multiple "#VAR(Table, ****)" commands where **** is the type, which I'm guessing is probably TABLE).

Another weird thing is that after inserting the new Prologue the game just skips it (Start new game > Fades to black > Moonbrooke cutscene) I figured maybe I have a a couple of lines too long.
Hmm, that one does sound odd. Assuming you haven't (possibly accidentally) made any code changes, my first guess would be that your new prologue text starts with an end token, causing the game to skip ahead to the Moonbrooke cutscene, though I haven't spent much time looking at bank 7, so it could easily be something else. The text length shouldn't be an issue; I haven't tried it, but I think the game just keeps printing text until it reaches an end token.

I thought it would be like doing the title screens in DW1. Room for 32 spaces with an even amount of spaces on either side of the text to center it as long as it adds up to 32. Could that not be the case here?
If you look at spacer.tbl, you'll see the first byte controls the left indent (if you need an indent outside the 1 - 15 range, you can add it to the table file yourself), and then text is stored using the main (non-script) 8-bit encoding, with lines terminated by $FF, so that's quite different than the DW1 title screen, which was bascially a 32x28 rectangle with some RLE.

In short, your explanation reads to me like something written from a programmer to a programmer, and I personally can't make heads of tales of it.
That was actually my sad attempt at end-user documentation right there - I was hoping the examples would flesh out the technical descriptions, but apparently not :P. Table switching is by far the most complicated thing about table files, and if you're going to be writing your own, you need to know the gory details.

If a basic table entry looks like "03=cat" and means that the byte 03 corresponds to the text "cat", then one kind of table switch entry looks like "!00=<[Animal Name]>,<@animals>:1". That means that the byte 00 triggers a switch to a new table with the ID "animals", reads exactly 1 match from that table and then reverts to the current table (the one that contains that "!00=<[Animal Name]>,<@animals>:1" entry), and that the text "[Animal Name]" appears in the extracted text immediately before whatever got matched in the "animals" table. So if your main table (which I'm going to call "main") and "animals" table look like this:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
80=main

@animals
03=[cat]
04=[dog]
then the bytes 000380 would correspond to the text "[Animal Name][cat]main" and the bytes 000480 would correspond to the text "[Animal Name][dog]main". But different games implement table switching in different ways and you might not want to see all the switches listed in your text, so there are lots of options. First off, the "label" part is optional, so you could say:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
80=main

@animals
03=[cat]
04=[dog]
and then the bytes 010380 would correspond to the text "[cat]main". You might want to read exactly 2 or exactly 3 matches from the "animals" table instead, so you could say:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
and then the bytes 02030480 would correspond to the text "[Animal Name][cat][dog]main" and the bytes 0303040580 would correspond to the text "[Animal Name][cat][dog][parakeet]main". You can do that for any positive number of matches, not just 1, 2, or 3. If you want to read exactly 1,000,000 matches from the "animals" table, go for it:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
and then the bytes 04... would get you a really long text! On the other hand, maybe you don't want to read a predetermined exact number of matches. Maybe you want to keep reading animal names until you come across a certain terminator, such as if your list of animals was terminated with a FF byte:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
In that case, the bytes 050304FF80 would correspond to the text "[Animal Name][cat][dog]main". Or maybe there is no explicit terminator and you want to keep reading animal names until you come across something that doesn't match any known animal:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
and then the bytes 06030480 would correspond to the text "[Animal Name][cat][dog]main". Another way of accomplishing the FF-terminated example is this:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
!FF=<>,-1
and then the bytes 060304FF80 would also correspond to the text "[Animal Name][cat][dog]main". You're also not limited to switching to just one table; if a 07 byte means that the game reads an animal name and then a colour, you could represent that as:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
!07=<[Animal Name/Colour]>,<@animals>:1,<@colours>:1
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
!FF=<>,-1

@colours
00=
and then the bytes 07030080 would correspond to the text "[Animal Name/Colour][cat]main". And then you can play with how many matches each table entry counts as; if there's something special about elephants and they count for twice as much as cats or dogs, you could represent that as:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
!07=<[Animal Name/Colour]>,<@animals>:1,<@colours>:1
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
06<2>=[elephant]
!FF=<>,-1

@colours
00=
and then the bytes 020680 would correspond to the text "[Animal Name][elephant]main". You're also not limited to switching only one table deep; with tables like these:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
!07=<[Animal Name/Colour]>,<@animals>:1,<@colours>:1
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
06<2>=[elephant]
!07=<[Dog Breed]>,<@dogbreeds>:1
!FF=<>,-1

@colours
00=

@dogbreeds
00=[Labrador]
01=[Chihuahua]
the bytes 00070080 would correspond to the text "[Animal Name][Dog Breed][Labrador]main". Counting can also be transitive, so if you needed to include matches in the "dogbreeds" table when counting matches from the "main" table, you could say:
Quote
@main
!00=<[Animal Name]>,<@animals>:1
!01=<>,<@animals>:1
!02=<[Animal Name]>,<@animals>:2
!03=<[Animal Name]>,<@animals>:3
!04=<[Animal Name]>,<@animals>:1000000
!05=<[Animal Name]>,<@animals>:$FF
!06=<[Animal Name]>,<@animals>:0
!07=<[Animal Name/Colour]>,<@animals>:1,<@colours>:1
80=main

@animals
03=[cat]
04=[dog]
05=[parakeet]
06<2>=[elephant]
!07=<[Dog Breed]>,<@dogbreeds>:1
!08=<[Dog Breed]>,<@dogbreeds>:1+
!FF=<>,-1

@colours
00=

@dogbreeds
00=[Labrador]
01=[Chihuahua]
and then the bytes 02080080 would also correspond to the text "[Animal Name][Dog Breed][Labrador]main", where "[Labrador]" gets counted as a match against both the "animals" -> "dogbreeds" switch and the "main" -> "animals" switch.

Maybe I return to the idea of grinding for all the 1/256 rare drops that we ultimately need the Japanese text from.
Instead of spending hours grinding for rare drops from multiple rare enemies, some other approaches include finding where your party's items are stored in RAM and editing the desired items into your inventory, finding the drop rate data and pumping all the drop rates up to the maximum, or finding the code that decides whether you get a drop or not and forcing it to always give a drop. Actually obtaining the items in-game gives you the benefit of seeing the mystery strings in more context than a text dump gives, so you can observationally verify which items use which strings rather than having to wade through the code or make questionable assumptions such as the order of item text being in any way related to the order of the items.


As for my little update, it turns out that DW3 uses 8 different control codes for various English monster name pluralization rules, but 1 of them is a "do nothing" rule and 3 of them are wrong, so really it only needs 4 codes. For translation, however, it took me a fair bit of work to get down to 14 codes... looks like I've got some more ASM work in my future!
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on December 07, 2019, 12:35:14 pm
@abw

Thank you as always. I wanted to provide a different kind of update. I've been taking some time off from working specifically on the steps I'm at with this project in order to focus on learning the basics of 6502. I know I've said that for awhile, but I've finally made it an element of my schedule where I spend some time doing learning and exercises every day. I'm very surprised to find that if I had a shit day or even had a few drinks that evening, the daily consistency still allows me to  continue to make distinct progress. Virtually all the problems we've been facing lately have required me to have some kind of ASM / debugging knowledge so I thought it was time to make that the focus. It's not like I have more than 3 people clamoring for a release date of DQ3 Delocalized anyway.  :laugh:

I'm sure I'll have questions in the future, but hopefully they will be at a higher level and I won't require these 3 page answers.

See ya'll soon.  :beer:
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on December 08, 2019, 09:45:56 pm
No worries! Glad to hear the 6502 learning is progressing well - as with most things in life, practice brings improvement :thumbsup:.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on January 07, 2020, 10:09:24 pm
Okay big update from me. After a bunch of frustration and procrastination, I figured out and fixed the kinks that were giving me trouble.

If some newbie romhacker (or maybe just new to abcde) is reading this, then you might want to give this a nice long read!

One thing I was missing before that I stupidly thought atlas/cartographer/abcde handled by itself before was adding #JMP commands before every block. Looking at an unaltered ROM I was noticing that the Prologue was being inserted directly AFTER the menu text. So yeah #JMP($Address) or #JMP($Start Address,$End Address) is very important!

Okay now maybe I messed up somewhere and this works perfectly fine for abw and anyone else, but I was still getting an error message that there was a duplicate table ID, @main. Again, if there's newbie romhackers using this, make sure your atlas.bat has all the tables you need! I double checked and the only table I had that was labelled @main was dw2.tbl. I fixed it and found success by changing the table used in various menus (directions, names, etc) from dw2.tbl to menu.tbl and editing menu.tbl to add values FA=[End-FA] and FF=[End-FF]. Worked like a charm!

Then there was the Prologue. First I was getting instances where it just completely skipped the text and now it was hanging on a blank screen doing nothing. Drove me nuts, I thought maybe my lines were too long or something.

abw I love you, but I think your spacer.tbl is wonky.

I think there's a genuine typo in your DW2 Cartographer.txt as well. After comparing an untouched ROM with one that's been altered I FINALLY noticed that Atlas/abcde was changing the pointer at $1CAB2 from 8A to 0A which is silly because there's only one string in the Prologue and I'm not changing its location. Now correct me if I'm wrong, but I think this is because the base pointer in cartographer.txt is listed at 14010 when it should be 1C010. At least that's the only reason I can think of that the pointer value is 8000 lower than it should be. Have to admit, I've figured that part out, but don't know how to fix it, I'm just manually editing $1CAB2 after I do an insertion.

Okay so back to spacer.tbl. Once I figured out the pointer problem I was getting text in game, but it would freeze when it got to my third line in the first stanza. It's supposed to be this

Code: [Select]
Once upon a time, a young hero
  descended from the legendary
  warrior Erdrick defeated the
dreaded Dragonlord, and restored
 peace and light to the world.

Yeah those are some long lines, I figured even if you have to do a total of 32 lines (number left spaces*2 + number of letters in a line)

I tried cutting it down just for testing sakes and got this

Code: [Select]
       Once upon a time,
         a young hero         
       descended from the
    warrior Erdrick defeated
      dreaded Dragonlord
    and restored peace and
      light to the world.

And what's weird, this is how it looked in game

(https://i.imgur.com/esKiWQc.png)

And I looked in the ROM and instead of having $07 for 7 spaces it put in $0F and $5F. I mean this doesn't mean I'm stuck I'll probably just use Pointer Tables since I don't have to worry about Pointers here, but I figured you should know.

Also, thanks Chicken Knife for giving me your latest Atlas.txt you used. I thought maybe you had a working "complete" version (menus, names, and everything) so I could see what was wrong with mine, but it was still helpful.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Chicken Knife on January 08, 2020, 10:08:29 am
I actually have all the menu stuff on a separate atlas file. I really only needed that for one purpose (fixing SHILD) so I didn't bother combining it all together. I can send you the separate file tonight.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on January 08, 2020, 11:08:01 pm
One thing I was missing before that I stupidly thought atlas/cartographer/abcde handled by itself before was adding #JMP commands before every block.
Good news, I've made some upgrades for the eventual v0.0.6 release - the Cartographer code will now output #VAR/#ADDTBL/#ACTIVETBL and #HDR/#JMP commands for you!

Again, if there's newbie romhackers using this, make sure your atlas.bat has all the tables you need!
v0.0.6 might also help with this - there was a line buried in the abcde::Atlas help text about how you were supposed to load all your table files on the command line instead of inside the Atlas command file, but I've cleared that up now.

I fixed it and found success by changing the table used in various menus (directions, names, etc) from dw2.tbl to menu.tbl and editing menu.tbl to add values FA=[End-FA] and FF=[End-FF]. Worked like a charm!
Be careful with this - if you try inserting 2 or more adjacent spaces with the menu table, you'll get the menu's $82 - $87 control codes, which will quite possibly result in a mess when non-menu code encounters them.

abw I love you, but I think your spacer.tbl is wonky.
;D

I think there's a genuine typo in your DW2 Cartographer.txt as well. After comparing an untouched ROM with one that's been altered I FINALLY noticed that Atlas/abcde was changing the pointer at $1CAB2 from 8A to 0A which is silly because there's only one string in the Prologue and I'm not changing its location. Now correct me if I'm wrong, but I think this is because the base pointer in cartographer.txt is listed at 14010 when it should be 1C010.
For the prologue, I've got #JMP($1CAC2, $1CC22) and #HDR($14010) in both my translation's actual insert script and abcde's example, and both of those commands are correct. The value of the pointer at $1CAB1 is $8AB2, and $8AB2 + $14010 = $1CAC2. I also get $8AB2 as the pointer value when running abcde's example insert script, so I think something else is going on here. Any chance you could send me a link to your insert script so I can take a look?

I tried cutting it down just for testing sakes and got this
I notice your quote has a bunch of trailing spaces on the second line ("hero         "); my guess is that those are probably what's causing the issues on the third line. It looks like the prologue display code might only handle 30 bytes per line, so 32 bytes could cause some weirdness - what happens if you take those extra spaces out?

I thought maybe you had a working "complete" version (menus, names, and everything) so I could see what was wrong with mine, but it was still helpful.
I probably had a complete insert script when I started my translation, but it got cannibalized along the way and has been altered to work with all the ASM changes I made, so it doesn't really apply to the original script very well any more.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: Choppasmith on January 10, 2020, 12:56:39 pm
Good news, I've made some upgrades for the eventual v0.0.6 release - the Cartographer code will now output #VAR/#ADDTBL/#ACTIVETBL and #HDR/#JMP commands for you!

Nice! That'll be a huge help!

Quote
Be careful with this - if you try inserting 2 or more adjacent spaces with the menu table, you'll get the menu's $82 - $87 control codes, which will quite possibly result in a mess when non-menu code encounters them.

Well the only thing I really need dw2.tbl for anyway is stuff like Crests and Moonbrooke/Cannock names, and those can just be edited manually if need be, but I'll keep a lookout thanks!

Quote
;D
For the prologue, I've got #JMP($1CAC2, $1CC22) and #HDR($14010) in both my translation's actual insert script and abcde's example, and both of those commands are correct. The value of the pointer at $1CAB1 is $8AB2, and $8AB2 + $14010 = $1CAC2. I also get $8AB2 as the pointer value when running abcde's example insert script, so I think something else is going on here. Any chance you could send me a link to your insert script so I can take a look?
I notice your quote has a bunch of trailing spaces on the second line ("hero         "); my guess is that those are probably what's causing the issues on the third line. It looks like the prologue display code might only handle 30 bytes per line, so 32 bytes could cause some weirdness - what happens if you take those extra spaces out?

Okay, yeah I did have some extra spaces there and it worked fine when I removed them (I think I was going to add those spaces for trial and error, then decided against it and forgot. Typicial). I noticed I was still getting some freezing, and noticed it was just spacer.tbl messing things up. Some lines it would put a space ($5F) instead of $FF at the end of a line and the game NEEDS that byte for the number of spaces ($01-$0F) and as said above it seems Atlas/abcde seemed to misinterpret some of those preceding spaces as regular ($5F) spaces. Also, worth noting, when you get to around 28 characters in a line (not including the spaces) things will flicker a bit.

Also, here's my atlas.txt
http://www.mediafire.com/file/o0hmhps1m4ss5oa/Atlas%25282%2529.txt/file

I'm really curious what's causing that Prologue pointer to be changed.
Title: Re: Dragon Warrior 1, 2 & 3 Hacking Discussion
Post by: abw on January 11, 2020, 09:59:50 pm
Nice! That'll be a huge help!
Yeah, it's slow progress in between other things, but at least it is progress!

I noticed I was still getting some freezing, and noticed it was just spacer.tbl messing things up.
The freezing appears to be due to trying to insert lines longer than 30 bytes - it seems like the game really doesn't like that. I haven't spent much time looking at the prologue code, so I'm not sure what its actual limitations are.

Some lines it would put a space ($5F) instead of $FF at the end of a line and the game NEEDS that byte for the number of spaces ($01-$0F) and as said above it seems Atlas/abcde seemed to misinterpret some of those preceding spaces as regular ($5F) spaces.
I wasn't able to reproduce this myself, but I see that our table files have diverged a fair bit, so that isn't terribly conclusive evidence, especially if you were using the menu table when inserting the prologue.

Also, worth noting, when you get to around 28 characters in a line (not including the spaces) things will flicker a bit.
Yeah, the original game keeps its line lengths down to 28 bytes including spaces, 24 excluding. My translation seemed okay at 29 bytes including spaces, 26 excluding. I was getting some flickering with your "One hundred years passed..." line; you might want to try splitting/shortening that one.

I'm really curious what's causing that Prologue pointer to be changed.
You're missing the #HDR($14010) command to adjust the pointer calculations for ROM bank 7, which means the #HDR($-3FF0) is still active; $1CAC2 - $-3FF0 = $20AB2, which has $0AB2 as its low 2 bytes.