Romhacking.net

Romhacking => Personal Projects => Topic started by: matal3a0 on February 12, 2016, 04:47:05 pm

Title: Zelda2MapEdit
Post by: matal3a0 on February 12, 2016, 04:47:05 pm
Hi all!
I'm working on a cross-platform overworld editor for Zelda II - The Adventure of Link.

If anybody wants to try it out, it's all on github: https://github.com/matal3a0/Zelda2MapEdit
Title: Re: Zelda2MapEdit
Post by: Grimlock on February 17, 2016, 08:55:30 am
Sounds interesting, do you see any way to add additional tiles to the overworld?  If I could have a full sheet of tiles to work with on the overworld along with separate tile sheets for the overworld encounters, towns, and palaces I'd be all in on making a full scale level design and graphics hack of Zelda 2.

From what Trax told me in the past it would be fairly complicated though.
Title: Re: Zelda2MapEdit
Post by: Trax on February 17, 2016, 10:09:50 pm
I don't know exactly how to compile and run Python code, so I can't see the result on screen. However, I guess it's a good start. Matal, can you post a screenshot to see the progress in visual form?

I skimmed through the code, and there are several places where you could loop the process instead of doing everything sequentially. Also, you should never refer to your tiles as "Palace" or "Desert", and always use the numeric value. Use words only when needed, like when you want to actually display the word on screen. Use indexes as much as possible...

Grimlock, I may have found an interesting solution on the matter of limited tiles. I'll email you on that...
Title: Re: Zelda2MapEdit
Post by: Grimlock on February 17, 2016, 10:21:54 pm
You should never refer to your tiles as "Palace" or "Desert", and always use the numeric value. Use words only when needed, like when you want to actually display the word on screen. Use indexes as much as possible...

I would have to agree with Trax on that point.  You never know how extensive the end users project may end up being.  My approach would be to toss all the existing graphics out the window and start out fresh.  The original "Desert" tile could end up being repurposed to virtually anything. 

Grimlock, I may have found an interesting solution on the matter of limited tiles. I'll email you on that...

Very cool!  Imagine how nice the overworld could be made to be with the proper resources!
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 18, 2016, 06:19:56 am
Thanks for the input guys. I've uploaded a screenshot on https://github.com/matal3a0/Zelda2MapEdit/blob/master/screenshot01.png.
If you want to run it, just install python from https://www.python.org/downloads/windows/ and run the code directly.
I'm aware of the places where the code can be looped instead, it's on the todo-list ;) Bad choice of data structure from the beginning. I just haven't settled on a new structure for it.

Agree on the naming of the tiles, in the big picture it would probably be better to not use names. But it's never presented to the user of the editor in any way. And if the user has hacked the tileset, then the images of the tiles would be wrong as well. It will have to do for now I think.

Currently putting my energy into being able to move Palace 6 and the location where you play the flute to bring it out of hiding.



Title: Re: Zelda2MapEdit
Post by: Dr. Floppy on February 18, 2016, 05:24:56 pm
Can you make it so the "lava" squares don't shunt into "Grassland" battle zones in the Western hemisphere?
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 19, 2016, 02:05:35 am
Not sure what you mean with: so "lava" squares don't shunt into "Grassland" battle zones
Probably some "feature" I've overlooked, could you elaborate? Or perhaps show an example?


Also, just created a first demo. Latest commit introduced the ability to "paint" the tiles on the map, instead of click once for each tile. This makes editing much easier. Just hold down the left mouse button and move.

https://youtu.be/Oco615fPNn0
Title: Re: Zelda2MapEdit
Post by: Dr. Floppy on February 19, 2016, 06:05:44 pm
Not sure what you mean with: so "lava" squares don't shunt into "Grassland" battle zones
Probably some "feature" I've overlooked, could you elaborate? Or perhaps show an example?

The reddish terrain on the road to the 7th Palace = "lava". When those squares are used on da West$ide, enemy collisions default to the "grass" encounter map.
Title: Re: Zelda2MapEdit
Post by: Trax on February 20, 2016, 05:35:21 pm
Matal, the link to your screenshot is broken. But I saw your video. Looks pretty neat...

My editor takes the graphics directly from the ROM, so whatever changes were made to the graphics in a hack, it reflects in the editor. But I also added an option to show the tiles using the original graphics...

Palace 6 is troublesome for map editors, because there's more to it than just the Overworld data. Its Y location is set to 0 initially and its X location is set to wherever you want it to be. When you play the Flute, the game checks if you have the correct X/Y position (hard-coded), and then sets the Y position of the Palace's Key Area to a hard-coded value in code. The game also manually sets the tiles for the palace to replace the desert square there. You still need to have a strip of terrain with 0 length, or else your map becomes shifted. For desert, the byte would be 04. In the original game, because the type of terrain is desert on the left and the right as well, you have to force a single unit of desert to be created there. Any editor must take this into account. I made it by implementing a tool called "Bracket" tool. With this, you can "cut out" a strip of the same terrain into multiple strips. In other words, you tell the editor to have, for example, two strips of 4 units instead of one strip of 8 units...

The place in the code where you can find the position of the Palace 6 Call Spot is in bank 2:

Y Position: 8372, value 64
X Position: 8378, value 2D


The Palace appears two tiles south of the Call Spot. According to the type of tile there, you will get:

0E -> 04   Rock   -> Desert
0F -> 04   Spider -> Desert
04 -> 02   Desert -> Palace
06 -> 00   Forest -> Town


On the Lava question. The original game didn't have any Lava tile in West Hyrule, so they didn't bother to work on the areas for that type of tile. The data table is in the region's corresponding bank, i.e. Bank 1 for West Hyrule and bank 2 for East Hyrule. Here's the reference for West Hyrule:

4409: Code values for Random Battle Areas (7 * 2 = E bytes)

North-West Hyrule / South-West Hyrule

5D 5E   Desert
62 63   Grass
67 68   Forest
6F 70   Swamp
74 75   Graveyard
79 7A   Road
7B 7C   Lava

xx.. ....   Enter Code (1 = Middle of Screen 1)
..xx xxxx   Area Code

5D 5E -> 1D 1E
62 63 -> 22 23
67 68 -> 27 28
6F 70 -> 2F 30
74 75 -> 34 35
79 7A -> 39 3A
7B 7C -> 3B 3C
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 21, 2016, 08:18:12 am
Wow, lots of info even without asking, thanks Trax!
Screenshot fixed..

I have been thinking of reading the graphics directly from the rom as well. But I skipped this since I wanted to get started with editing, and then I forgot about it. It's a nice feature to have, I'll try to implement it in the future.
If I would like to get started on extracting the graphics, any hints on documentation on this?

Thanks for the info about Palace 6, this confirms my findings. Perhaps I'll implement something similar, or perhaps I will make it find the location of the palace, and automatically insert a single tile there.

Too bad I don't have a Mac available, would really like to try your editor..

Edit:
I have looked at the code about the spawn-location for palace 6. But in all my roms of Zelda 2, the values are in another place than in your documentation:
8382: 64
8388: 2D

From hexeditor:
00008370   0E C8 BD 56  83 91 0E 60  AD 06 07 C9  02 D0 1F A5  ...V...`........
00008380   73 C9 64 D0  12 A5 74 C9  2D D0 0C A9  0B 8D 25 07  s.d...t.-.....%.
00008390   E6 01 E6 01  4C 79 DF AD  63 05 C9 0F  F0 01 60 20  ....Ly..c.....`
Title: Re: Zelda2MapEdit
Post by: Turambar on February 21, 2016, 04:50:24 pm
You'll have to account for whether or not the rom has a header.
A header adds 16 bytes to the beginning of the rom.
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 22, 2016, 02:33:34 am
Ok, that makes sense. Which would mean that the documentation of the locations for palaces, caves etc. on http://datacrystal.romhacking.net/wiki/Zelda_II:_The_Adventure_of_Link:List_of_areas refers to a rom with header, since all those addresses match?
Title: Re: Zelda2MapEdit
Post by: Trax on February 22, 2016, 06:36:58 pm
I can't speak for anyone, or anything, but I always use real addresses, so that it corresponds to the addresses in code. I just add the 0x10 offset instinctively. In an hex editor, it usually means to simply move the cursor one line down, assuming a line is 0x10 bytes...
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 26, 2016, 07:10:03 am
Still working on palace 6..
I now have a single desert tile written in correct position on the map, and the palace spawns when I play the flute. But the map is still shifted south of the palace.

I made a comparison of the original (left) and my encoding (right) here https://www.diffchecker.com/t6f56lz1
As you can see there is no difference in the encoded map at the end, just some higher up.
Line 73 is where the palace is.

screenshots: https://www.dropbox.com/sh/mcjt1m7mv0cjhmy/AADa5wFSgzzRvtcPhj0W8rAwa?dl=0

Anyone have an idea why it behaves like this?
Title: Re: Zelda2MapEdit
Post by: Trax on February 26, 2016, 11:48:30 pm
The most probable explanation is that when you save, the resulting data is not the same as the original, even if you didn't modify anything. The original map data is not "optimal". There are various spots where a strip is cut into 2 parts, and end up taking 2 bytes of map data. So, obviously, if you presume the Palace 6 Spot will be at a specific offset in ROM, then you will not get it right...

That's why it's better to implement brackets. If you don't, then the only "simple" alternative for the end user would be to set an arbitrary single unit of something else in the desert at the right coordinates, so to force the editor to save an extra byte at that spot. Then, manually change the unit there by a unit of Desert (04) using an hex editor. You also get the same problem with Hidden Town of Kasuto, where you have to "force" a single unit of Forest...
Title: Re: Zelda2MapEdit
Post by: matal3a0 on February 29, 2016, 07:08:18 am
Yup, I have discovered that the original map isn't "optimal". This is clearly seen in the diff that I shared earlier.
I made one mistake, the total number of bytes didn't match the original map. I made a new edit of the map that has the same number of bytes, and the shift is gone, yay!
But.... instead of a desert at the spot, I get a mountain!?

Looking in my hexeditor, I have ..54 04 74.. in the rom, which means a single desert tile in the middle there.
But when I look at the map contained in memory, it gets changed to 54 0B 74, as if the palace has been completed, which is hasn't..
And this is even before the palace gets called with the flute.  :banghead:
Title: Re: Zelda2MapEdit
Post by: Trax on February 29, 2016, 11:11:29 pm
This is strange. The code states that you need both the palace's item and the crystal placed for the palace to turn into stone. Do you know how to use FCEU's debugger? If you do, set a breakpoint at the exact location of the 04 value which is supposed to be the palace's location, and report here the place in code where the break occurs. If not, then contact me by email and I'll take a look at the modified ROM and see what's wrong...

In the meantime, try this. Take note of the offset of the 04 value in ROM (from 9046), and add it to 7C00. Then, go to 8791 in the ROM. The bytes there should be 08 7F. Replace these bytes by the little-endian version (reverse the two bytes) of your value (which should be somewhere around 7Exx or 7Fxx). Then test again...
Title: Re: Zelda2MapEdit
Post by: matal3a0 on March 01, 2016, 02:30:46 am
Definitely related to the code which turns palaces into stone. Changed the offset for the rock placement over palace six, and now I get a rock next to the palace instead =)
I can confirm this with FCEU debugger.. I'll do some more testing with a fresh rom, perhaps I messed up something else.
Really appreciate all the help!
Title: Re: Zelda2MapEdit
Post by: matal3a0 on May 04, 2016, 03:47:56 am
Hi all, it's been a while..

Just pushed an update to github. I've implemented a bracketing-tool which I just call a breakpoint-tool. Now I can put single tiles for Palace 6 and New Kasuto. When loading a rom, the breakpoints present in the maps are automatically detected and placed in the editor.

The bug with Palace 6 being a mountain I was chasing before doesn't appear anymore. Could possibly be because of some cheat code I accidently used when testing.

The editor is now actually quite usable ;)

Now I'll move on to enabling moving Palace 6 and New Kasuto.
Title: Re: Zelda2MapEdit
Post by: itemdrop on May 16, 2016, 11:25:01 am
I actually played around with it for about 4hr yeasterday. i really enjoyed it. the only issue ive had which is a minor one is over a little bit of time the program starts to become very sluggish. a lot of delay to inputs. So you have to close the program and restart it. I dont mind at all dont want to sound like im complaining. i think this is a great tool. thanks for creating it.
Title: Re: Zelda2MapEdit
Post by: matal3a0 on May 17, 2016, 06:37:12 am
Thanks for the feedback itemdrop! I haven't had that much time playing around with it myself, been busy coding =)
Can you give some more details? After how long time or how much editing do you have do to before you experience the slowdown? Running on windows, linux?
All feedback is appreciated!
Title: Re: Zelda2MapEdit
Post by: Jeville on May 17, 2016, 03:34:35 pm
It doesn't slow down by time, but by how many changes are made. Non-stop tile placement clicking for a few minutes (under 5) will slow the process by a second. I was clicking on the map enough times yesterday for it to slow down to 3 seconds per click lol. Like the other poster, I don't mind restarting the editor to fix it. I will spend a lot of time with it and thanks for this. :) I'm on Windows.
Title: Re: Zelda2MapEdit
Post by: itemdrop on May 17, 2016, 11:19:53 pm
im on windows 8.1 and yeah as poster above says Non-stop tile placement clicking for a few minutes will slow the process by a second. i think i got to 4 seconds yesterday. i used it for over 4 hrs.
Title: Re: Zelda2MapEdit
Post by: matal3a0 on May 18, 2016, 02:24:08 am
Thanks, I'll look into it.

Edit:
wow, it gets really slow on my win7 machine. But in my linux machine (which is a virtual machine running under the same win7), it's not that slow at all..

Edit2:
Ok, just pushed an update to github. I have removed some code, now everything seems faster. But now there is a slight delay before locations and breakpoints are drawn on the map again.
Please give it a try and see if you can live with this..
Title: Re: Zelda2MapEdit
Post by: Jeville on May 22, 2016, 12:46:22 am
Thanks, it's a lot better now. Does it matter whether I use the last tile choice or not? I don't really know its function.
Title: Re: Zelda2MapEdit
Post by: matal3a0 on May 22, 2016, 01:40:39 pm
I'm not really sure what you mean by last tile choice?
Title: Re: Zelda2MapEdit
Post by: itemdrop on May 22, 2016, 06:12:21 pm
I think he means the grey box with the red lines, I beileve.
Title: Re: Zelda2MapEdit
Post by: matal3a0 on May 24, 2016, 07:52:39 am
Ah, that's the "breakpoint tool". It's used to divide strings of the same tile into multiple strings.

For example 16 mountains in a row: BBBBBBBBBBBBBBBB would be encoded as FB (one byte) in the rom.
If a breakpoint is inserted at position 5, it would become two strings BBBB + BBBBBBBBBBBB, which will be encoded as 3B BB (two bytes).

This is needed for Palace 6 and New Kasuto to function properly. They need a single tile of desert and forest respectively.

The original map also contains several of these breakpoints, the encoding isn't perfect.
My algorithm finds these when loading the maps and places them automatically. This enables you to load an original map, save it, and the bytes are precisely the same in the rom =)

Perhaps I should move that button further from the other buttons.

Edit: Just pushed an update to github. I added some labelframes to organize the buttons a little bit.
Title: Re: Zelda2MapEdit
Post by: Grimlock on June 03, 2016, 12:18:26 am
Njosro at Board-2 is working on a similar project.  I bet there's a lot of info the two of you could share with each other:

Link to his thread:
http://acmlm.kafuka.org/board/thread.php?id=8227&page=1 (http://acmlm.kafuka.org/board/thread.php?id=8227&page=1)
Title: Re: Zelda2MapEdit
Post by: matal3a0 on June 03, 2016, 01:57:40 am
Yup, already in contact with him, thanks!
Title: Re: Zelda2MapEdit
Post by: njosro on June 03, 2016, 09:20:59 am
Yeah we've been messaging each other for a little while  :laugh:
The current task is figuring out the hidden palace and new kasuto.
For files with header:
0x01df78: the y value to place hidden palace (warp spot) when it appears
0x01df79: the y value to place new kasuto (warp spot) when it appears

These are only for the area warps; they don't affect the visual tiles on the map.

I'll put everything I've got for reading in area information for posterity.

For the hidden palace and new kasuto, I had to find everything on my own.
Information about the area warps: http://datacrystal.romhacking.net/wiki/Zelda_II:_The_Adventure_of_Link:ROM_map#Overworld_Areas

You know how you have the locations of each area's X and Y values? There's two more bytes to go along with them as you will see in the link above. And it's not 1 byte = 1 piece of info. They went and squished multiple pieces of information into each byte.

So, to loop through the areas, my code looks like this: (this is GML)

Code: [Select]
//load the action areas
/* REMINDER NOTES from datacrystal and me!
byte0: .xxx xxxx = Y position
       x... .... = External Flag (used in byte3)    (This means that the area takes you to a palace or town)
byte1: ..xx xxxx - X position
       .x.. .... - right exit flag                              (This is the bit for the Right Exit checkbox in my editor)
       x... .... - upper exit flag                            (This is the bit for the Upper Exit checkbox in my editor)
byte2: ..xx xxxx - Scene number
       xx.. .... - Horizontal position to enter within scene
       00 = enter from the left
       01 = enter at x=256 or from the right for 2 screens scenes
       10 = enter at x=512 or from the right for 3 screens scenes
       11 = enter from the right for 4 screens scenes
byte3: ...x xxxx - World number
       ...0 00xx - (requires External) Warp to same numbered exit in overworld #xx
       ...x xx.. - (requires External) Warp to overworld, town, town2, palace, palace2, great palace
       ..x. .... - Forced enter from the right edge of screen
       .x.. .... - Pass through
       x... .... - Fall in hole
*/
for (region=0;region<4;region+=1) //region is 0=west hyrule, 1=death mtn, etc
{
    if argument1=0 //argument 1 is whether to load from file or just reload from data structure
        break
    ds_map_destroy(area_raws[region])
    area_raws[region]=ds_map_create()
    ds_map_destroy(area_locations[region])
    area_locations[region]=ds_map_create()
   
    switch region
    {
    case 0: pos=$462F; stop=$4665 break; //($ is GM notation for hex value.)
    case 1: pos=$610C; stop=$6149 break;
    case 2: pos=$8627; stop=$8665 break;
    case 3: pos=$A10C; stop=$A149 break;
    default:pos=$462F; stop=$4665 break;
    }
    Ybegin=pos //byte 0
    Xoffset=$3F //byte 1
    Moffset=$7E //byte 2
    Woffset=$BD //byte 3
    i=0
    data=''
   
    while (pos<=stop) //shove all the raw data into the data structure area_raws
    {
        file_bin_seek(file,pos) //byte0 data
        data=dectobin(file_bin_read_byte(file),8) //(dectobin: arg0: decimal value, arg1: number of bits to make it as a string)
        file_bin_seek(file,pos+Xoffset) //byte1 data
        data+=dectobin(file_bin_read_byte(file),8)
        file_bin_seek(file,pos+Moffset) //byte2 data
        data+=dectobin(file_bin_read_byte(file),8)
        file_bin_seek(file,pos+Woffset) //byte3 data
        data+=dectobin(file_bin_read_byte(file),8)
       
        ds_map_add(area_raws[region],i,data) //add to data structure
        i+=1 //where in map to store data
        pos+=1 //where in file to retrieve bytes
    }
    ds_map_destroy(backup_area_raws[region])
    backup_area_raws[region]=ds_map_create()
    ds_map_copy(backup_area_raws[region],area_raws[region]);
   
    i=0
    pos=Ybegin
    while ds_map_exists(area_raws[region],i) //store the y-x coordinates as 13 bit binary strings. y first, then x
    {
        data=string_copy(ds_map_find_value(area_raws[region],i),2,7) //extract y pos in binary
        data+=string_copy(ds_map_find_value(area_raws[region],i),11,6) //extract x pos in binary
        ds_map_add(area_locations[region],i,data) //'yyyyyyyxxxxxx'
        i+=1
    }
}

Then it reads from the area_raws hash map to know where to place the white squares.

So basically there's the y byte, x byte, m byte, and w byte. The x byte is $3F from the y byte. The m byte is $7E from the y byte, and the w byte is $BD from the y byte. All the time.