News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Memory Map Visualising  (Read 2044 times)

RyanfaeScotland

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
    • My Brill Game Site
Memory Map Visualising
« on: December 27, 2015, 01:41:22 pm »
Hey folks,

I'm in the process of revamping a section of my site in preparation for a large, long term project and wanted to get your thoughts on the best way to visualise ROM / RAM memory maps along with a bit of feedback on what I've went with so far (if you would be so kind!).

You can see what I've went with on the site here: http://www.mybrillgamesite.com/projects/?id=86 and I like to think it is pretty self explanatory. The ROM: section is the section I have updated, the Savestate: section is how it originally looked.

Down the line I'll be adjusting the Savestate section as well to be the RAM section, adjusting the offsets as a result, giving it similar headers and functionality along with the ability to show / hide the offsets adjusted for savestates created in different emulators (currently they show only GENS).

The layout is based on what I think is useful to have and takes inspiration from the layouts I've seen at SonicRetro, on the Super Mario World disassemblies and in other people's hacking notes.

Nothing is set in cement yet and I'm still not sure about peripheral things like the borders I'm using, what other items will appear in the key, what exactly will appear in descriptions, so if you see anything at all you think is missing / would be useful please let me know.

Note also that the 'Opcode / Hexcode' column is intended to be 8 bytes wide because I think that's the longest a typical 68K instruction is, isn't it? I know it can be longer with pre and post statements but in practice I should have enough width to display all code there, right?

Links to other sites with good examples also welcome and any questions please ask.

Note - The section marked 'Unknown' is still to be populated. I've got the disassembly here and want to get it displayed line by line once the section is expanded. Should be up in the next few days.
« Last Edit: December 27, 2015, 01:59:00 pm by RyanfaeScotland »

FAST6191

  • Hero Member
  • *****
  • Posts: 3022
    • View Profile
Re: Memory Map Visualising
« Reply #1 on: December 27, 2015, 02:44:44 pm »
Assuming you do not mean some kind of graphical colour map a la https://windirstat.info/images/windirstat.jpg or I guess what https://www.youtube.com/watch?v=rpMpmm0nlEM was doing I approach it on a case by case basis.

I lean slightly more towards nested and relative approaches, absolute position in a file has its uses but if it is a self contained array or something then it will win its own spot and be considered by itself. I am sure if you went through my post history it will tell a different story but hey.

I am not the greatest fan of doing things like http://kiwi.ds.googlepages.com/sdat.html though if you are programming something I can see it being better than something like you linked or http://llref.emutalk.net/docs/?file=xml/ncer.xml

On the site the table code could possibly be cleaner but it is by no means bad. The collapsing stuff does not work if javascript is disabled, however it seems you can not even disable javascript any more on many browsers (in this case it was noscript that did it for me) so you can blame that on me. I would rather have a lot of scrolling than miss data though.
I am not a fan of picture navigation but you have at least got all the picture alt text in there.
The embedded videos and projects use url schemes/formats like http://www.mybrillgamesite.com/videos/?id=101 where I would have chosen something more memorable/search friendly for the URL. Said videos despite being in youtube also seem to want flash, more and more people and sites are going flashless these days.
Your mobile browser support is not broken but not there by the looks of things (granted I only messed around on http://mobiletest.me/iphone_5_emulator/?u=http://www.mybrillgamesite.com/hacking/ for a little while). To be fair you do say best viewed on but that is about as useful in 2015 as a dripping blood banner and a purple stars background and died at about the same time as those things mentioned.
Nothing in the page footer as far as navigation? That is stylistic at best though.
I have to say ew joomla.

RyanfaeScotland

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
    • My Brill Game Site
Re: Memory Map Visualising
« Reply #2 on: December 27, 2015, 03:41:06 pm »
Hey FAST6191 thanks for the reply, I was only really looking for feedback on the memory map section so extra thanks for the additional pointers across the rest of the site!

Here's a little response to your comments, I still have to check out the sites you linked to but will need to wait till the kids are in bed for a proper look.

Assuming you do not mean some kind of graphical colour map a la https://windirstat.info/images/windirstat.jpg or I guess what https://www.youtube.com/watch?v=rpMpmm0nlEM was doing I approach it on a case by case basis.

Actually part of having the colour coded sections is so that it can be viewed like a colour map. Expand all sections then zoom your browser out (CTRL Minus on most browsers with CTRL 0 to return to 100%) and it should give a good indication of where large sections have been worked on, need worked on and so on. Obviously it'll make a lot more sense once the rest of the data is in and I'm interested to see myself just how clear it is when expanded and zoomed out.

I lean slightly more towards nested and relative approaches, absolute position in a file has its uses but if it is a self contained array or something then it will win its own spot and be considered by itself. I am sure if you went through my post history it will tell a different story but hey.

I am not the greatest fan of doing things like http://kiwi.ds.googlepages.com/sdat.html though if you are programming something I can see it being better than something like you linked or http://llref.emutalk.net/docs/?file=xml/ncer.xml
Yea, even as work progresses things will be pretty dynamic for the layout and as I come across data structures I'll see how it looks in the current fixed format and probably play with separating them out as well. Agreed, I don't like the first link although the second one is pretty neat.

On the site the table code could possibly be cleaner but it is by no means bad. The collapsing stuff does not work if javascript is disabled, however it seems you can not even disable javascript any more on many browsers (in this case it was noscript that did it for me) so you can blame that on me. I would rather have a lot of scrolling than miss data though.
I am not a fan of picture navigation but you have at least got all the picture alt text in there.
The embedded videos and projects use url schemes/formats like http://www.mybrillgamesite.com/videos/?id=101 where I would have chosen something more memorable/search friendly for the URL. Said videos despite being in youtube also seem to want flash, more and more people and sites are going flashless these days.
Your mobile browser support is not broken but not there by the looks of things (granted I only messed around on http://mobiletest.me/iphone_5_emulator/?u=http://www.mybrillgamesite.com/hacking/ for a little while). To be fair you do say best viewed on but that is about as useful in 2015 as a dripping blood banner and a purple stars background and died at about the same time as those things mentioned.
Nothing in the page footer as far as navigation? That is stylistic at best though.
I have to say ew joomla.

JavaScript is one of the most useful things on the web. If people want to make rods for their own back by turning it off I'm not in the business of making the rods more flexible for them.

Yup alt text and my own captions.

My tables are beautiful. :)

The URLs are something on my todo list to fix. A bi-product of the move to Joomla sadly.

That's interesting about the flash issue, I'll look into getting that changed as well.

Given the content of my site I doubt people will be using it on their phone but if they are it renders well on Android, I'll have a look at iPhone out of interest but it's low priority to me.

Nope, footer is comments and site isn't big enough IMHO to need secondary navigation.

Yea, Joomla, I moved it to there as my work was migrating their site to it and I wanted a practice run with it first. It isn't actually that bad but I doubt I use it enough to have made the move worth while.

December 30, 2015, 10:08:38 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Ok, being able to read and write assembly is awesome I really don't know why I didn't commit myself to this sooner!

Here are my original notes on two pointers in Toejam & Earl (T&E from now on):

Quote
01:590C Subtracts 1 from the length of time left for tomatoes
01:5914 Starts the tomatoes flashing if the length of time (at D0) is less than 300 (012C)

And now that I understand what is actually happening, have improved my process with GENS Tracer, KMOD and Hacking versions I've now got the following notes on the same section:

Quote
*
* From around this address controls the length of time left for presents
*

000158F0 3002                     MOVE.W    D2,D0
000158F2 48C0                     EXT.L     D0
000158F4 207C 00FFDE50    MOVE.L    #$00FFDE50,A0
000158FA 4A30 0800            TST.B     $08(A0,D0.W)
000158FE 588F                      ADDQ.L    #$4,A7
00015900 6C4C                     BGE       $0001594E
00015902 6048                      BRA       $0001594C
00015904 3002                      MOVE.W    D2,D0
00015906 48C0                      EXT.L     D0
00015908 D080                     ADD.L     D0,D0
0001590A 204A                     MOVE.L    A2,A0
0001590C 5370 0800            SUBQ.W    #$1,$08(A0,D0.W) *Subtracts 1 from the amount of time left on the present.
00015910 3030 0800            MOVE.W    $08(A0,D0.W),D0
00015914 0C40 012C            CMP.W     #$012C,D0      *Checks if less than 300 cycles are left on present.
00015918 6302                      BLS       $0001591C
0001591A 6030                     BRA       $0001594C
0001591C 3002                     MOVE.W    D2,D0
0001591E 48C0                     EXT.L     D0
00015920 D080                     ADD.L     D0,D0
00015922 0C72 0064 0800   CMP.W     #$0064,$08(A2,D0.W) *Checks if less than 100 cycles are left on present.
00015928 6314                      BLS       $0001593E
0001592A 3002                     MOVE.W    D2,D0
0001592C 48C0                     EXT.L     D0
0001592E D080                     ADD.L     D0,D0
00015930 7200                     MOVEQ     #$00,D1
00015932 3232 0800            MOVE.W    $08(A2,D0.W),D1
00015936 700C                     MOVEQ     #$0C,D0
00015938 C280                     AND.L     D0,D1
0001593A 6712                     BEQ       $0001594E
0001593C 600E                     BRA       $0001594C

And I've already got a fair bit worked out: where starting lives are set, where they are reduced at death, where health decreases, where starting presents are set, where inventory is cleared. I know this is a drop in the ocean of the whole ROM but I had to start some where and it made sense to work around the data I had already worked out.

So now on 1 player instead of starting out with 3 lives and 4 pairs of Bonus HiTops you start with 10 lives and 16 pairs of Rocket Skates; which isn't necessary as they each last infinitely long (which is highly impractical as it means you can't enter the lift so can't get past level 1 but fun none the less!).

So, it is now 3am and I have to go to sleep (my wife is going to kick my ass when I go upstairs) but I'm so psyched at what I've managed to do tonight I just want to keep going and get the whole game figured out!

I'm probably going to get my site updated soon with the routines I figured out so far and see how it looks using the memory map layout I've got planned, after that I might start up a thread in the Personal Projects page to keep updates in and try keep me jazzed about it. I don't have any real plans for the game other than get it disassembled but I like the idea of making my own presents, adjusting how some of them work (unlimited rocket skates that work like Super HiTops would be great) and having Toejam and Earl start on different levels are some of the things I'm playing with. Also interested to see how the random map generator works, think that will interest a few people.

Still looking for feedback on the memory map visualising if anyone has any. Cheers.  :woot!:  ;D  ;D

January 01, 2016, 01:56:36 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
That's the page just about completed in terms of revised layout. If you've got a minute spare can you check it over and make sure everything makes sense and that there isn't any confusion over what offsets I'm referring to:
http://www.mybrillgamesite.com/hacking/?id=86

I've still to make the RAM table and Additional Info table borders match the ROM table and I haven't decided if I'll add colour to the RAM table yet as I think it might make the page too noisy.

My initial idea of having every address and instruction listed (collapsed initially) won't work as it makes joomla crap out because there is so much code (1.5 million lines of HTML I ended up with!) so I'll stick to keeping the 'Unknown' stuff as unexpandable with just a little warning when you try.

For image data, and maybe for the unknown sections as well, what I plan to do is have a link that opens another little window showing the tilemap / spritemap and the raw data that creates it but no rush on getting that in.

I'm going to create a topic in Personal Projects soon for tracking the project as well so it will be available for discussion there (this topic is centered around how to visualise the memory map rather than the actual disassembly in general).
« Last Edit: January 01, 2016, 01:56:36 pm by RyanfaeScotland »

RyanfaeScotland

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
    • My Brill Game Site
Re: Memory Map Visualising
« Reply #3 on: January 27, 2016, 08:41:43 am »
I've completed an update to the memory map visualisation on the page which is more in keeping with what I had pictured in my head. Sharing here since I reckon it is rather good (if I do say so myself) and may be of interest to others in a similar boat.

The first visualisation (which is still present for now) is the visualisation with the actual code / disassembly explanation along side it. I'm not too keen on this one as I think it is a bit crowded, will become hard to navigate as it grows in size, is rather unwieldy code wise and a pain to update Also it will likely become obsolete once the completed assembly file is available. However it remains for now as it is a way to share information I discover as I progress but I will likely look for something better:



The one I've just recently added and am quite pleased with is accessed by clicking the little link on the 'percentage deciphered' information you can see in the screenshot above:



Here we get a proper indication of how the memory in the ROM breaks down, how much has been deciphered, where clusters of data are or large unexplored / unknown spots.

The code behind it is also a lot easier to update given it is a series of JavaScript loops spitting out styled divs (and js understands hex natively making the loops REALLY easy to deal with).

I'd be a little happier if 1 block = 1 byte but that would mean over 1million divs on the page for a 1MB rom and my browser keeps crapping out when trying to render that many. I have managed to render the RAM map at this scale though (65536 divs) and it works most of the time (interestingly it doesn't work if you click the link directly but does if you right-click and select 'Open in new tab'). I will probably look at moving away from divs to a canvas instead but the end result will look very much the same.

The thing I really like about this is you can zoom the bowser in and out to get a really clear view of the overall progress, especially if you have a massive monitor like the ones we use at work:



So as you can see at a glance, still a lot of work to be done on the disassembly! But I'll soon be moving to work exclusively on that once everything to facilitate doing so is in place.

Here are the links to the 3 maps if anyone wants to see them live, be aware though that the 2 bigger ones may crash your browser:

ROM Map with detailed explanations - http://www.mybrillgamesite.com/hacking/?id=86

ROM Map Summary Visualisation - http://www.mybrillgamesite.com/rawHTML/TE_ROM_MemoryMap.html

RAM Map Summary Visualisation - http://www.mybrillgamesite.com/rawHTML/TE_RAM_MemoryMap.html (Right click -> Open in new Tab if it crashes your browser)

EDIT - ROM Map Summary Visualisation Via HTML Canvas - http://www.mybrillgamesite.com/rawHTML/TE_ROM_MemoryMap%20-%20Canvas.html - This is the ROM Map this time rendered as an image via the HTML5 Canvas object. It renders a lot better as may be expected with it even being able to draw the ROM Map on a 1 block = 1 byte scale! That's 1,048,576 blocks! So far I've only drawn the blocks in red as a test but in time I'll switch the maps over to use this so I can get a finer grained scale without worrying about breaking browsers. (Although I think 1=1 is probably a bit excessive for the ROM.)
« Last Edit: January 27, 2016, 06:35:43 pm by RyanfaeScotland »