SNES VWF routine optimization

That routine looks good, though I certainly wouldn't consider myself a master of optimizing, so my opinion isn't any form of validation! This thread has been helpful. I've actually been in the process of developing a 'universal' VWF routine for Tenshi No Uta after I discovered there are at least 4 or more different text engines (cutscenes, main dialog, menu 12x12, menu tilemap 8x16, battle) used.

So, I'm trying to develop a routine that will work with all of them. This will of course include some different initializations and defined constants or offsets, but should for the most part work. I have picked up a few tricks from this code I have already put to good use.

Your routine is geared toward a RAM windows based engine. Clearly, the core can be used for VWF of any kind, but what surrounding logic or ideas might you have if the VWF were for a text system that was originally tile map based (item lists, menus etc), or the common 12x12 three tile DMA'd to VRAM engine for example?

Would you adapt your code? Would you instead adapt the text engine to fit your code, or something else? I'm interested in any ideas you may have for a flexible general purpose VWF core that could potentially serve the variety of types I've mentioned above.

You probably wouldn't be using $4308 for starters!

 

This thread has been helpful.

For me too, cheers ecst! :D
Yeah, thought it'd be fun to get some tech discussion going on just for the hell of it.

Your routine is geared toward a RAM windows based engine. Clearly, the core can be used for VWF of any kind, but what surrounding logic or ideas might you have if the VWF were for a text system that was originally tile map based (item lists, menus etc), or the common 12x12 three tile DMA'd to VRAM engine for example?

Would you adapt your code? Would you instead adapt the text engine to fit your code, or something else? I'm interested in any ideas you may have for a flexible general purpose VWF core that could potentially serve the variety of types I've mentioned above.

Yeah, I've faced this issue a lot. What you really want to do is break down the entire concept: what variables make up a VWF system?

First thing's first -- don't try merging the full string rendering loop into one big routine -- even if you can, don't. It will be too intimidating to maintain. JSRs aren't that expensive, break things down into very small blocks always. Make a unique render_string whenever you need it, have them all call out to render_character.

No matter what, you're writing your data direct to RAM, whether it's one tile, three tiles, or a whole line. So for that, you need a RAM address calculation step.

Next, you need to index into your font and pull out the width of the char. You'll need one for each font you have.

Next, there's both input and output font depth. Input can be 1, 2 or 4bpp. Output will really only be 2bpp or 4bpp.

Next, there's the layout of RAM you're writing to. This can vary greatly as well. Is it 8x16 native format? Is it split up into 8x8 chunks ala sprite font format? Something else?

Lastly, there's the bitpos update code. Very simple and universal, but your variable offsets may not be. And of course, you have to update the game variables that tell it where to blit tiledata later on and all that crap.

So put it all together:
- where are your work variables at in memory for this function?
- RAM base calculation
- font base calculation
- core loop {
--- input font depth (read portion)
--- output font depth / memory layout (write portion)
- }
- bitpos update
- game variable update (almost guaranteed to be unique per system)

Now, you need to look at all of your routines. What parts of the above do they share? What parts are similar enough that it makes sense to add conditionals to do different things for different routines? What parts are so different that it's a bad idea to combine them into the same routine?

In my personal experiences, especially with Daikaijuu, I've found that 100% of my 8x8 VWFs could share a single char rendering routine very easily. They always output to BG3 @ 2bpp, I always had enough RAM for the whole string in a single blit, I controlled the DMA transfer myself, etc. Very easy to share it all.

But 12x12s ... they tend to be so wildly different that it's often better to just make them unique systems.

The most pressing example I can remember was BL. All of the 12x12 systems were virtually the same, write the whole window into a gigantic RAM block. But the menu 12x12 was a living hell. 4bpp instead of 2bpp, it only blit one or two tiles at a time like Dual Orb 2, drawn using OAM data, so instead of linear 8x16 layout, it used a blocked 4x 8x8 layout. So after the first 8 lines were drawn, you had to adjust the address to render to the bottom 8 lines. And halfway across the screen horizontally, you had to go to the next tileset which was after the end of the first one entirely. Again, this is due to the way OAM sprites are laid out in memory. And the auto shadow I added to it -- man, that was tough. I should see if kammedo still has that code. It was so different that it'd be stupid to try and share any VWF code from that with the others.

And then the hardest VWF I've ever written -- for Der Langrisser. Ten tries over the course of two years to pull that one off. The horror ... that deserves an article of its own, though. But the best part? The entire game shared that same routine for all 12x12 text. I should say, that was both a blessing and a curse, but I digress ...

Now, if you have a match for the whole core loop part above in two or more VWFs, yeah, feel free to combine those. You can stick the RAM calculation into DBR:Y before calling a generic render_character in that case, and have the original-game-engine variables updated afterward in the unique sections of code.

I'd be happy to discuss methods for passing variables to functions in more depth if you like.
- jsr function; dw static_data; ...
- Memory variables for dynamic data
- Stack look-ahead reservation for dynamic data
- Using registers (including D and DB) to hold arguments
- Using the namespace keyword for safe, generic code that shares variable memory addresses
- Callee vs Caller stack-based function calls
- etc ...

Then again, if your core loop is only ~14 instructions anyway, and all the outside sections are different, is there even much advantage to them sharing the same routine in the first place? Might be interesting to see where we could go with a specialized macro system to blast the bits and pieces together with some glue code.

You probably wouldn't be using $4308 for starters!

I might :P
I'd modify my emu to print any accesses to $4308/9 to the console, and play through all of the text engine portions of the game. If the game never used it -- sure, why not?

Otherwise, I'd probably hook the game entry point where the stack gets set up, and reserve my own memory from $1fe0-$1fff, so I could do whatever I wanted with that anywhere else in the game :)
Also pretty dangerous though. You have to be really careful when screwing with the first 8k of memory.

Yeah, I've faced this issue a lot. What you really want to do is break down the entire concept: what variables make up a VWF system?

Yeah. I got tired of writing the same basic code over and over again to accomplish the same task slightly differently. My conclusion after I wrote it out on paper was similar to yours. A true universal VWF routine is not feasible, but if you can break it down enough and use some abstraction and high level language concepts, you can come up with a good chunk of reusable and/or modular code nonetheless.

It also helps to set some standards for yourself such such as assuming you will never use a larger font than 8x16 and it will in 90% of the cases be 2bpp. You can decrease complexity by setting some limitations up front. In the more rare case where 4bpp is needed for the final output or OAM or other special format, you can always tack on a conversion routine at the end assuming the extra overhead wouldn't be a performance bottleneck for your specific game.  Worst case you can take your VWF routine template and modify it for any rare specific cases. It'll probably be OK for most cases. 100% is never a realistic goal anyway, but majority is.

So put it all together:
- where are your work variables at in memory for this function?
- RAM base calculation
- font base calculation
- core loop {
--- input font depth (read portion)
--- output font depth / memory layout (write portion)
- }
- bitpos update
- game variable update (almost guaranteed to be unique per system)

Now, you need to look at all of your routines. What parts of the above do they share? What parts are similar enough that it makes sense to add conditionals to do different things for different routines? What parts are so different that it's a bad idea to combine them into the same routine?

Agreed. Breaking down into smaller, modular, routines should yield best results. The idea would be to plug and play the necessary modules from your module library for any given VWF. Though I forsee flexibility coming with an overhead price. To keep it really flexible, there may be a good deal of parameter passing going on. I've been looking at C function style stack usage to handle a lot of that. Or perhaps utilizing DB, and Direct Page. Anyway, I'd like to see some way for variable access other than $7fXXXX. You're limited with what opcodes you can use if you want to write out full addresses. It seems to lead to more bloated code than necessary and it becomes more difficult to maintain.

In my personal experiences, especially with Daikaijuu, I've found that 100% of my 8x8 VWFs could share a single char rendering routine very easily. They always output to BG3 @ 2bpp, I always had enough RAM for the whole string in a single blit, I controlled the DMA transfer myself, etc. Very easy to share it all.

Yes, sometimes there is a decision to be made of whether to code to compliment the game, or change the game code to compliment your code. That's a tough call sometimes. Sometimes one appears easier than the other and ends up being a tedious mess. Then you find out it would have been much easier to choose the latter method afterall.

But 12x12s ... they tend to be so wildly different that it's often better to just make them unique systems.

Indeed. They are the biggest trouble maker against this goal. It may be best to try and force the hand of the game in this case to conform to your set 'standards'. Something like force one character rendering at a time, take over the DMA to VRAM yourself etc..  Then they become much more likely candidates to just use your existing routines. Though there's always going to be those special cases where you rethink your plan and work with the game rather than against it or you're in for a world of hurt tearing it to pieces to accomplish relatively simple tasks.

What are we doing here? Hacking or rewriting the game engine? If I were going to write a game engine, it would be clean from scratch! I always have to step back and look at what I'm doing and why I'm doing it. Am I really just trying to hack the text system for a nicer font or expansion? Or have I really been trying to change fundamental engine constructs setting myself up for a long uphill battle?

Then again, if your core loop is only ~14 instructions anyway, and all the outside sections are different, is there even much advantage to them sharing the same routine in the first place? Might be interesting to see where we could go with a specialized macro system to blast the bits and pieces together with some glue code.

Right. Sometimes, it just isn't worth it.

A true universal VWF routine is not feasible, but if you can break it down enough and use some abstraction and high level language concepts, you can come up with a good chunk of reusable and/or modular code nonetheless.

Yes, now this I like. The main reason I omitted a macro parser in the new xkas, I wanted something much more powerful.

I want a high-level system that doesn't result in low-level bloat. I know, contradictory. But there has to be some sort of middle ground, eg:

Code: [Select]

render_character<depth = 4, layout = { 0, 2, 1, 3 }, outputbase = $7e8000, outputcalc = "xba; asl #2">
Inside the code, make it a complex system with C++-template-like code:

Code: [Select]

lda #12*depth //loop counter setup
.loop:
  <grab_font_data>
  sta $0020,y; xba; ora $0000,y; sta $0000,y
  if<depth == 4> {
    <grab_font_data>
    sta $0060,y; xba; ora $0040,y; sta $0040,y
  }
  ...
Once it gets into final code form, everything gets unrolled.
It'd be a bitch to create, but a true reusable module. Use the same code not just for one game, but for every game you work on. Again, except for the impossibly tricky and super timing sensitive stuff.

It also helps to set some standards for yourself such such as assuming you will never use a larger font than 8x16 and it will in 90% of the cases be 2bpp.

Best way to handle 16x16 is to detect a width > 8 and call render_char twice to fuse two parts together. An absolute waste to double the loop runtime just for 'W' and 'M' at 9 pixels wide each.

you can always tack on a conversion routine at the end assuming the extra overhead wouldn't be a performance bottleneck for your specific game

Yeah, I have some pretty shocking 2bpp->4bpp conversion routines. One of them, the background color at 2bpp is transparent, but the 4bpp background color is not, so I have to use multiple ORs+XOR to get the mask to change all color 0's (and only color 0's) to color 15's. It's ... pretty scary.

Another part I ran into once, I had exactly enough RAM to render the whole string at 2bpp, but the system needed 4bpp. So what I did was render it at 2bpp, convert the first half to 4bpp in-place, blit to VRAM, render again at 2bpp, and convert the second half. Was pretty tricky and I had to use different algorithms to avoid overwriting needed data while converting. Very complicated, but the end result was fantastic. So much better than being lazy and just rendering at 8x8.

Yes, sometimes there is a decision to be made of whether to code to compliment the game, or change the game code to compliment your code.

I went gung-ho with Daikai, tore the entire game apart. It just ended up causing all sorts of issues later on. I had to rewrite almost every little thing the game did for me before. Broke all the control codes, etc. It was empowering and fun, though.

You end up breaking subtle things. There was a part in Daikai where because I was manually waiting for vsync to blit each char, a translucent scrolling layer would freeze while text was writing. The stupid game wasn't using NMI to scroll the fade layer, and was expecting to get control back after every letter. I didn't notice when I wrote it because most of the game is bland and lacks effects like that.

The more I've been programming ... the more I've been as minimal with my hooks as possible. I try very hard not to use my own DMA routines, if I can massage the original ones into working. I'll go out of my way to avoid needing to hook two sections of code for a VWF routine. ~95% of them I pull off with only a single well-placed hook.

-----

EDIT: missed one.

Anyway, I'd like to see some way for variable access other than $7fXXXX. You're limited with what opcodes you can use if you want to write out full addresses.

Yeah, no inc/dec sucks.

You can always set DB to $7f for 16-bit opcodes (and give up quick $7e/etc access, sigh.)
Other than that, it's a tough problem when you go screwing with the game's RAM. Touch the wrong address and the game starts crashing randomly.

My idea was a custom debug function that counts how many times each memory address is written. Run the game to get a feel for really safe areas. Maybe even have beta testers run the game for a few hours to try and make sure its safe (Emerald Dragon would probably burn you here, heh.)

But to get a lot of bulk RAM for things like rendering an entire screen full of items ... my only real idea was to increase the save RAM size and use it as work RAM (its hardware safe, won't wear the chip down.) Would even work on most copiers.

Biggest issue with that is the hardware cart mod people would have a lot more work cut out for them. But an extra 8kbytes of 100% safe memory ... very tempting.

A cheap non-linear 8-bytes can be had via: [$00-3f:80-bf]:[$430b, $431b, ... $437b]. These were apparently planned for something ... but never actually used in the final hardware revision. No H/DMA process will ever touch them, and I've only seen one or two games ever write to them.

$43xc-e are open bus, and $43xf is a mirror of $43xb. Weird stuff. You could have fun screwing with people disassembling your code, though. Write to $43xf, read the results later back from $43xb ;)

Yes, now this I like. The main reason I omitted a macro parser in the new xkas, I wanted something much more powerful.

I want a high-level system that doesn't result in low-level bloat. I know, contradictory. But there has to be some sort of middle ground, eg:

Code: [Select]

render_character<depth = 4, layout = { 0, 2, 1, 3 }, outputbase = $7e8000, outputcalc = "xba; asl #2">
Inside the code, make it a complex system with C++-template-like code:

We're definitely on the same page here. That is exactly the kind of thing I had in mind as ideal solution. All you need to specify are the game specific parameters and code and you're good to go. The template idea there is a good conceptual (since the syntax and functionality doesn't exist) representation of the goal. I am working in that direction within the confines of the current capabilities. Define a few offsets or constants, alter the few custom routines like the output calc routine, and away you go based on that information. Realistically, I'll probably only develop this far enough to meet my immediate needs and refine it as I move to include other VWF routines.

I think refinement process is a realistic approach. Get a base system down. Then, every time you include a new VWF system in the mix, you refine the routines so they can handle it and then of course back test and ensure your previous ones still work. The further you go and the more systems you make it work with, the better and more flexible it should become. You need to be careful of bloating your code however and moving toward less manageable code. Going out of your way for flexibility can create an overcomplicated mess.

I think we've definitely nailed it here on a conceptual level. I'll see how much of that I end up putting to good use in real application. There's always trade offs for time and motivation. Start out with a sound plan to move a mountain, yet end up just walking through the pass or around it instead!

Best way to handle 16x16 is to detect a width > 8 and call render_char twice to fuse two parts together. An absolute waste to double the loop runtime just for 'W' and 'M' at 9 pixels wide each.

I'd just as soon switch to a thinner font. It's not worth the cycles, not worth the extra code, and not worth the time. Besides, isn't the point of a VWF hack to better facilitate English text? If you end up still not neing able to fit the text you need, because your font is too damn wide, what good is it? I tried a thick font in Emerald Dragon for awhile because it looked nice, but then as I went along, text fit poorly. Have more than two large words on a line, sorry, won't fit. Oh, you want to use that font on the menu too? Yeah right, you only have 24 pixels of screen space for that word. I quickly abandoned that idea for a thinner font, and even the font I went with in the end is much thicker than the ones you typically use. Thinner fonts tend to be more useful in the end.

I went gung-ho with Daikai, tore the entire game apart. It just ended up causing all sorts of issues later on. I had to rewrite almost every little thing the game did for me before. Broke all the control codes, etc. It was empowering and fun, though.

You end up breaking subtle things. There was a part in Daikai where because I was manually waiting for vsync to blit each char, a translucent scrolling layer would freeze while text was writing. The stupid game wasn't using NMI to scroll the fade layer, and was expecting to get control back after every letter. I didn't notice when I wrote it because most of the game is bland and lacks effects like that.

The more I've been programming ... the more I've been as minimal with my hooks as possible. I try very hard not to use my own DMA routines, if I can massage the original ones into working. I'll go out of my way to avoid needing to hook two sections of code for a VWF routine. ~95% of them I pull off with only a single well-placed hook.

Our experiences, though different, have taught us many of the same lessons! My more recent hacking work is much more consolidated than my work from years back. I have some work where  freely hooked in here and there wherever I wished. However, years of debugging experience has taught me that's going to give you headaches later.Your Dai Kai example is a good one. You break a lot of small things you don't even notice at first. Minor graphical glitches due to timing changes or DMA modification suck. You mess with one wrong RAM location you thought was free, and then you see a random crash much later and have one hell of a debugging trip trying to figure out what happened. I've also placed hooks before that I thought were hit in all applicable cases, but then found out later certain branches of code went around it. Frustration to the max.

Yeah, no inc/dec sucks.

You can always set DB to $7f for 16-bit opcodes (and give up quick $7e/etc access, sigh.)
Other than that, it's a tough problem when you go screwing with the game's RAM. Touch the wrong address and the game starts crashing randomly.

My idea was a custom debug function that counts how many times each memory address is written. Run the game to get a feel for really safe areas. Maybe even have beta testers run the game for a few hours to try and make sure its safe (Emerald Dragon would probably burn you here, heh.)

But to get a lot of bulk RAM for things like rendering an entire screen full of items ... my only real idea was to increase the save RAM size and use it as work RAM (its hardware safe, won't wear the chip down.) Would even work on most copiers.

Biggest issue with that is the hardware cart mod people would have a lot more work cut out for them. But an extra 8kbytes of 100% safe memory ... very tempting.

A cheap non-linear 8-bytes can be had via: [$00-3f:80-bf]:[$430b, $431b, ... $437b]. These were apparently planned for something ... but never actually used in the final hardware revision. No H/DMA process will ever touch them, and I've only seen one or two games ever write to them.

$43xc-e are open bus, and $43xf is a mirror of $43xb. Weird stuff. You could have fun screwing with people disassembling your code, though. Write to $43xf, read the results later back from $43xb

Yeah. Choosing RAM locations can be tricky business. Memory usage monitoring is also sometimes limited in it's effectiveness depending on the game engine.

Yeah, Emerald Dragon is my worst SNES debugging nightmare. It's miserable. Most everything is dynamic in a rolling RAM command queue buffer. Even printing characters to the screen is dynamic. It renders the character in RAM (where it renders is somewhat dynamic in a smaller rolling window), then adds a command to a dynamic command queue with instruction to DMA it to VRAM later. This dynamic rolling command queue includes nearly all possible game commands and register writes. So, it goes through executing commands and along the way writes characters to VRAM.

So if you want to debug a line of text, you don't have any concrete information other than the ROM locations. You're kind of forced to work forward instead of reverse engineer a problem. The RAM locations involved will be different every time. It's a real problem for me and my short free time blocks because I often can't spend that much time in a single debugging session. Then when I go back to it later, hardly any information I had is useful anymore. Start over. It makes finding problems a real pain. Couple that with it's insanely stupid menu structure which I still don't fully understand under some conditions and you can imagine why I've lost some hair on my head!  I don't think the game uses 16-bit mode ever except in a rare case where it was necessary to make something work on the SNES. Emerald Dragon reeks of a game written for an 8-bit computer ported to a console. It also reeks of poor programming quality. The original game shows 11680exp left when you gain a level instead of 0. I corrected that in the English version. What caused that? Well, they simply subtracted the exp and let the result roll into negative and used it for printing. It's reset later, but not for the printing. It is always specifically 11680 due to the algorithm used to generate the number. It was never made for negatives. I forget the details now, but it's always 11680. Who releases a game like that? Testing anyone? Ok, I need to go cool down now.

Good info about $43XX. I may use it sometime if I remember.

We're definitely on the same page here. That is exactly the kind of thing I had in mind as ideal solution.

Lordtech tried with his C compiler, but that just isn't going to work: at 2.58MHz, and with only one math-capable register, you literally cannot afford the speed penalty for compiled code. The language and your programming has to understand that you only have three registers.

We need a good place to discuss a higher level construct for the SNES. I always have ideas, but they never really go anywhere alone. If we get it all together and solidified, I'll write the compiler.

I think refinement process is a realistic approach.

Definitely. That's how I started with my template library. I got sick of C++ STL and Boost getting in my way, so I started making my own stuff. I'm up to ~95kb for that thing now, and it saves me so much time.

And even with the refinement, know when not to bother: that 12x12 OAM text system, for instance.

Start out with a sound plan to move a mountain, yet end up just walking through the pass or around it instead!

The same can be said in reverse: I had to abandon bsnes at v0.005 and start from scratch because I didn't plan enough. My current core suffers a bit from too much abstraction -- I've been scaling it back a bit lately. You'll never know how much framework you need until you do something. But experience means getting as close as possible on the first try, so you never have to throw all your old work out.

Thinner fonts tend to be more useful in the end.

I was guilty of the Chicago craze back in 2000, too. I look back in awe. Bold fonts are so inappropriate, especially for dialog in role playing games you'll be reading for dozens of hours.

I mean think about it. Where else do you see large blocks of dialog written solely in bold fonts? It's actually annoying to read text like that everywhere else. Why are video games the exception?

Our experiences, though different, have taught us many of the same lessons!

Yeah, unfortunate as it may be -- we have to learn the lessons on our own. Not something easy to teach to new people. Especially when two out of three people you spend three years teaching disappear. Or die, sigh.

So ... have you started breaking down routines, too?
I used to spend hours staring at long, 30mb trace logs, trying to find magic constants. Total waste of time. Start creating an actual disassembly. Reduce the loops, label each function and variable, etc. Hit every section of code, even if you have to reverse / disable conditions. Sometimes, you'll even hit an in-game debugging system that way.

adds a command to a dynamic command queue with instruction to DMA it to VRAM later

Very common. Every game I've seen has one of these. They're pretty neat. I once tried using it myself by modifying two 16x8 transfers to four 8x8 transfers and ended up overflowing the buffer, heheh.

Emerald Dragon reeks of a game written for an 8-bit computer ported to a console.

Try Tekkaman Blade. In one of my hooks, I swapped to 16-bit accumulator for something, and the game started crashing on occasion. Turned out if the NMI triggered in that section, it never bothered to set P.M/X, and so the next LDA would crash the game. Could've forced its hand, but in the end I just wrote 100% of the code in 8-bit mode, like the game itself. That was fun.

And I did all that in exchange for ten icons for Daikaijuu that I never ended up finishing. What a terrible trade in retrospect.

You could make a VWF routine into a macro where almost any part could be changed. That would help with it being versatile and used on different games. I did this with DQ3 for the menu system and I created a way of easily declaring menus and pointing to the right location. Macros are great!

Lordtech tried with his C compiler, but that just isn't going to work: at 2.58MHz, and with only one math-capable register, you literally cannot afford the speed penalty for compiled code. The language and your programming has to understand that you only have three registers.

We need a good place to discuss a higher level construct for the SNES. I always have ideas, but they never really go anywhere alone. If we get it all together and solidified, I'll write the compiler.

While I agree with you, I think the C compiler's failed to catch on primarily due to lack of libraries. What good is using a high level language if you still need to do everything low level? You're still going to be poking all the registers and micromanaging with his C compiler. More users would jump on board if you could really take advantage of that high level functionality.  But yes, the system isn't really powerful enough performance wise to do a full featured game in C. I also can't see straight C ever being very useful for ROM hacking. When it comes to ROM hacking, you're ball and chained to some degree with low level.

I do think the intermediary type template language we've discussed could be a nice balance between the two. You can always create a new topic here on the subject and see if anyone is interested or has ideas. Unfortunately, I don't forsee many being that interested. It would only be useful to those of who already know SNES programming and are likely to do multiple projects or projects that would benefit from these templates. That probably only leaves a few of us.

I was guilty of the Chicago craze back in 2000, too. I look back in awe. Bold fonts are so inappropriate, especially for dialog in role playing games you'll be reading for dozens of hours.

I mean think about it. Where else do you see large blocks of dialog written solely in bold fonts? It's actually annoying to read text like that everywhere else. Why are video games the exception?

The best games of that time period used such thick fonts such as Final Fantasy VI and Chrono Trigger for notable examples. In addition, our most popular and historic fan translations do so as well such as Final Fantasy V, Tales of Phantasia, and Star Ocean. I have to confess, I think it looks nice and is in fact easily readable. I actually prefer those thicker fonts on my TV and through filters on emulators.  It seems to fit games visually too. I can't imagine a thinner font in Wozz. I'm not much of a font guy though. In most cases, a font is a font to me. I only care about it's readability for the most part.

Yeah, unfortunate as it may be -- we have to learn the lessons on our own. Not something easy to teach to new people. Especially when two out of three people you spend three years teaching disappear. Or die, sigh.

Amen brother! And people wonder why I'm so selective on who I give my one on one time teaching time too. 

So ... have you started breaking down routines, too?
I used to spend hours staring at long, 30mb trace logs, trying to find magic constants. Total waste of time. Start creating an actual disassembly. Reduce the loops, label each function and variable, etc. Hit every section of code, even if you have to reverse / disable conditions. Sometimes, you'll even hit an in-game debugging system that way.

I usually don't go too crazy there. I make notations on variables and routines and figure out what needs to be figured out. This is the most time consuming part of the project. No luxury time there to waste if I actually want to complete a project before I'm too old. Also, maybe 80% of the time, I reverse engineer something, write some code and I'll never touch that part again. Sometimes I don't even bother to note it. I'll do it all mentally during a session, finish the hack I was working on, and move on. I've never been one of those people to take my trace file and break it all up, comment every line.  I just note the things I need to note in my note file and look at the routines and code of interest to what I'm doing. On occasion it may come back to bite me, but most of the time it doesn't. On the occasion it does, it can be beneficial to look at it fresh again. 

Very common. Every game I've seen has one of these. They're pretty neat. I once tried using it myself by modifying two 16x8 transfers to four 8x8 transfers and ended up overflowing the buffer, heheh.

Funny, I don't think any other games I have looked at do. They all have some DMA code specific to the text to handle the transfer. Hell, I've seen games manually transfer each character to VRAM via $2118!! Many of them have commands or controls for the game engine, but typically, the text has it's own transfer code in most of the games I've looked at.

Our contrasting statements solidify a cardinal rule of mine to never assume anything in ROM hacking! Each game you look at has potential to do something you don't expect! It's easy to become narrow minded when you expect the game to do something too much.

I mean think about it. Where else do you see large blocks of dialog written solely in bold fonts? It's actually annoying to read text like that everywhere else. Why are video games the exception?

It used to be the norm in some places....

I also can't see straight C ever being very useful for ROM hacking. When it comes to ROM hacking, you're ball and chained to some degree with low level.

Very true. I just hate the fact that no matter how much I document, the code always seems hard to go back and understand later on. Compared to C, where it's very easy to understand again. There has to be some middle ground. Get readability and absolute control at the same time. Maybe something as simple as a 65c816 version of Terse?

You can always create a new topic here on the subject and see if anyone is interested or has ideas.

No offense, really; but not after that patching format discussion. The best case scenario I see for that is we only end up with four new intermediate languages, and lots of intense debate over whether to use "<>" or "{}" for brackets (with a hundred obligatory analogies to other languages.)

I actually prefer those thicker fonts on my TV and through filters on emulators.  It seems to fit games visually too. I can't imagine a thinner font in Wozz. I'm not much of a font guy though. In most cases, a font is a font to me. I only care about it's readability for the most part.

Hmm ... interesting. My roommate with bad eyes had no problems with my ultra-thin 8x8 font from Daikai on a 20" TV, but you may have a point there. Especially in regards to long-term readability.

The most important thing is consistency. I never liked the way the FF6 patch had a thin dialogue font, and a thick item font. Same in reverse for BL.

On occasion it may come back to bite me, but most of the time it doesn't. On the occasion it does, it can be beneficial to look at it fresh again.

It doesn't take me much time to tear the routines apart anymore. I keep organized folders full of notes for each routine I disassemble these days. They prove helpful quite often for me at least. But, so long as you have no trouble understanding your old code when you need to edit it ... probably not much need to do the same.

Eh, maybe it's just experience. But I feel like it's a lot easier lately to understand the whole game system this way. Stuff that used to make no sense in trace logs suddenly comes together.

It used to be the norm in some places....

And you see what that got us :P

I can't imagine a thinner font in Wozz.

That's three translations with the same font. I'm calling it Transcorp Font!

Yeah, I suppose they put everything in bold font for readability.

No offense, really; but not after that patching format discussion. The best case scenario I see for that is we only end up with four new intermediate languages, and lots of intense debate over whether to use "<>" or "{}" for brackets (with a hundred obligatory analogies to other languages.)

Yes, clearly you can tell what's important to people around here during discussions. Unfortunately, too many discussions go in that direction around here anymore. I can certainly understand your discouragement. 

Hmm ... interesting. My roommate with bad eyes had no problems with my ultra-thin 8x8 font from Daikai on a 20" TV, but you may have a point there. Especially in regards to long-term readability.

The most important thing is consistency. I never liked the way the FF6 patch had a thin dialogue font, and a thick item font. Same in reverse for BL.

I have used a 27", so maybe that and the distance seated has something to do with it. That's just my own personal preference is all anyway. You probably can't say definitive which is better for anyone other than yourself. Everybody's eyes and viewing circumstances are different.

Eh, maybe it's just experience. But I feel like it's a lot easier lately to understand the whole game system this way. Stuff that used to make no sense in trace logs suddenly comes together.

Yes, the more experience you have, the more quickly and easily things seem to make sense. I am probably 10x faster at figuring out things than I used to be. That's one reason I'm encouraged to continue with my my platform of familiarity. Things go much more efficiently that way which is important with time at a premium.


KingMike:

I don't even remember where I got that font anymore. I may have half created it myself based on something else. I cannot take full credit though as I'm sure I didn't make it from scratch. I'm too art retarded for that. Many some smaller Chicago variant or something? I aimed for simplicity. The font ended up being pretty bland, but I think that makes for good easy readability. I'm one of those guys that doesn't mind everything in Times New Roman. Simple, somewhat bland, but gets the job done.

I personally enjoy thick fonts, although it is less about thickness then readability, and I just see, and recognize thick fonts better.

Alas I have nothing more to add, fonts are like an intense puzzle from hell for me, as my art ability is barely on par with that of a 1st grade fingerpainting class.

It is not a myth either, I knew you guys did one on one sessions, zen masters 

I've been working on this on and off. I wrote a new VWF core for the RAM window engine and am now trying to get two other 12x12 routines in the game to also work with it. It's coming along OK. I still have to figure out a way to handle the different text locations from routine to routine that I'm happy with. One uses a 16-bit RAM offset to decompressed text in RAM, while another is  a 24-bit pointer index to ROM,  while the third is 16-bit to ROM. They are all handled slightly differently with where the address is stored and how the final offset is indexed.

I was thinking about having a 'standard' that the core routine would follow and each VWF would need a few lines of initialization to take the offsets and store them in the standard format. By standard format, I will probably use a 16-bit x offset and use the DB register for bank. Something like: lda $000,x where y would be the 16-bit index and DB would contain the bank whether that be anywhere in RAM or ROM. The negative to doing this will be some added complexity on updated the actual game variables at the end, and some extra overhead up front. This standard form could also be utilized later though to look at the next character and handle control codes. Speaking of which...

The second obstacle is control codes. Control codes dictate when your bit position, RAM locations etc. should be reset. And in some cases such as tile mapped based engines, control codes could do some other funky things that are problematic to handle with the VWF such as a blinking cursor tile or other such things requiring tile boundaries. I'm not sure yet of the best way to handle this. I thought of looking at the next character after the current character is rendered and determine what to do if it's a control code, the determine whether I need to reset the bit pos, increment up a tile etc. This will be specific code to each routine since control codes vary wildly from engine to engine, but having a standard way to handle it will ensure best fit with all the reusable portions and pretty much act as a template easily adaptable to each individual VWF.

Any further discussion on handling any of the above mentioned items?



EDIT1: Even though no one is interested, I still am and this has been coming along great. I've got an update on TransCorp with some more details. 3 supported systems and continued work for 2 more. I will post some sample code here of how the system generally works later when it's further developed. But basically it's like:

1. Initialize system variables with information from the game

2. Run the system:
jsr GetChar      ;gets text char
jsr GetCharInfo   ;calc font offset and vwf length
jsr OutputRAMCalc   ;calc VWF output RAM location   
jsr RenderChar   ;Render Char
jsr jsr CalcCurPos   ;calculate bit/tile position for next character

3. Control Code handling, update game variables with resulting VWF system variables.


That's the basic concept I'm using. So, for each VWF, you simply initialize the 'standard' system variables with the proper information which is specific to each system. Run the system, then write a bit more code at the end for any special control handling and pull the 'standard' system variable information and put it back into the game variables.


Much of it works in a similar fashion to macros and templates. Except this is a single set of routines and the code truly is shared. Unlike a macro which just copies and pastes the same code. So, there's some overhead to the system, but it should be plenty fast enough for most cases. If you hit something where speed is truly an issue, you can either use the system as a template and streamline it specific to that VWF, or just code a new one for those rare cases. My goal was a general purpose flexible VWF system that should cover most cases.

So why would you code the control codes in? I mean, they originally had code for themselves, and if they use the text routine for whatever purpose (I've never actually seen that...) other than writing strings into the current text, i.e. blinking cursors, then they should have a call for that to your text engine, right? I usually went with the standard system for control codes and just manipulated the calls to the text routine to be like

jsr ResetBits
jmp TextRoutine

in case it needed to be reset etc. I think even if you find a general system it'd still require a datatable etc and some code actually checking all that. And this might not work in cases where you have to continue after a control code in the actual text without using a new tile (think blinking cursos mid-sentence)!
I'd think that this needs to be game-specific and has a great performance and effectiveness (as in covering most cases) penalty.

cYa,

Tauwasser