DTE - Getting Started?

[justin3009]

This is going to be for Mega Man X3 an SNES rom.  (I wasn't sure if this was supposed to go in the newbie board or not considering I'm wanting to get started on the basics first and then code)

What I'm aiming at right now is getting started with a DTE.  Even though there's a few bugs in the game that still have to be kinked out (They're graphical though..not related to this type of coding), I want to implement a DTE for a much better text storing system and really compress the amount of space used for text.  Though, the text itself can be written out straight in a hex editor.  Ex: 41 = A, 42 = B, 43 = C and what not so I'm not sure if that'll change the playing field a bit more.

I guess for questions:

1. What exactly is a DTE? - I know it's a type of dictionary compression system..I think.  I've seen it in Chrono Trigger where it loads a specific chunk of text and compresses it down to a single byte (or something of that fashion).
2. What is needed for a DTE?  A chunk of space for pointers and the "compressed" text, I'm pretty sure of that.  And I think a spot in RAM to check certain values?
3. How does a DTE work?
4. Are there any specific examples that can be looked at through code/game to get a better idea? - AKA Pseudocode or an actual written routine for an SNES game that's commented and talked through like the VWF one's are.

Sorry if this is overbearing or extremely newbish (Thus why I was thinking the Newbie Board), but I'm wanting to get a start on this and learn more on other areas of ASM so I can truly modify my current project into something that can be more flexible in space and modding.

[Gideon Zhi]

At its very basic, DTE is a dictionary compression code which has a single key byte (most usefully, a range of key bytes) output two characters. You typically don't need any pointers for this, just a table of value-pairs which you can index based on your key byte. So you have, like, 80=th, 81=e , 82=is, etc. The way I usually program it is to store a flag in the nether regions of memory. My code initializes the flag when the string starts loading, and when it finds a value in my DTE range it flips the flag, prints out the first value, decrements the read stream so it loads the same byte again, flips the flag back, then prints out the second value and continues on.

You'll want a util to optimize your DTE table. There should be a few on the site.

There are a couple of docs on the site with DTE code in them. Here's one I wrote for Digital Devil Monogatari: Megami Tensei several years ago.
http://www.romhacking.net/docs/mt1dte.txt

[RedComet]

Might give this a look too: http://www.romhacking.net/docs/142/

[justin3009]

This doesn't seem to difficult for the most part.  It's a little weird to me still but it really doesn't seem too complicated.  I have the relatively important text code documented and labeled for MMX3 so it shouldn't be too hard to pick out where I put this stuff.

So for a basic start up, would it be like this?

1. Find the pointer data in which text loads from (In Mega Man's case it's a single byte that loads the text from a pointer)
2. Loads a specific table of values?  (Like 90 to B0) to check if it's DTE or not.  If so, go to new code, if not just go back to the original.
3. Test the value then...store to X?

I'm already lost D:  It really doesn't sound complicated but then looking at the code and it severely throws me off.

[Gideon Zhi]

Well, most games read their text byte-by-byte when displaying it to the screen. So you'll have a bit of code that reads a byte from ROM, then does something with it (loads some font tile or other, updates a tilemap somewhere, etc.) You want to intercept the game as it streams through its text, and test each byte of text as it gets read to see whether it falls within your DTE range. If it DOES, that's when you start flag-flipping and feeding other values not in your DTE range (iow, regular text bytes) back to the game.

So... sort of like this:
1) Game loads a byte in its text stream.
2) Check to see if the byte is within your DTE range (90-B0, using your example.) If it's not, return to the routine and do nothing.
3) If it IS within your range, flip a flag to tell the game you're in DTE Mode, load the first value in the corresponding pair, decrement the pointer to the current spot in the text stream (frequently x, y, or some register being accessed indirectly) then return to the routine with your new value, the first in the pair, and let it do whatever it was going to do with the value in the first place.
4) Next time the routine comes around and the game loads a byte, you'll still be loading the same byte, so it will be in your DTE range. Your code should notice that you've toggled your flag, and flip it back. Then you just load the second value in the pair and continue as normal.

[justin3009]

Alrighty, I think I have the idea now on this.

Though I've gotten most of the other stuff done, I'm confused on this part:

Code Select Expand

  LDY $0658        ;This was the first command I overwrote with the jump to the routine.
  LDA ($BE),Y      ;This is the second.


There wasn't any sort of code like that in the text code that I've gotten traced and written down.  I had it reading the letter value from 7E0604 which is where each letter gets written and stored to 1 at a time.  Then it'd check if that was a value of 17 to 1A (For testing purposes) then run through the code.

Edit: I saw this area but..it doesn't seem right.

Code Select Expand

$00/E62E A0 0E       LDY #$0E                A:013E X:0030 Y:0000 P:envMXdIzc
$00/E630 20 3B 87    JSR $873B  [$00:873B]   A:013E X:0030 Y:000E P:envMXdIzc
$00/873B C2 20       REP #$20                A:013E X:0030 Y:000E P:envMXdIzc
$00/873D B9 AD 97    LDA $97AD,y[$06:97BB]   A:013E X:0030 Y:000E P:envmXdIzc
$00/8740 85 10       STA $10    [$00:0010]   A:9881 X:0030 Y:000E P:eNvmXdIzc
$00/8742 A0 00       LDY #$00                A:9881 X:0030 Y:000E P:eNvmXdIzc
$00/8744 A6 A4       LDX $A4    [$00:00A4]   A:9881 X:0030 Y:0000 P:envmXdIZc
$00/8746 C2 21       REP #$21                A:9881 X:0000 Y:0000 P:envmXdIZc
$00/8748 B1 10       LDA ($10),y[$06:9881]   A:9881 X:0000 Y:0000 P:envmXdIZc - I'm really confused on what I should be doing.

[hanhnn]

There are a couple of docs on the site with DTE code in them. Here's one I wrote for Digital Devil Monogatari: Megami Tensei several years ago.
http://www.romhacking.net/docs/mt1dte.txt

I clicked the link and got this

Forbidden

You don't have permission to access /docs/mt1dte.txt on this server.

[BRPXQZME]

Do you use Internet security software or a proxy? If you use anything that strips the HTTP referrer, you will want to turn that off.

[Gideon Zhi]

Alrighty, I think I have the idea now on this.

Though I've gotten most of the other stuff done, I'm confused on this part:

Code Select Expand

  LDY $0658        ;This was the first command I overwrote with the jump to the routine.
  LDA ($BE),Y      ;This is the second.


There wasn't any sort of code like that in the text code that I've gotten traced and written down.  I had it reading the letter value from 7E0604 which is where each letter gets written and stored to 1 at a time.  Then it'd check if that was a value of 17 to 1A (For testing purposes) then run through the code.

Edit: I saw this area but..it doesn't seem right.

Code Select Expand

$00/E62E A0 0E       LDY #$0E                A:013E X:0030 Y:0000 P:envMXdIzc
$00/E630 20 3B 87    JSR $873B  [$00:873B]   A:013E X:0030 Y:000E P:envMXdIzc
$00/873B C2 20       REP #$20                A:013E X:0030 Y:000E P:envMXdIzc
$00/873D B9 AD 97    LDA $97AD,y[$06:97BB]   A:013E X:0030 Y:000E P:envmXdIzc
$00/8740 85 10       STA $10    [$00:0010]   A:9881 X:0030 Y:000E P:eNvmXdIzc
$00/8742 A0 00       LDY #$00                A:9881 X:0030 Y:000E P:eNvmXdIzc
$00/8744 A6 A4       LDX $A4    [$00:00A4]   A:9881 X:0030 Y:0000 P:envmXdIZc
$00/8746 C2 21       REP #$21                A:9881 X:0000 Y:0000 P:envmXdIZc
$00/8748 B1 10       LDA ($10),y[$06:9881]   A:9881 X:0000 Y:0000 P:envmXdIZc - I'm really confused on what I should be doing.

I'll take a look later today. Can you give me a few offsets for where text appears in the game? Like the intro after X and Zero blast the bee robot, for example. An address for the font or a table file would help too.

[justin3009]

PM Sent, hopefully that's enough information.  I may have to actually send you the rom though since the table was added manually awhile back.