News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Griever

Pages: [1]
1
ROM Hacking Discussion / Re: Golden Sun text compression
« on: January 24, 2015, 01:39:31 am »
Sorry for necroposting, but I've figured all out once I've constructed tree and tried to crawl in it by hands.
In general, iteration bitstream is a binary tree, serialized in pre-order traversal. Source stream is a plain path in that tree. Whole compression scheme is a Huffman with individual tree for each char.

2
ROM Hacking Discussion / Re: Golden Sun text compression
« on: October 29, 2014, 01:02:29 am »
Last time I checked Golden Sun used an LZSS derivative, while the second game used Huffman coding.
Yes, LZSS is used by Golden Sun for tileset compression. The algorithm was straight, so I had no problem with it.
This code is surely not a straightforward Huffman. Can anyone, please, take a look at graphs, I've pointed in top message and try to identify compression scheme?

3
ROM Hacking Discussion / Golden Sun text compression
« on: October 28, 2014, 02:39:15 pm »
Hello, I'm digging into different text compressions and found one in Golden Sun for GBA is quite interesting.
Sure, I know about Labmaster's doc and tool, but unpacking routine was just rewrited atop on asm code, and I'm trying to get the idea of compression.
Here's graph of 1 character decode. That's all pretty complicated but the part I don't understand is this one.
Hard to explain, but I'll try.
The function receives previously decoded char code. Each of this chars has some kind of path in recursive tree (which I wanted to discuss here), so actual source text bitstream just decides either to parse tree one round or emit char.
Before each tree there's codes of characters, which can be put after this previously decoded char. So one tree parse returns offset-back to appropriate character. Source text bitstream just decides if you need to parse one more time and increase this offset-back.
Now for path in tree: if you go right (1), you get out of parsing, if you get left(0), you go into recursive tree, so you'll have to meet more 1s to get out of parsing and so on. At the end of parsing you have to count how much 1's did you get  and that will be offset-back.
Source bitstream can also skip some bits from the beginning of this tree path, so you can set a root in any place of this path, thus changing result offset-back.

For me all this path in recursive tree storage is overcomplicated and unoptimal (not saying, I don't understand it at all). But I'm sure there should be some simple explanation how this scheme works.
Maybe it's modification of well known algorithms or someone has seen this kind of tree in previous?

4
Personal Projects / Re: VBA-SDL-H bugfix
« on: September 26, 2014, 06:52:14 am »
Quote
no$gba debugger is now free
That was breaking news for me, thanks.  Guess, nobody will need console tool like VBA-SDL-H, while gui emu is available.
Speaking of which on No$GBA.  Were you actually able to get it to work?  It just instantly crashes for me every single time no matter which version I use.
It worked OK on Golden Sun. Where exactly does crash happens?

5
Personal Projects / VBA-SDL-H bugfix
« on: September 25, 2014, 07:05:56 am »
So I was summing up information for GBA debugging emulators and found, that VBA-SDL-H is the only opensource and free option for honest romhackers. Please correct me if I'm wrong.
Looks like Labmaster doesn't care about it anymore, in fact sources are hard to find nowadays. There's also a VBA-SDL-H2 available at poke community, which has some additional functions.
Anyhow, there's a bug in 'dsave' command: both address and size are read from the same command argument.
Spoiler:
Code: [Select]
void debuggerDumpSave(int n, char** args)
{
  u32 address;
  u32 size;
  char *file;
  FILE *f;

  if (n==4){
    file=args[1];
    if (!dexp_eval(args[2], &address)){
      printf("Invalid expression in address.\n");
      return;
    }
    if (!dexp_eval(args[2], &size)){//should be args[3]
      printf("Invalid expression in size");
      return;
    }

    f=fopen(file,"wb");
    if(f==NULL){
      printf("Error opening file.\n");
      return;
    }

    for(u32 i = 0; i < size; i++) {
      fputc(debuggerReadByte(address), f);
      address++;
    }

    fclose(f);
  } else
    debuggerUsage("dsave");
}
The same bug passed to VBA-SDL-H2.
I wanted to update VBA-SDL-H2 code, compile emulator and release it at github and here, but I'm quite new at community policies and regulations. Is this appropriate? Won't that offense rights of Labmaster, poke-community or VBA programmers?

6
Kirby's Adventure - LZ/RLE combined scheme with a few variations algo  ;)

7
How did you managed to extract a scipt with unused bytes between the pointers with irregular rate? Have you added each pointer manually?

8
If only Kruptar was open source
Ah, yes, Djinn does not want that for some reason :)
Quote
Since you mention Cartographer and Atlas by name, you might want to mention the differences and things Kruptar does not do that they do such as the linked entries and what not.
Done.
Quote
Since there is no string terminator, how can Kruptar know if is a pascal style string or terminated string?
By plugin. At simple string with byte terminator case, you select standard plugin, which searches for the terminator and returns appropritate length string. If you select ShortString.kpl, plugin will return string, based on first byte, which is string length. Kruptar also has a few modifications plugins for various schemes. You can check them all in Help -> Plugins... And, of course you can code yours plugin in Delphi or in C, if first byte is stringLength-1 or any scheme you will face.
Quote
Do you need plugins to get the script extraction/insertion working or can it be used for harder encrypted coded scripts as some hackers were complaining about sometimes?
Plugin is used for any extraction/insertion. Even in null-terminated cases. If you need to modify/compress string you can code your own plugin which does this.
Quote
Also do you happen to know what's the lastest version of this app?
7.1.1.17 - 22.08.2011
Latest version is always here
Yup, unfortunately lately Djinn didn't modify Kruptar at all. Guess, the tool works OK for her current projects. But there were some plans for writing Kruptar 8 which were not realized yet.
Quote
I think it would be a good idea for you to go through and expand upon the functionality of rest of the options in the same manner.
Sure, will do.
If anybody will have additional notes on lack of explanation in my document, feel free to point them out here.

9
Thank you, dear Nightcrawler for a lively and comprehensive response, I'll try to fix all remarks.
What if the pointer the pointer points to is a different format?
Ok, emphasized that in document. Both pointers should have the same format.
Quote
I can't open that project, too many errors about missing ROMs and tables.
But of course you can't - I cannot distribute Beetlejuice(U).nes ROMs. Just choose your ROM right after error happens and everything will work just fine.
Quote
Where is the archive of your English tutorial?
Here. You will also need ROM to open Final Fantasy project.
Quote
I don't know how DTE or Dictionary is handled in Kruptar. Why isn't it just in the basic table?
That is. Check the project.
Quote
You can't have a multi-character entry (ex. AB=abc) in your table?
Fixed this in document. Supports multi byte and multi character entries, \n sequence, no linked entries and control codes.
Quote
What about dictionaries with and without (fixed width) terminators?
As Kruptar will use dictionary group for that - you will specify string length for that group - no need to double clarify that in turorial.
Quote
Are you telling me I can input a UTF-8, UTF-16, S-JIS, EUC, Big5, or ISO 8859-1 encoded table file and all of them will work perfectly?
Fixed this in document. UTF is specified by BOM, other encodings should be specified by "CP#" #- encoding number.
Quote
Does Kruptar even support SNES HiROM ExHiROM, LOROM00, LOROM80?
It supports LOROM scheme, as you know. Difference between LOROM00 and LOROM80 is by constant offset, which is added to pointer value, which is, in fact, ptReference. As far as I understand, HiROM pointer value is just a simple pointer, no need for special options for general case pointers.
Quote
The 00,01, and 02 high bytes are off by themselves in a single byte table.
Fixed that in document. 3 byte pointers can be splitted for 1 Hi byte and 2 Low bytes, set apart by ptSplittedPtrs
Quote
<$23>I hate Pascal strings.
Pascal strings are perfectly handled by standard plugin, which is always distributed with Kruptar. You just need to choose it in grPlugin.

10
Without aishsha, I probably would have brushed it off entirely.
Yeah, she's one handsome girl ^_^
Quote
If you have any interest this utility gaining traction in the English ROM hacking world, I'd highly recommend English documentation, help, and tutorials. The way it stands, it's full of abbreviations with a bunch of options and no explanation of any of them.
Well, that's why this topic and my tutorial is created for.
Ok, let's go step by step:
Quote
Pretend someone doesn't know what 'ptPtrToPtr' is or does in there lists.
As I said in doc: Pointer to pointer to string. - you mean I should be more verbose for even such simple things?
Quote
If I recall, it couldn't even handle the most basic cases of DTE, MTE/Dictionary and other table structures that fit within basic tokenization.
No, sir. Here is the project for MTE in NES Beetlejuice. And in archive of my english tutorial, you can find project for NES Final Fantasy 1, which uses DTE. All of that is handled by bare Kruptar natively - no plugins needed.
Quote
I didn't even see any documentation on the table file structure Kruptar expected either. I assume there is nothing but CLRF and StringTerminators?
Absolutely - only what I've mentioned in doc.
Quote
I can't say I understand the table encoding either. Is it UTF-8 only? S-JIS?
Any encoding you wish. I could expain more if you'll be more specific.
Quote
It seems to block things out sometimes like the wrong encoding for some characters, or it just does absolutely nothing and no message at all was displayed...
Most often it's not a problem with tabel file. Maybe you've messed something with text/pointer offsets
Quote
I don't think the program handled Pascal style strings, SNES pointers (except LoROM), pointer table hierarchies, sub-strings, data relocation, or split 24-bit pointers either to name a few.
Sorry?
Quote
Basically, I was only able to use this utility on basic NES level scripts with run-of-the-mill small pointer table followed by basic text strings with absolutely nothing fancy. Everything else would require plugins or simply be impossible.
Could you, please, give an example of what you call fancy thing?

11
I agree! it is a nice little underrated app :-) couldn't it easily dump scripts from any rom/iso game correct?
Right. Any file, almost any pointer format. Plugins can be used in case of script compression.

12
For me Kruptar - is the best utility for text. Not only because it's like Atlas/Cartographer in nice GUI pack. It's just more flexible, powerfull and user-extendible.
Still it's strangely not so popular here, so I've made a small document for those, who interested, which explains in short what is Kruptar, how it works and how to write fully functional plugins for that tool, using C and MinGW.
Questions and recommendations on improvement are very welcome.
Update:
Added code for Langrisser scheme: string starts with 3 ID bytes, which should be passed to script at editable form and each line should have odd length, so plugin also must make alignment of string.

13
ROM Hacking Discussion / Re: [psx] Einhander Virtual File System
« on: January 25, 2011, 01:01:10 pm »
Nah, what should I see? I know how and where to binindex and binpack0 are loaded. Just RAM addresses now:
Code: [Select]
sll     $v0, $v1, 1
addu    $v0, $v1
sll     $v0, 2
subu    $v0, $v1
sll     $v0, 2
lui     $1, 0x8005
addu    $1, $v0
lwl     $v1, 0x6F3B($1)
lui     $1, 0x8005
addu    $1, $v0
lwr     $v1, 0x6F38($1)
nop
swl     $v1, 0x40+var_2D($sp)
swr     $v1, 0x40+var_30($sp)
jal     CdPosToInt
move    $a0, $s1
lw      $v1, unk_80062AF4
nop
lhu     $a0, 0x12($v1)   # load 0b from .exe
lhu     $a1, 0x10($v1)   # load 00 from .exe
                         
sll     $a0, 2           # get 2c - offset in the binindex.bin
sll     $v1, $a1, 2     
addu    $v1, $a1
sll     $v1, 6
addu    $a0, $v1
lui     $1, 0x8006
addu    $1, $a0          # Add offset in binindex
lhu     $v1, 0x5080($1)  # The beginning of BinIndex in RAM
                         # load a word at 2C in binindex
                         # thats a sector offset (0x01BF)
move    $a1, $s1
addu    $s0, $v0, $v1    # V0 - is a sector offset of binpack0.bin at CD
                         # add offset from the beginning of binpack0
                         # and get absolute position
jal     CdIntToPos
move    $a0, $s0

The only problem here is that code itself has a bunch of commands like:

Which randomly access binindex.bin.
Still guess I'm right - there is no system in this infernal game.

14
ROM Hacking Discussion / Re: [psx] Einhander Virtual File System
« on: January 24, 2011, 03:00:45 pm »
And that's excacly what I was doing from the very beginning. Actually FLIRT signatures helped me a lot to get where I am now and that's the problem:
Almost no bytes are actually read from binindex (no breakpoints halted), except 0x2C - there are no overall processing function for processing binindex it is randomly accessed. I'm just confused, cause such file system structure has no sense - just obfuscation purposes maybe. Or maybe I was too blind to see what's really going on in the code.

15
ROM Hacking Discussion / Re: DMA problem
« on: January 24, 2011, 12:24:46 pm »
Они говорят 'wise men learn by other's faults, fools by their own'.
Здорово получилось, поздравляю.

16
ROM Hacking Discussion / [psx] Einhander Virtual File System
« on: January 24, 2011, 12:18:30 pm »
Good day, everyone!
I can't figure out Einhander's virtual file system. The game has one BININDEX.BIN and a few BINPACK0.BIN, BINPACK1.BIN, BINPACK2.BIN etc. BINPACKs have raw data in them and sector aligned.
Sure it looks like BININDEX is a header file with offsets of end files in it, but here, what I've learned:
1) BININDEX is sector aligned and has a few sections inside itself. There are no filenames in it - guess it's ok for some PSX games
2) Offset 0x2C in BININDEX is an end_file offset in sectors from the beginning of BINPACK0.BIN (offset is equal to 0x01BF). 0x2C is loaded from executable itself and there are no continuum data in the neighbourhood in sys.exe
3) I've corrupted data 0-0x20 in BININDEX and it didn't affect the game, also they are not readed. So all these meaning bytes are actually not used!
My guess is that this game does not have any VFS: just a file BININDEX with offsets, which are placed in random pattern and they are loaded in each time by appropriate code.
Can it be right, or I'd mistaken somewhere?

17
News Submissions / Re: Utilities: Genesis: A New Beginning
« on: September 10, 2007, 05:57:10 am »
wow! that's great utility and I'm dying of using it! BUT I'm complete noob with all these hook functions and with tracers based on .txt log files (it's completely different from FCEUXD tracer). I've searched, but haven't found any tutorials on these themes (and YES I've read g8z et al's two docs but didn't understand anything! :( (guess, it's for intermediate hackers) )

Pages: [1]