Help Modifying Program (Custom LZSS Decompressor for F-Zero GX/AX)

Started by StarkNebula, April 10, 2013, 09:38:08 PM

Previous topic - Next topic

StarkNebula

Edit: Rereading what I posted, it's definitely lacking. Let me be a little more specific: My main hurdle at the moment is getting info on the ReadException return. I've looked around a lot and I've come up empty handed. The only posts similar have no answers and the only documentation I could find was on D Language's official website - which while somewhat useful, gives no details on how to manage the bug (as far as I saw). I'm curious to know if it has anything to do with the ring buffer used. The comment (in \src\lz) specifies the buffer starts at 0 specifically for F-Zero GX's files. Would shifting the start value solve anything? If it does, any idea how I could find the correct starting position for AX's files (in the header)? Any idea where to start?


I've been trying to modify a small script/program which unarchives a unique LZSS files - it was made to decompress F-Zero GX .LZ files. It works just great for that, however, it doesn't work for F-Zero AX (the arcade version). The issue isn't that big, though, as the actual program has a debug feature. My 2 issues are; 1) I only started picking up C a few weeks ago and thus cannot find what exactly is wrong or how to get it to work with AX LZ files, 2) the debugger spits out hex values, and while I can read them, I have no idea how it is supposed to help me.

On that note, I'd like to ask 2 things. Firstly, can anyone recommend a good tutorial on how to apply hex? Once you find how to convert it to binary or other, it's fine and dandy, but how do you use those values? Secondly, can anyone help with finding either the bottleneck or issue the script is having?

Here is the program, along with working examples in \examples\packed_examples. AX files which will not unpack can be found in \examples\ax_packed_examples. http://www.mediafire.com/?jjijnhev8yt8yej

The error the console returns is this:
C:\Users\...\gxpand unpack input output
UnLZ output\st87.gma.lz
std.stream.ReadException@std\stream.d<46>: not enough data in stream
----------------
47EA54
47E8CB
----------------

C:\Users\...\gxpand>save
'save' is not recognized as an internal or external command, operable program or batch file.

C:\Users\...\gxpand>close
'close' is not recognized as an internal or external command, operable program or batch file.


The debug feature can be found in \src\binarystream.d at the bottom of the document.

What I'm looking for is any tips and some guidance to figure out how to solve this. If someone knows/wants to modify it themselves, fantastic, but if all you can do is point me in the right direction, that'd be great.


r57shell

Some details:
Program entry point: gxpand+ax\src\main.d 266 line
int main(string[] args)
Then u use "unpack". Program goes to 282 line
unpack(dirEntry(inputPath), outputPath);
Enters into unpack function at line 184
void unpack(DirEntry input, string outputPath)
Then going into 192 line, because it is file.
Stream inputStream = new std.stream.File(input.name);
scope (exit) inputStream.close();
unpackFile(inputStream, outputPath);

Enters into unpackFile function, which is located at line 141
void unpackFile(Stream input, string outputPath)
Next, going into branch at line 147
writeln("UnLZ ", outputPath);
Creating memory stream, and at line 152:
gxLzDecode(input, output);
Starts main decoding function. Entering into gxLzDecode defined in file "\gxpand+ax\src\gx\lz.d" at line 305:
void gxLzDecode(Stream input, Stream output)
{
BinaryStream s = new BinaryStream(new EndianStream(input, Endian.LittleEndian));
uint inputSize = s.read!uint;
uint outputSize = s.read!uint;

ubyte[] inputArr = new ubyte[inputSize];
input.readExact(inputArr.ptr, inputArr.length);

ubyte[] outputArr = lzDecode(inputArr);
enforce(outputArr.length == outputSize, "gxLzDecode: Invalid stream.");
output.writeExact(outputArr.ptr, outputArr.length);
}

Here we see, that new BinaryStream created based on file from filepath <input>.
Right from beginning of file, it reads two "uint" (unsigned integers but I don't know how many bytes).
First intended to be input size, and second intended to be output size.
And, right after them, reading input bytes with length = input size.

So, we see there is reading from BinaryStream instance, which is defined in file "\gxpand+ax\src\gx\binarystream.d"
And, there is method "read" at line 17, which is used for reading "uint":
T read(T)()
When reading of uint occurs, then in this function program going into branch at line 29
source.read(x);
But, source is instance of FilterStream, which is defined in library "std.stream" which included at line 2.
So, if you check reference, you will find here:
class FilterStream: std.stream.Stream;
It means, that FilterStream inherits Stream, and some before
abstract class Stream: std.stream.InputStream, std.stream.OutputStream;
It means, that FilterStream inherits InputStream, and somewhere on same page inside description of InputStream:
Quoteabstract void read(out uint x);
Read a basic type or counted string.
Throw a ReadException if it could not be read. Outside of byte, ubyte, and char, the format is implementation-specific and should not be used except as opposite actions to write.

I don't want to test this program, and I don't run it at all. That's all just my guesses.
But, I don't think, that your files less than 8 bytes to make exception of reading two ints.
I guess, when program reads incorrect file size, which is much bigger than input file, and trying at line 312 of file "\gxpand+ax\src\gx\lz.d":
input.readExact(inputArr.ptr, inputArr.length);
It fails. Because... From documentation:
Quoteabstract void readExact(void* buffer, size_t size);
Read exactly size bytes into the buffer.
Throws a ReadException if it is not correct.

P.S. I never ran it. It's just my guesses. I don't even know D language. This is some simple example how to understand someone else's code.