News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Magic Kids; Scubbing McD's from M.C. Kids  (Read 34929 times)

Alchemic

  • Jr. Member
  • **
  • Posts: 12
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #40 on: November 14, 2012, 10:05:06 pm »
I did some poking around, and I think I've figured out the text compression. It's a kind of Lempel-Ziv:

  • 4 bits: size of the "source" argument for pastcopies, in bits
  • 4 bits: size of the "length" argument for pastcopies, in bits
  • Stream of compressed bits; read each byte most significant bit to least

The stream of compressed bits consists of a sequence of these two commands:

Pastcopy
  • 0 (source) (length)
  • Copies previously-decompressed data.
  • The number of bits used by "source" and "length" is specified in the first byte of compressed data.
  • The "source" is how many bytes ago to copy from, and a source of 0 indicates the end.
  • The "length" is the number of bytes to copy, less three (so a length of 0 copies 3 bytes, 1 copies 4, etc).

Literal
  • 1 nnnnnnnn
  • Outputs one literal byte.

A sample decompression from 0x4AAC:

Code: [Select]
B5 82 41 23 10 A8 15 52 D1 69 B9 C8 00 0C 16 DB CC 82 A7 70 ...

1011 = Pastcopy sources are eleven bits long
0101 = Pastcopy lengths are five bits long

1 00000100 = '\x04'
1 00000100 = '\x04'
1 00011000 = '\x18'
1 00001010 = '\x0A'
1 00000010 = '\x02'
1 01010100 = 'T'
1 01101000 = 'h'
1 01101001 = 'i'
1 01110011 = 's'
1 00100000 = ' '
0 00000000011 00000 = "is "
1 01101101 = 'm'
1 01111001 = 'y'
1 00100000 = ' '
1 01010011 = 'S'
1 01110000 = 'p'
...

The letters are ASCII, and there are control codes in the text; 0x01 ends a string, 0x02 is newline, not sure about other codes.

Some known blocks of compressed text (end addresses inclusive):
  • 0x4010 to 0x4608
  • 0x4609 to 0x4AAB
  • 0x4AAC to 0x5082
I'm not sure if there are more or not.

Python decompressor source:
Code: [Select]
# M.C. Kids Text Decompressor
# Written by Alchemic
# 2012 Nov 13
#
# This code uses python-bitstring version 2.2.0:
# http://code.google.com/p/python-bitstring/

import sys
import bitstring





def decompress(romFile, startOffset):
    # Define some useful constants.
    BIT_PASTCOPY = 0
    BIT_LITERAL = 1

    # Open the ROM.
    romStream = bitstring.ConstBitStream(filename=romFile)
    romStream.bytepos += startOffset

    # Allocate storage for the decompressed output.
    decomp = bytearray()

    # Read the first byte.
    # (It specifies the size of pastcopy's two arguments.)
    copySourceSize = romStream.read('uint:4')
    copyLengthSize = romStream.read('uint:4')

    # Main decompression loop.
    while True:
        nextCommand = romStream.read('bool')

        if nextCommand == BIT_PASTCOPY:
            # 0: Pastcopy case.
            copySource = romStream.read(copySourceSize).uint
            copyLength = romStream.read(copyLengthSize).uint
            copyLength += 3

            # A copy source of 0 indicates the end.
            if copySource == 0:
                break
           
            for i in xrange(copyLength):
                decomp.append(decomp[-copySource])

        elif nextCommand == BIT_LITERAL:
            # 1: Literal case.
            literalByte = romStream.read('uint:8')
            decomp.append(literalByte)

    # Calculate the end offset.
    romStream.bytealign()
    endOffset = romStream.bytepos

    # Return the decompressed data and end offset.
    return (decomp, endOffset)





if __name__ == "__main__":

    # Check for incorrect usage.
    argc = len(sys.argv)
    if argc < 3 or argc > 4:
        sys.stdout.write("Usage: ")
        sys.stdout.write("{0:s} ".format(sys.argv[0]))
        sys.stdout.write("<romFile> <startOffset> [outFile]\n")
        sys.exit(1)

    # Copy the arguments.
    romFile = sys.argv[1]
    startOffset = int(sys.argv[2], 16)
    outFile = None
    if argc == 4:
        outFile = sys.argv[3]

    # Decompress the data.
    outBytes, endOffset = decompress(romFile, startOffset)

    # Write the decompressed output, if appropriate.
    if outFile is not None:
        outStream = open(outFile, "wb")
        outStream.write(outBytes)
        outStream.close()

    # Report the size of the compressed data and last offset.
    sys.stdout.write("Original compressed size: 0x{0:X} ({0:d}) bytes\n".format(endOffset - startOffset))
    sys.stdout.write("Last offset read, inclusive: {0:X}\n".format(endOffset - 1))

    # Exit.
    sys.exit(0)

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD\'s from M.C. Kids
« Reply #41 on: November 15, 2012, 12:01:33 pm »
I am at a lack of words to properly express the true magnitude of your awesomeness.

November 15, 2012, 09:02:00 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Those three blocks are indeed the entire game text. Brilliant job.

I don't suppose you could write a re-insertion script? I have zero python skill.
« Last Edit: November 15, 2012, 09:02:00 pm by keithisgood »

Alchemic

  • Jr. Member
  • **
  • Posts: 12
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #42 on: November 17, 2012, 12:32:57 am »
As requested, here's a compressor. Some observations:
  • It's not quite as space-efficient as whatever the original programmers had, so if you recompress the game's text without any modifications, you'll get a result a few bytes larger than the original compressed data.
  • Mixed up some control codes in my previous post: 0x01 clears the text window, while 0x00 ends a string within a block.

Python compressor source:

Code: [Select]
# M.C. Kids Text Compressor
# Written by Alchemic
# 2012 Nov 16
#
# This code uses python-bitstring version 2.2.0:
# http://code.google.com/p/python-bitstring/

from __future__ import division

import sys
import bitstring





def compress(inBytes, sourceArgSize, lengthArgSize):
    # Define some useful constants.
    BIT_PASTCOPY = 0
    BIT_LITERAL = 1

    # Prepare for compression.
    currentIndex = 0
    endIndex = len(inBytes)
    output = bitstring.BitArray()
    output += bitstring.pack('uint:4', sourceArgSize)
    output += bitstring.pack('uint:4', lengthArgSize)

    # Main compression loop.
    while currentIndex < endIndex:
        bestSource = 0
        bestLength = 0

        # Compare what's coming up to what we've most recently seen.
        searchLimit = min(
            currentIndex,
            (1 << sourceArgSize) - 1
        )
        for i in xrange(1, searchLimit):
            # Don't look too far ahead at what's coming up:
            # - No further than can be encoded in one command.
            # - Not past the end of the input.
            lookaheadLimit = min(
                (1 << lengthArgSize) - 1 + 3,
                endIndex - currentIndex
            )

            # Count how many sequential bytes match (possibly zero).
            currentLength = 0
            for j in xrange(lookaheadLimit):
                if inBytes[currentIndex - i + j] == inBytes[currentIndex + j]:
                    currentLength += 1
                else:
                    break

            # Keep track of the largest match we've seen.
            if currentLength > bestLength:
                bestSource = i
                bestLength = currentLength

        # Write the next command.
        if bestLength >= 3:
            output += bitstring.pack('uint:1', BIT_PASTCOPY)
            output += bitstring.pack('uint:n=v', n = sourceArgSize, v = bestSource)
            output += bitstring.pack('uint:n=v', n = lengthArgSize, v = bestLength - 3)
            currentIndex += bestLength
        else:
            output += bitstring.pack('uint:1', BIT_LITERAL)
            output += bitstring.pack('uint:8', inBytes[currentIndex])
            currentIndex += 1

    # Write the terminating bits.
    output += bitstring.pack('uint:1', BIT_PASTCOPY)
    output += bitstring.pack('uint:n=v', n = sourceArgSize, v = 0)
    output += bitstring.pack('uint:n=v', n = lengthArgSize, v = 0)

    # Return the compressed data.
    return output.tobytes()





if __name__ == "__main__":

    # Check for incorrect usage.
    argc = len(sys.argv)
    if argc < 2 or argc > 4:
        sys.stdout.write("Usage: ")
        sys.stdout.write("{0:s} ".format(sys.argv[0]))
        sys.stdout.write("<inFile> [outFile] [outOffset]\n")
        sys.exit(1)

    # Copy the arguments.
    inFile = sys.argv[1]
    outFile = None
    if argc == 3 or argc == 4:
        outFile = sys.argv[2]
    outOffset = 0
    if argc == 4:
        outOffset = int(sys.argv[3], 16)

    # Open, read and close the input file.
    inStream = open(inFile, "rb")
    inBytes = bytearray(inStream.read())
    inStream.close()

    # Compress the data.
    compressedOptions = []
    for sourceArgSize in xrange(10, 12):
        for lengthArgSize in xrange(3, 6):
            sys.stdout.write("Compressing: {0:d},{1:d}".format(sourceArgSize, lengthArgSize))
            thisOption = compress(inBytes, sourceArgSize, lengthArgSize)
            sys.stdout.write(" = {0:d} bytes\n".format(len(thisOption)))
            compressedOptions.append(thisOption)
    outBytes = min(compressedOptions, key = len)
    sys.stdout.write("Done.\n\n")

    # Write the compressed output, if appropriate.
    if outFile is not None:
        # Mode r+b gives an error if the file doesn't already exist.
        open(outFile, "a").close()
        outStream = open(outFile, "r+b")
        outStream.seek(outOffset)
        outStream.write(outBytes)
        outStream.close()

    # Report statistics on the data.
    sys.stdout.write("Uncompressed size: 0x{0:X} ({0:d}) bytes\n".format(len(inBytes)))
    sys.stdout.write("Compressed size: 0x{0:X} ({0:d}) bytes\n".format(len(outBytes)))
    sys.stdout.write("Ratio: {0:f}\n".format(len(outBytes) / len(inBytes)))

    # Exit.
    sys.exit(0)

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #43 on: November 17, 2012, 09:54:35 am »
How would you like to be credited in the game's opening credits? There's ample space for whatever title you may like.
"Ultimate Potentate of Text Compression" "Stephen Hawking Has Nuthin' on Me"? Your call.

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #44 on: November 17, 2012, 11:36:18 am »
How about "It's just LZ77"?

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #45 on: November 17, 2012, 12:47:02 pm »
Whatever the text compression is, it's beyond me. I'm just grateful for the help!

DarknessSavior

  • Hero Member
  • *****
  • Posts: 5031
  • Darkness.
    • View Profile
    • DS: No, not the Nintendo one.
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #46 on: November 17, 2012, 01:23:35 pm »
Whatever the text compression is, it's beyond me. I'm just grateful for the help!
"It's just LZ77" is a joke. I know nothing about compression, but I assume that's harder than say...RLE (the easiest, from what I've heard)? I think it's meant to be something like, "Oh, it was just LZ77. No big deal". That kinda thing.

~DS
Red Comet: :'( Poor DS. Nobody loves him like RC does. :'(
Sliver-X: LET ME INFRINGE UPON IT WITH MY MOUTH
DSRH - Currently working on: Demon's Blazon, Romancing SaGa, FFIV EasyType.
http://www.youtube.com/user/DarknessSavior

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #47 on: November 18, 2012, 10:22:27 am »
Ah. Well that's what happens when an English major reads a programming joke. Here I am, ruining the fun for everyone.

Crysta

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #48 on: November 18, 2012, 11:32:18 am »
Yknow, I just realized that the topic title says "Scubbing"...

Mirby

  • Jr. Member
  • **
  • Posts: 61
  • The Power's in the Pins
    • View Profile
    • Mirby Studios
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #49 on: November 18, 2012, 11:42:18 pm »
Well you can't just scrub away McDonald's from things, you need to scub it. It's much more in-depth and super effective against McDonald's residue.  ;D

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD\'s from M.C. Kids
« Reply #50 on: November 19, 2012, 09:27:57 am »
Ha! I'd never noticed that. Upside: a google search of Scubbing puts this thread on the front page. TAKE THAT PEOPLE WHO'VE MIS-TYPED "SCRUBBING BUBBLES!"

November 24, 2012, 10:04:36 am - (Auto Merged - Double Posts are not allowed before 7 days.)
New script written!



Still working out a few kinks with the re-inserter being less efficient. Hopefully will have the finished ips soon.
« Last Edit: November 24, 2012, 10:04:36 am by keithisgood »

Mirby

  • Jr. Member
  • **
  • Posts: 61
  • The Power's in the Pins
    • View Profile
    • Mirby Studios
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #51 on: November 27, 2012, 02:27:26 am »
i'm pretty excited 'bout this, i must admit. looks great :3

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #52 on: November 28, 2012, 02:27:02 pm »
Okay, not sure if this is error on my part or something else. Re-inserting the compressed the text into the rom (via Alchemic's Python script) causes mass problems with most all text related events. I don't have a lot of knowledge of programming, but I'm guessing the ROM uses pointer data which doesn't match to the new script, causing all sorts of funky problems.

If anyone wants to take a look and see where I'm going wrong, the text files corresponding to the game's 3 blocks of text data are linked below:

https://www.dropbox.com/s/kmo9v2cpq7qglkb/MagicKidsText1.txt
https://www.dropbox.com/s/lqnhjglper6w0vm/MagicKidsText2.txt
https://www.dropbox.com/s/igyt399z11pzok5/MagicKidsText3.txt

I've been using either the Python IDLE GUI or Notepad to edit these.

fellowroot

  • Jr. Member
  • **
  • Posts: 46
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #53 on: December 04, 2012, 11:59:37 pm »
I think that it is so funny that someone is actually doing this. Take that you big corporate companies who target their future customers when they are innocent children with their advertising. I could just see some fat kid playing this game back when it first came out thinking "Man, I could really go for some Mc Donalds about now" after the M symbols were burned into his retinas. I think the only way this advertising could have been any worse would be if it had an in game menu so that you could think about what to order while playing the game. lol

Anyway, I think you are doing a great job with this hack. All the graphics look nice and your idea is in good spirit. In fact, when your done it will probably be regarded as an actual game rather than an interactive sales commercial.

But I do have to say, I love some of the original music M.C. Kids has to offer. 

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #54 on: December 05, 2012, 09:01:47 am »
Yeah; even if I had the skill to swap the music, I wouldn't. Some good stuff.

MathUser2929

  • Hero Member
  • *****
  • Posts: 1644
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #55 on: December 05, 2012, 10:13:30 am »
I think it's interesting that you are trying to remove the McDonalds references. Id like to see what sprites you replace McDonald's characters with so post screens if you get the chance.

There was also a official hack of Cool spot to remove most of the 7up references. I think it was the official EU ROM. they replaced the 7up bottle in the intro with a generic bottle that was changed to a red color. The Spot character remained tho. I guess this was cause Fido Dido was the official mascot in EU, not Spot.

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #56 on: December 05, 2012, 02:17:53 pm »
Sure. Repeating some of the images posted previously in this thread, the graphical changes go as follows:

Ronald McDonald becomes Walter the Wizard:



Birdie the Early Bird becomes Bettie the Apprentice:


Grimace the [whatever Grimace is] becomes Silver the Gorilla:


The Professor becomes The Engineer:


CosMC the Alien becomes Cosmo the Astronaut:


Hamburglar becomes The Dragon:



And the swapped out portraits for each character:


I worked within the tile layout and palettes already used in the game to make it simpler on my end. I may make a few tweaks in the future, but overall I'm pleased with the sprite/graphics work. As soon as I figure out how to make the text compression script play nice, I'll have a finished .ips.

MathUser2929

  • Hero Member
  • *****
  • Posts: 1644
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #57 on: December 05, 2012, 03:03:17 pm »
Those are some interesting changes. The hack could make for a interesting playthrough.

fellowroot

  • Jr. Member
  • **
  • Posts: 46
    • View Profile
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #58 on: December 06, 2012, 04:18:36 pm »
When you are finished with the hack and its available to download I'll go buy and then eat a big fat Mc Donalds hamburger while playing it.  :D

keithisgood

  • Full Member
  • ***
  • Posts: 192
  • [DANGER ! ! ! ! ! ! ] [TERROR HORROR]
    • View Profile
    • keithisgood
Re: Magic Kids; Scubbing McD's from M.C. Kids
« Reply #59 on: December 06, 2012, 04:34:01 pm »
Imma let you finish but the McD's chicken sangwich is the best of all time.