News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Dictionnary compression - how to create the dictionnary?  (Read 5640 times)

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Dictionnary compression - how to create the dictionnary?
« on: July 15, 2011, 11:04:30 am »
Hello everyone.

I'm french and I've been working on Samurai Pizza Cats translations (NES version).
For now, the script has been extracted and translated from japanese to french.
Now, I have to reinsert the translation, and this is where things become complicated.

Samurai Pizza Cats seems to use dictionnary compression for its script. The thing is I can't figure out how to create a new dictionnary from my own translated script.
To be more precise, what i want to find is an algorithm or a document that will help me to create a words list for my dictionnary that is specific to my script.

I hope you guys could help me or redirect me to the answer.

Regards.
Through dreams I influence mankind

Tito Santana

  • Jr. Member
  • **
  • Posts: 23
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #1 on: July 15, 2011, 11:26:36 am »
You want to find out which words in your script are used the most so that you can change them with the ones already in the game's dictionary, right?

If I understand right, then WordCounter can do what you want.

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Re: Dictionnary compression - how to create the dictionnary?
« Reply #2 on: July 15, 2011, 11:37:52 am »
Thanks for your answer Tito Santana.

Indeed I want to find out which words are used the most. However when I say "word" I mean group of x characters (not necessarily group of characters between two blank characters).
My goal is to optimize my script compression with the best dictionnary possible. It means that items that will be within it don't have to be complete.
Through dreams I influence mankind

Tito Santana

  • Jr. Member
  • **
  • Posts: 23
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #3 on: July 15, 2011, 11:48:53 am »
Maybe this utility will do it?

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Re: Dictionnary compression - how to create the dictionnary?
« Reply #4 on: July 15, 2011, 12:03:44 pm »
This tools seems very promising, unfortunately the link is broken :S
Through dreams I influence mankind

Tito Santana

  • Jr. Member
  • **
  • Posts: 23
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #5 on: July 15, 2011, 12:21:28 pm »
The 'Download This File' link? If so your refferals may be messed up because the link works ok for me.

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Re: Dictionnary compression - how to create the dictionnary?
« Reply #6 on: July 15, 2011, 12:29:55 pm »
In fact, when i try to download a file from Romhacking.NET server, i get 403 error (You don't have permission to access /utils/Martial.rar on this server.).
... and I have the same error when i try to displays pictures (which explains why i don't see logos anymore).
Through dreams I influence mankind

Tito Santana

  • Jr. Member
  • **
  • Posts: 23
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #7 on: July 15, 2011, 12:34:00 pm »
It is your referrers. What browser do you use?

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Re: Dictionnary compression - how to create the dictionnary?
« Reply #8 on: July 15, 2011, 12:35:23 pm »
Iceweasel 3.5.16
Through dreams I influence mankind

Klarth

  • Sr. Member
  • ****
  • Posts: 495
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #9 on: July 15, 2011, 12:37:25 pm »
Use ScriptCrunch with substring settings.  It'll take a few minutes to process a large script, but you'll get what you're looking for.  PM me if you need help on how to use it.  Basically, edit one of the included configuration files and run it from the command line.

Tito Santana

  • Jr. Member
  • **
  • Posts: 23
    • View Profile
Re: Dictionnary compression - how to create the dictionnary?
« Reply #10 on: July 15, 2011, 12:38:56 pm »
In about:config what is your network.http.sendRefererHeader set to?

Ridculle

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
    • TRAF
Re: Dictionnary compression - how to create the dictionnary?
« Reply #11 on: July 15, 2011, 12:43:54 pm »
It is set to 0.

I set it to 2 and both images/links came back :)

July 15, 2011, 12:50:41 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Thank you both for your help!

Now I'm gonna working on finishing this bloody project!
« Last Edit: July 15, 2011, 12:50:41 pm by Ridculle »
Through dreams I influence mankind

Gil Galad

  • Full Member
  • ***
  • Posts: 186
    • View Profile
    • Homepage of Gil Galad
Re: Dictionnary compression - how to create the dictionnary?
« Reply #12 on: July 19, 2011, 03:15:39 pm »
Here is an example to look at for ScriptCrunch. It's what I used for Dracula II since it has dictionary and I also added DTE.

Code: [Select]
[DTE]
; DTE at [80-BF]
DTEEnable=true
DTEBegin=80
DTEEnd=BF

[Dictionary]
; Dictionary enabled
DictEnable=true
DictBegin=FC00
DictEnd=FC7F
DictEntrySize=2
DictUseWholeWords=true
DictMinString=3
DictMaxString=10

[Script]
ScriptFile=Main_Test.txt
LineComments=//
BlockComments=/*,*/
LineStartComments=#
IgnoreBlocks=<,>

[Table]
OutputTable=dracula_eo.tbl
InputTable=dracula_e.tbl
OutputFrequencyTable=freq_1.txt
PerformSizeAnalysis=true

[Insert]
; DTE doesn't need the extra features for Dictionary insertion
InsertFile=
EndTag=
FixedStringLen=0
PadString=

I'm not using the bottom insert function but using all the rest. Using whole words for dictionary. Dictionary in this game is two bytes; FCxx. First byte FC says it's a dictionary entry and the number afterwards is which dictionary entry it is.

Homepage of Gil Galad || New Forum

“I don’t know half of you half as well as I should like; and I like less than half of you half as well as you deserve. ”