News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Kruptar - advanced script extraction/insertion tool now documented  (Read 13474 times)

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
For me Kruptar - is the best utility for text. Not only because it's like Atlas/Cartographer in nice GUI pack. It's just more flexible, powerfull and user-extendible.
Still it's strangely not so popular here, so I've made a small document for those, who interested, which explains in short what is Kruptar, how it works and how to write fully functional plugins for that tool, using C and MinGW.
Questions and recommendations on improvement are very welcome.
Update:
Added code for Langrisser scheme: string starts with 3 ID bytes, which should be passed to script at editable form and each line should have odd length, so plugin also must make alignment of string.
« Last Edit: July 30, 2013, 02:16:36 pm by Griever »

RetroHelix

  • Full Member
  • ***
  • Posts: 147
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #1 on: June 03, 2013, 01:26:59 pm »
Nice one! Added your documentation to my bookmarks.  :thumbsup:

DSwizzy145

  • Sr. Member
  • ****
  • Posts: 427
  • Super Famicom Fanatic
    • View Profile
    • Super Famicom Game List A-C + SNES Game List
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #2 on: June 03, 2013, 01:57:02 pm »
I agree! it is a nice little underrated app :-) couldn't it easily dump scripts from any rom/iso game correct?

FAST6191

  • Hero Member
  • *****
  • Posts: 2626
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #3 on: June 03, 2013, 02:40:45 pm »
I discovered Kruptar when floating around the internet one day but I have not really investigated the user extensible/true scripting side of things too much yet. I had a quick skim of these and they could be quite useful, I always like to see scripting done in a reasonably common language as well (quite often I see people complain about professors that teach in their own language they made and then we do nearly the exact same thing).

@DSwizzy145 I am sure several of us around here could bring out their favourite insanely complex scripting language sporting game ( http://www.romhacking.net/forum/index.php?topic=12953.0 being one of my favourites) or oddbeat dictionary compression that would trouble things but by and large basic Kruptar (coupled with Oriton -- the table making program on the same site) is right up there with Atlas and Cartographer in terms of capabilities and has a GUI/workflow that I can see a lot of people get on with far better than those just mentioned. It (though more likely oriton) might leave some working in some Japanese to whatever circles wanting a tiny bit in some regards though most of that is just shortcuts that the Japanese to English world has come to use often.

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #4 on: June 04, 2013, 11:35:46 am »
I agree! it is a nice little underrated app :-) couldn't it easily dump scripts from any rom/iso game correct?
Right. Any file, almost any pointer format. Plugins can be used in case of script compression.

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5765
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #5 on: June 04, 2013, 03:04:41 pm »
I evaluated Kruptar some time ago while developing TextAngel. While it was probably the closest existing tool in concept to what I doing, it left quite a bit to be desired, especially for an English reader. I couldn't even use the utility to do anything worthwhile without help from aishsha. The program was not intuitive to me at all and English documentation was scarce, if any. In fact, all the links I previously had are dead now. There's really nothing that encompasses exactly what it can do or how to use it. Without aishsha, I probably would have brushed it off entirely. Even with aishsha, I only got it to do a minimal amount of things.

If you have any interest this utility gaining traction in the English ROM hacking world, I'd highly recommend English documentation, help, and tutorials. The way it stands, it's full of abbreviations with a bunch of options and no explanation of any of them. Pretend someone doesn't know what 'ptPtrToPtr' is or does in there lists.

I think it also relied too much on needing to write specific plugins for basic things. If I recall, it couldn't even handle the most basic cases of DTE, MTE/Dictionary and other table structures that fit within basic tokenization. Many of which are outlined here. I didn't even see any documentation on the table file structure Kruptar expected either. I assume there is nothing but CLRF and StringTerminators? I can't say I understand the table encoding either. Is it UTF-8 only? S-JIS? It seems to block things out sometimes like the wrong encoding for some characters, or it just does absolutely nothing and no message at all was displayed... Hell, it'll let me feed it a JPEG file! :-\ Again, documentation or program feedback would be helpful.

I don't think the program handled Pascal style strings, SNES pointers (except LoROM), pointer table hierarchies, sub-strings, data relocation, or split 24-bit pointers either to name a few.

Basically, I was only able to use this utility on basic NES level scripts with run-of-the-mill small pointer table followed by basic text strings with absolutely nothing fancy. Everything else would require plugins or simply be impossible. Perhaps I'm mistaken on some things, but without documentation, how am I to know? It's got a lot going for it, and I like the general concept. :) However, it's got a ways to go to be useful for anything beyond the most basic scripts it seems.
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

DSwizzy145

  • Sr. Member
  • ****
  • Posts: 427
  • Super Famicom Fanatic
    • View Profile
    • Super Famicom Game List A-C + SNES Game List
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #6 on: June 04, 2013, 03:48:24 pm »
Right. Any file, almost any pointer format. Plugins can be used in case of script compression.
Awesome! cuz i plan to dump Clock Tower - The First Fear PS1 version & maybe a couple of Unlicensed Super Famicom games to get the gist of things before I start to tackle on a few Adventure texted based Super Famicom games I've been wanting to see translated for sometime. ;D

Category

  • Jr. Member
  • **
  • Posts: 24
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #7 on: June 04, 2013, 03:56:11 pm »
I was unaware of Kruptar until this post, looks nice! Could help in my little Starflight MD project :)

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #8 on: June 05, 2013, 12:54:01 pm »
Without aishsha, I probably would have brushed it off entirely.
Yeah, she's one handsome girl ^_^
Quote
If you have any interest this utility gaining traction in the English ROM hacking world, I'd highly recommend English documentation, help, and tutorials. The way it stands, it's full of abbreviations with a bunch of options and no explanation of any of them.
Well, that's why this topic and my tutorial is created for.
Ok, let's go step by step:
Quote
Pretend someone doesn't know what 'ptPtrToPtr' is or does in there lists.
As I said in doc: Pointer to pointer to string. - you mean I should be more verbose for even such simple things?
Quote
If I recall, it couldn't even handle the most basic cases of DTE, MTE/Dictionary and other table structures that fit within basic tokenization.
No, sir. Here is the project for MTE in NES Beetlejuice. And in archive of my english tutorial, you can find project for NES Final Fantasy 1, which uses DTE. All of that is handled by bare Kruptar natively - no plugins needed.
Quote
I didn't even see any documentation on the table file structure Kruptar expected either. I assume there is nothing but CLRF and StringTerminators?
Absolutely - only what I've mentioned in doc.
Quote
I can't say I understand the table encoding either. Is it UTF-8 only? S-JIS?
Any encoding you wish. I could expain more if you'll be more specific.
Quote
It seems to block things out sometimes like the wrong encoding for some characters, or it just does absolutely nothing and no message at all was displayed...
Most often it's not a problem with tabel file. Maybe you've messed something with text/pointer offsets
Quote
I don't think the program handled Pascal style strings, SNES pointers (except LoROM), pointer table hierarchies, sub-strings, data relocation, or split 24-bit pointers either to name a few.
Sorry?
Quote
Basically, I was only able to use this utility on basic NES level scripts with run-of-the-mill small pointer table followed by basic text strings with absolutely nothing fancy. Everything else would require plugins or simply be impossible.
Could you, please, give an example of what you call fancy thing?

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5765
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #9 on: June 05, 2013, 03:11:14 pm »
Ok, let's go step by step:As I said in doc: Pointer to pointer to string. - you mean I should be more verbose for even such simple things?
Yes. What does it mean? What does it do? You select the option as true. So, this pointer table is full of pointers that point to pointers that point to a string. Great. What if the pointer the pointer points to is a different format? How do you specify both formats? What if it doesn't actually point to a string but a string table. So, you need a pointer to a pointer that points to a string table? I don't know the answer to any of these because there's no explanation for how it works, how it finds the end string in that case, or what it does.

Quote
No, sir. Here is the project for MTE in NES Beetlejuice. And in archive of my english tutorial, you can find project for NES Final Fantasy 1, which uses DTE. All of that is handled by bare Kruptar natively - no plugins needed.
I can't open that project, too many errors about missing ROMs and tables. Where is the archive of your English tutorial? That's another area your document doesn't address. I don't know how DTE or Dictionary is handled in Kruptar. Why isn't it just in the basic table? It's simple tokenzation. You can't have a multi-character entry (ex. AB=abc) in your table?  It seems you need an entire special group or something. That seems crazy to me. What about dictionaries with and without (fixed width) terminators? Again, no information on this stuff. That's the kind of thing I'm talking about.

Quote
Absolutely - only what I've mentioned in doc.
Right.. So the table doesn't support anything but single characters. No multi-character entries, no control codes, no 'linked' entries or other features Cartographer, Atlas, and others support. That's what I figured. :(

Quote
Any encoding you wish. I could expain more if you'll be more specific.Most often it's not a problem with tabel file.
You don't a see a problem that you can load a JPEG file (or any file that exists) as your table and the program doesn't even say anything? Are you telling me I can input a UTF-8, UTF-16, S-JIS, EUC, Big5, or ISO 8859-1 encoded table file and all of them will work perfectly?

Quote
Sorry?Could you, please, give an example of what you call fancy thing?
Here's something simple that I believe is not supported in any way.

You have a a set of 24-bit pointers. Does Kruptar even support SNES HiROM ExHiROM, LOROM00, LOROM80? All I see is 'LoROM'. Which is that even? How about headers? I assume none, but it's not safe to assume anything.

Anyway, the upper byte is split off by itself in a single byte table, the lower 16-bits in another double byte table. So you have pointers.

00abab
01bbcc
024545

The 00,01, and 02 high bytes are off by themselves in a single byte table. The abab,bbcc,and 4545 are in another. Can ptSplittedPtrs do this? Read one byte and two bytes to combine into a pointer? I have no idea. It doesn't say anything other than it splits them in two.

Now, each of those 24-point pointers points to another pointer table (format irrelevant). Each of those pointers points to a string table (no pointers to each individual string). That's a basic pointer hierarchy.

Lastly, the strings in the string table look like this:

<$23>I hate Pascal strings.

Each string starts with the length (including the length byte) followed by the string with no termination.

This is just a small sampling of the types of things I'm talking about. I don't think Kruptar was able to handle any of the projects I tried to use it with. It's great for basic NES stuff and systems like the GBA or DS with nice absolute pointers and simple organizational structure with plain strings, but it just doesn't seem to do anything more.
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

justin3009

  • Hero Member
  • *****
  • Posts: 1617
  • Welp
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #10 on: June 05, 2013, 04:34:44 pm »
I was given files of Kruptar that allows editing of Tales of Phantasia's  script though the 300 roms inside it may be trying to load a decompressed format script. I haven't checked too much into it yet.
'We have to find some way to incorporate the general civilians in the plot.'

'We'll kill off children in the Juuban district with an infection where they cough up blood and are found hanging themselves from cherry blossom trees.'

xander

  • Jr. Member
  • **
  • Posts: 24
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #11 on: June 06, 2013, 05:10:18 pm »
Like everyone else i discovered kruptar long ago but there was no english documentation. I tried signing up on the forum thinking there might be a video/picture tutorial somewhere i could use for reference. Thier forum system wasnt even working (i gave up after 3 tries).
xander

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #12 on: June 07, 2013, 01:27:38 pm »
Thank you, dear Nightcrawler for a lively and comprehensive response, I'll try to fix all remarks.
What if the pointer the pointer points to is a different format?
Ok, emphasized that in document. Both pointers should have the same format.
Quote
I can't open that project, too many errors about missing ROMs and tables.
But of course you can't - I cannot distribute Beetlejuice(U).nes ROMs. Just choose your ROM right after error happens and everything will work just fine.
Quote
Where is the archive of your English tutorial?
Here. You will also need ROM to open Final Fantasy project.
Quote
I don't know how DTE or Dictionary is handled in Kruptar. Why isn't it just in the basic table?
That is. Check the project.
Quote
You can't have a multi-character entry (ex. AB=abc) in your table?
Fixed this in document. Supports multi byte and multi character entries, \n sequence, no linked entries and control codes.
Quote
What about dictionaries with and without (fixed width) terminators?
As Kruptar will use dictionary group for that - you will specify string length for that group - no need to double clarify that in turorial.
Quote
Are you telling me I can input a UTF-8, UTF-16, S-JIS, EUC, Big5, or ISO 8859-1 encoded table file and all of them will work perfectly?
Fixed this in document. UTF is specified by BOM, other encodings should be specified by "CP#" #- encoding number.
Quote
Does Kruptar even support SNES HiROM ExHiROM, LOROM00, LOROM80?
It supports LOROM scheme, as you know. Difference between LOROM00 and LOROM80 is by constant offset, which is added to pointer value, which is, in fact, ptReference. As far as I understand, HiROM pointer value is just a simple pointer, no need for special options for general case pointers.
Quote
The 00,01, and 02 high bytes are off by themselves in a single byte table.
Fixed that in document. 3 byte pointers can be splitted for 1 Hi byte and 2 Low bytes, set apart by ptSplittedPtrs
Quote
<$23>I hate Pascal strings.
Pascal strings are perfectly handled by standard plugin, which is always distributed with Kruptar. You just need to choose it in grPlugin.

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5765
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #13 on: June 07, 2013, 02:53:36 pm »
Good improvements so far. :) We only discussed a few options. I think it would be a good idea for you to go through and expand upon the functionality of rest of the options in the same manner. I could generate the same type of questions for most other options since there is no details about what they do specifically. Unfortunately, I don't have any time to go through further specific items as it was some time ago that I studied Kruptar and I only had notes of certain cases.

Both pointers should have the same format.
Right, that's one big area where Kruptar is no good for me. This basically a two-level pointer hierarchy. Often pointers to pointers are not the same format. In fact, there are typically even more levels of pointers to get to the text (say 3 or 4 levels). In addition, the end result may not be an individual string at all, but a string table (group of strings with no individual pointers). Kruptar does not handle string tables in a 'ptPtrToPtr' type context. It will only handle string tables one by one as a group each as far as I am aware.

When I insert scripts like this, I need all pointers in the hierarchy (sometimes of several different formats) to be updated accordingly. These are items that are a little more 'fancy' that I refer to, but very common for me nonetheless. Kruptar is good at abstracting basic individualized string and pointer tables, but it starts to fall short when the organizational structure becomes more complex. Some concepts like 'ptPtrToPtr' are a good band-aid way to add some support for these types of things, but it quickly becomes limited.

Where I have tried improve upon Kruptar is taking the 'Group' concept farther. There needs to be more building blocks and more levels (beyond just one level Groups) to be able to build more complex structures. If only Kruptar was open source, I'd probably have built upon it rather than start another utility.

Quote
Supports multi byte and multi character entries, \n sequence, no linked entries and control codes.
Since you mention Cartographer and Atlas by name, you might want to mention the differences and things Kruptar does not do that they do such as the linked entries and what not.

Quote
Pascal strings are perfectly handled by standard plugin, which is always distributed with Kruptar. You just need to choose it in grPlugin.
I don't believe so. Since there is no string terminator, how can Kruptar know if is a pascal style string or terminated string? Secondly, pascal style strings can come in two flavors where the length byte itself may be included or not included in the length. Also, the length 'byte' may actually be multi-byte in some games. It would be impossible for Kruptar to handle these situations without some specific configuration settings for these items so Kruptar knows what to do.


It seems like the last version of Kruptar was from 2011. Is it dead?
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

DSwizzy145

  • Sr. Member
  • ****
  • Posts: 427
  • Super Famicom Fanatic
    • View Profile
    • Super Famicom Game List A-C + SNES Game List
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #14 on: June 07, 2013, 09:46:07 pm »
Do you need plugins to get the script extraction/insertion working or can it be used for harder encrypted coded scripts as some hackers were complaining about sometimes? (No offence) Also do you happen to know what's the lastest version of this app? consider the time i'd check before was just July 2012 releases i believe.

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #15 on: June 08, 2013, 09:45:57 am »
If only Kruptar was open source
Ah, yes, Djinn does not want that for some reason :)
Quote
Since you mention Cartographer and Atlas by name, you might want to mention the differences and things Kruptar does not do that they do such as the linked entries and what not.
Done.
Quote
Since there is no string terminator, how can Kruptar know if is a pascal style string or terminated string?
By plugin. At simple string with byte terminator case, you select standard plugin, which searches for the terminator and returns appropritate length string. If you select ShortString.kpl, plugin will return string, based on first byte, which is string length. Kruptar also has a few modifications plugins for various schemes. You can check them all in Help -> Plugins... And, of course you can code yours plugin in Delphi or in C, if first byte is stringLength-1 or any scheme you will face.
Quote
Do you need plugins to get the script extraction/insertion working or can it be used for harder encrypted coded scripts as some hackers were complaining about sometimes?
Plugin is used for any extraction/insertion. Even in null-terminated cases. If you need to modify/compress string you can code your own plugin which does this.
Quote
Also do you happen to know what's the lastest version of this app?
7.1.1.17 - 22.08.2011
Latest version is always here
Yup, unfortunately lately Djinn didn't modify Kruptar at all. Guess, the tool works OK for her current projects. But there were some plans for writing Kruptar 8 which were not realized yet.
Quote
I think it would be a good idea for you to go through and expand upon the functionality of rest of the options in the same manner.
Sure, will do.
If anybody will have additional notes on lack of explanation in my document, feel free to point them out here.

Scio

  • Full Member
  • ***
  • Posts: 155
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #16 on: July 04, 2013, 09:44:17 pm »
Just giving my feedback - the program is pretty good, but I've found a bug in the way it tries to recalculate the pointers. The program will try to remap pointers in the order they appear in, but sometimes this will eat chunks of text, no idea why this happens. A quick test is to extract and reinsert the original script in the same place. Since the text has the same length, the program will just change its order. But I've found that sometimes it will eat a few lines of text. Happens a lot if there are unused bytes between the pointers (but not on a regular rate).

Griever

  • Jr. Member
  • **
  • Posts: 52
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #17 on: July 07, 2013, 07:30:38 am »
How did you managed to extract a scipt with unused bytes between the pointers with irregular rate? Have you added each pointer manually?

Scio

  • Full Member
  • ***
  • Posts: 155
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #18 on: July 07, 2013, 12:51:29 pm »
EDIT: I was going to explain the "bug", but nevermind. Looks like I found the culprit: the Source and Destination tables had a different entry for "00=". One had a space while the other one didn't. That's why it was eating words and filling the space with 00s.

Well, since that wasn't a bug, then I have nothing bad to say about Kruptar. It sure makes extracting and inserting more intuitive. We need more programs like these.
« Last Edit: February 06, 2014, 09:35:52 am by Scio »

vince94

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
Re: Kruptar - advanced script extraction/insertion tool now documented
« Reply #19 on: September 05, 2013, 02:43:22 pm »
Hey, could someone help me out with this program?
The pointers in the game I'm working on are calculated by subtracting ten for the header and flipping the digits (all text is located at 06XXXX, so the pointers only consist of the last two digits), and the only control code in the pointer table is [00]=end, which is after every pointer.
Every time I have it scan the pointer area of the ROM (030010 to 040010), it spits out garbage. I've looked at the tutorial page on the original post, and while a lot of it makes sense, I don't know enough about pointers to find my way around the List settings panel.

EDIT: Um...heh...anybody?
« Last Edit: September 11, 2015, 02:10:17 pm by vince94 »