News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - aweigh

Pages: [1] 2
1
When I normally use a hex-editor (CrystalTile2 is the one I like using) in order to do simple stuff like:

- Translate into English an RPG's entire list of items, spells, enemy names, etc. Simple stuff that doesn't require fluency in japanese.

This is what it looks like (game in screenshot is Elminage 2 for PSP):


That "Longsword" wasn't originally in English. I did that. Go me, yay.

However, I recently got hands on RPG called "Students of the Round", and I thought I'd caught a lucky break because it has all files already "out", and even a folder itself dedicated to "table"-type files, i.e. the listings of the stuff I mentioned!

But this is what I'm looking at right now when I open the item.dat file:




Um... obviously I'm just a noob, but I guess this is my first encounter with compression? or is this "encryption"? Hell, please just someone be kind and tell me what all those 3's mean!

I tried using CONTROL + F to search for individual characters from a weapon's name, until I would eventually get a hit on one of the characters, and I noticed that there is definitely "weapon info" in that mess.

In the picture, look at the offset highlighted in yellow, and look at 0x4, i.e. the sudden "+3"; that is definitely the weapons accuaracy bonus. Checked throughout rest of the file and my theory holds.

So, I don't wanna jump ahead without even waiting for the help I'm asking for, but also: what do I do now?

2
THAt was a very good post. i can't say i disagree with anything you said.

my main point has basically become: "for a person with zero programming knowledge they can open a japanese computer RPG game's main .executable in any hex editor and scroll down until they find japanese text which they can then translate by hand".

for most of "these type" of users, and i include myself among this groupd, the goal is not to gain knowledge but to complete a game translation. if completing this game's translation requires a months-long immersion in assembly in order to hack the game's font then great, however it would be preferable if not because that's a lot of time "wasted" on ASM stuff; remember that for the majority of these would-be translators they are not doing it to release the translated game to the "scene", in fact the "scene" would be repulsed by their efforts and most probably badmouth their efforts and send the would-be translator into a confused depression as to what exactly he or she is supposed to do.

no, this theoretical group of would-be translators i am painting in this example, and i use myself as the basis, began to become interested in hacking game text because quite simply they ran across a japanese video game (or other language) that they've been wanting to play for years, and YEARS, and for whatever x or y amount of reasons it is NOW that the necessary steps are taken into doing something about it. the goal is not to release but to translate the game so you can PLAY IT YOURSELF.

from that point and as the person grows and learns more about everything related to hacking game text via hexadecimal editors they will absolutely start branching out and learning about other "methods". always remember that none of these "other methods" were on their radar, remember that this person is a gamer and has (probably) never taken a single comp. sci. course.

to these persons there is simply nothing simpler than opening a game's .exe or the game files in a hex editor and doing _immediate game-changing translation work_ on the game they want to play.

this will VERY quickly cause them to slam headfirst into all sorts of problems for which they don't have any answers and the ONLY answer they have is that the closest they've ever come to programming has been through hex editing. temping to call this scenario unique to myself but trust me you would be very, very surprised at just how many people all of what i've written describes to the TEE.

it is completely up to them whether or not they will be writing text extraction software that handles all of the game's pointers 1 or 2 years down the road from the day they first opened a game rom or file in a hex editor....

because, in the end, if for example they can translate all of the text using only crystaltile2, i.e. they are very lucky and their game of choice features no compression or other nasty things AND features a workable font that natively supports latin-ascii characters and all of that good stuff, then quite probably the only time this person will venture away from the text editor will be when they reach the moment when they realize they also have to translate the japanese text in the game's textures.

of course, as i mentioned previously, all of this mainly applies to would-be translators fixated on a SMALL-SCALE game which, having no knowledge of how to manipulate data before this, they will dive into using a hex editor. gigantic games that feature thousands of lines of text strings are OF COOOUURSE out of the scope of being worked on with 1 single hex editor. that goes without saying!

however the "small-scale game" caveat also applies to "small-scale hacks" like... you guessed it, someone who wants to change FF3's life spell's spell level! 1 or 2 days' worth of googling chracter control codes and shit and they will be able to do this.

any other kind of reply, such as "stop trying to do that and instead first learn all of this stuff which will not immediately help you change that spell's spell level" tends to rub people the wrong way.

3
...ok i've had a lot of time to calm down, heh.

i understand what you mean about the "backyard practices" and learning "bad habits". and i agree wholeheartedly that game-hacking (and/or "japanese-to-english translation", which is what 95% of the romhacking.net and gbatemp.net userbase want to achieve) is indeed incredibly difficult.

it is reverse-engineering, plain and simple. i've mentioned in my posts in this thread, and in another thread that is also titled something about hexadecimal editors, that i keep failing to see how a beginning game hacker (i.e. japanese-to-english translator) is doing himself or herself "wrong" by starting out "full hex" and learning from hexadecimal --> ida debugging and emulator debugging and cheat engine usage --> actual programming study.

now, see, here's the detail that i finally figured out; the thing that makes these hexadecimal editor usage threads make complete sense to everyone reading right now:

- the majority of the people posting on rh.net and gbatemp.net are not programmers and do not come from a programming background. myself included.

- to someone with zero programming knowledge and a game which they want to translate this theoretical rh.net/gbatemp.net user will, obviously, opt to open the game's files first in a hex editor because that is probably what one of his "buddies" will recommend they do first.

- this person is going to reach the following: "go ahead and continue translating all of this japanese text that i'm finding inside these files using this hex editor and continue making actual in-game progress", OR... "stop completely what i'm doing and take a step back and decide to put on hold this game's translation until i spend the necessary amount of time studying all of the programming knowledge i will need".

which do you think these users will choose? I know which one I chose! and because I chose the first option, I have already translated 1 game into english and have another half done.

HOWEVER... after exploring as much as could be explored via hex editor I obviously saw all of the limitations that I faced with each and every game I wanted to translate. this person will see that the japanese games all have monospaced fixed-width double-width fonts. I cannot do much about this via hex editor, so I must explore other avenues. And now this person will google and learn about the things he or she needs or wants to do which can be only be done by writing custom tools that will:

- extract text into a seperate tool for translation (and all of the baggage that comes with a tool like this).
- expand and patch all of the necessary shit for variable-width.
- (almost definite) probability of having to create or modify a tool for extracting graphical images.
- editing images and then re-inserting them into the game.
- translation-patch creation and distribution for the game.
- extraction tools for full-motion videos, audio and subtitles and other stuff like that.
- editing those things and re-inserting them.
- (probable) creation of new textures and new graphics for the game to accomodate the translation.

by the time this person is finished doing this game's translation the last time they will have edited the game files using only a simple hex editor will probably have been months, if not LITERALLY 1 or 2 years ago.

(the average time required for a translation group/person to go from zero programming knowledge to implementing variable-width and graphical hacking, etc, of a game is 1-2 years. This time-frame is taken from the anecdotal progress-blogs of translation projects for games such as Mother 3 and many, many others).

now we come back to the beginning: a person with zero programming knowledge can translate a game's text, AND even modify its font if they get creative with the emulator debugger + IDA + Cheat Engine and find the hexadecimal values for the font "weight", "width" and "height", and/or the hexadecimal values for the amount of pixels in-between each drawn game-letter.

And if they're lucky and the game does not require ANY font modification then they can probably translate the game's text as long as it is not compressed.

This is of course assuming it is a small-scale game, obviously, as 1 single person translating via hex editor a game that is massive such as a final fantasy or whatever will probably never finish, even assuming the font does not require modification. (yeah right!).

so, once again: a rh.net/gbatemp.net user on the majority is not a programmer and does not come from a programming background; they are people that usually want to translate 1 specific game into another language AND/OR they wish to "mod" that 1 specific game in odd ways that is basically only for their own private use... such as the OP asking how to change FF3's life spell's spell level from 5 to 4, and also asking how he could go about editing the ff3 "maps".

obviously a hex editor isn't going to help him much, if at all, in the map thing... but here's the thing: instead of immediately telling this person that he has to completely stop what he's doing with FF3 and instead go learn "programming knowledge" because he "does not know what he is doing"; the friendly neighborhood rh.net/gbatemp.net poster could reply instead with:

- the tools he'll need to extract the maps, the tools he'll need to edit them, and how to do this step-by-step, or at least with google links to relevant forum posts.

- explanations about what a hex editor can do for him and what he can do with it, and in this particular case the rh.net/gbatemp.net poster could download the ff3 rom (or plug his cart into his PC) and do the LIFE spell's spell level change and then demonstrate step-by-step how to replicate it using a hex editor, AND/OR provide relevant forum post links of similar things.

when the person, i.e. the OP in this scenario, begins asking about things that are beyond this scope then a helpful reply instructing them in the stuff they need to learn accompanied by hopefully many, many google links!

if this type of reply is anathema to you and instead you can only think of replying with all of the things he is "lacking" and without anything positive/concrete to ADD then your reply is of 0 (zero) use to the poster. and when in doubt... just don't reply!

an empty thread can probably serve better than a 2 or 3 pages of programmers arguing with complete beginners who do not know a programming language from the other about the best way to achieve something as simple as changing ff3's life spell's spell level.

and that's really the crux: the majority of the people posting for help on things like this OP are concerned with translation and a translation can be done via a hex editor. many have been done. especially for simple, small games like nes/snes roms that are well documented.

if a hex editor-only translation of a game offends you because it is of "poor quality" then remember that the person who did it... did not do it for you!

they did it so that the game could be enjoyed by people who really, really, REALLY want to play it! regardless of whether the translation is "top notch! master stroke!" or "dude did it using only a hex editor and atlas".

THE ONLY THING THAT MATTERS IS THAT A GAME CAN BE PLAYED IN YOUR LANGUAGE. NOTHING ELSE. HOW IT WAS ACHIEVED DOES NOT MATTER.

 

4








Please don't say such an ignorant thing as to tell someone who wishes to learn about game hacking that he should literally STOP using a hex editor. That is probably the most offensive and willfully detrimental piece of advice I've ever seen thrown to someone. How dare you tell someone to NOT LEARN?

I would be less offended if at least there had been then been many other examples of all of the other things he should be googling/studying/learning instead of hex editing but nope, absolutely nothing... Telling someone to stop doing something without a valid reason as to why they should stop and then not providing alternatives is worse than giving bad advice it's basically sabotaging the person.

Of course he can change a spell's spell level using a hex editor. In fact it's probably, _probably_, the simplest and fastest way to make such tiny changes in an RPG game.

---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------

My point of view on this "subject" is that hex editing is the single most essential tool in a game translation project. It is not the BEST tool for any single area, as there are better ways to translate text than by editing the strings in a hex editor; and it is not the BEST tool for editing anything at all that has to do with images/textures as you would need to be a bonafide hex-editing genius overlord to accomplish a texture edit that you could do in 2 minutes by simply extracting the game images and editing them with software and then re-inserting them; and it is certainly not the BEST tool for changing around game code as the majority of things needed in a translation project such as changing font width or hacking a new pointer table so you can make your own pointers and give yourself longer text strings to work with and of course for the implementation of variable width-- none of these things are done "better" with a hex editor".

HOWEVER beginning with a FIRM grasp of hexadecimal values can do and how the game itself utilizes them is extremely helpful for: DEBUGGING, REVERSE-ENGINEERING AND CODE MODIFICATION (ASM/ETC); and last but not least hex-knowledge is probably the single most important tool in the translation box when translating actual game text. Knowing the nitty-gritty of how what a HEADER is and what it does and knowing what CONTROL CHARACTERS for CARRIAGE-RETURN/LINE-FEED/ETC are of unbelievable importance.

Knowing what double-width is and why it exists and how it used and knowing what single-width is and why it exists and how it used and knowing the history and makeup of CHARACTER SET CODIFICATION, all the way back to the 1960's, will provide anyone with a better understanding of game translation due to the simple fact that they can immediately know how and why an NPC dialog box stops where it stops, and/or why TEXT WRAPPING issues happen.

These are very simplified examples, I know. I am not saying that you need to translate an entire game using a hex editor (although I have, and I enjoy editing hex more than I do implementing ASM OPCODES or editing textures for translation in Photoshop or dealing with anything else really).

I realize that you are speaking about this subject, i.e. game hacking, in a COMPLETE context. As you said yourself, level editing, etc.

I'm coming from a purely translation-oriented point of view as I have never been interested in changing a game's levels or in doing things like changing Cloud's 3D MODEL with SCOOBY-DOO sprites. Whatever floats your boat, I say, and indeed for those type of things you will NEVER, EVER need to use a hex editor!

However, some of the things one can do by editing hex values which a lot of people would say "should not be done in a hex editor" and instead done with "tools":

- I can open one the file containing the entire list of item data in any of Starfish studios' games (My goal is to eventually translate into english every single Starfish game, from 1999 to 2016); for example the ITEM.DAT file from Wizardry Empire 1, go to the item HOLY WATER which when used in battle produces the spell TZALIK which is useful versus undead enemies and I can LEARN the following things:

Each item has a hex value that assigns its DMG/Defense/Number of hits/special effect when used/special property it gives you passively or actively/what specific spell the item triggers when used that way/from what spell school that spell comes from/the amount of times that item can be used/etc. You can know instantly ALL OF THOSE THINGS simply from looking at the item's entry in a hex editor.

In the Holy Water item example I learned that it uses the spell Tzalik which the game assigns the hex value of '44', and guess what? when converted into decimal that relates directly to the placement of the spell in the game interface AND also relates directly to the spell LEVEL, i.e. 4th level in that spell school.

I also learned that it uses the value '68' to denote that this item is considered part of the PRIEST CLASS spell book. Items that trigger a MAGICIAN CLASS spell utilize the value '64' _in this particular game_.

Now guess what? I went back to the game's executable where all of the character class information and all of the spell information is there for our pleasure and I LEARNED that these values are no coincidence whatsoever!

Starfish utilized the hex value '64' as an invisible control character inside the double-width abbreviation of the Magician spell book, in this case what you see is M A G, and THAT is why items that utilize Magician spells are assigned the 64 value.

This isn't necessarily helpful at all for actually translating TEXT, _but_ it IS a completely LEGITIMATE example of how hex editing can be utilized to do things that people always tell you cannot be done or simply should not be done via hex. Armed with this game-specific knowledge I can play around and mod all of the items, all of the classes, all of the spells and switch whatever I wish around and effectively make a gameplay mod... all in less than 10 minutes and using a hex editor without ever using any other tool.

Now, the thing is the original poster was asking about two things: mainly about editing text, (naturally), and about how to change a spell's workings. I wanted to try to communicate that both things can be easily done with a hex editor if you study, REALLY study, how hexadecimal values relate to in-game factors.

Obviously if his questions were about how to extract a complete game script from a compressed game archive and then build a text-editing tool that will re-insert the text back into the archive and modify the appropriate headers then of course no one in their right mind would say "brah use windhex32" or whatever. Same with if he wanted to modify the game's music/sounds, or the game's textures/models.

If he HAD asked about that I would now be writing an extremely long post about how to use, STEP-BY-FUCKING-STEP, the myriad CONSOLE-SPECIFIC tools that are currently available so he can go and rip the textures/models/music/etc and then what programs he needs to download so he edit them, how to edit them to achieve what he wants, and then finally how he can go about re-inserting them.

I'm not saying It would be a guarantee of what I write working for him, since every game is completely different (a TRUTH I have had to learn the hard way...) but it would at least HELP HIM GET STARTED.

And, btw... STILL no need for any programming knowledge as of yet in ANY of these examples.

Where programming knowledge "comes in handy" is when reverse-engineering, which is what I'm teaching myself right now. (I need to implement a variable-width font hack into Wizardry Empire 3: Return of the Emperor for the PSP console). And fuck, I'm being generous here as reverse-engineering and implementing a few opcodes and/or expanding a "rom" are not REAL PROGRAMMING.

Some of the people in these communities, mainly here but also in GBATemp (they're much nicer and much more helpful to people than the people here on RH) have a somewhat high opinion of themselves in terms of how they are master coders and how game hacking is all about being a l33t programmer. Sure, in many, many examples, but not in others; such as someone asking how to translate into english a text string for example.

I had excellent success modifying the Wizardry 1 font utilize ADA/Cheat Engine but then I hit the hard truth that the second game's font I need to hack, Empire 3 on PSP, is not x86 architecture. I DID NOT EVEN KNOW WHAT x86 IS BEFORE THIS. However I googled and taught myself and of course asked for help from people (not from gbatemp/RH, but from an RPG website I post in, where one person actually went and step-by-step showed me how to use cheat engine to locate strings and eventually locate and modify the _HEX VALUES_ that determined the font's width, height, pitch, precision, output, quality, pixel size and its weight.

Once again, he probably only did it because it is an x86 game and thus easy as fuck to hack since absolutely EVERYTHING is spelled out for you inside IDA or even Cheat Engine. A toddler could figure most things out. The problems are always in console game hacking as they are... tedious to work with... to say the least.

I choose to look at it positively: if it hadn't been for my deciding to begin translating Empire 3 on PSP I would never have run into the font problems and I would never be on the path I am right now studying websites and forum posts and googling obsessively on my free time and learning more and more about reverse-engineering.

Dabbling with Elminage 2, also on PSP, I was forced to learn to extract and edit textures and translate the japanese characters on the textures into english and then re-insert them into the game. I had NEVER BEFORE EVER edited a single image file in my life before this. Once again an obstacle became a great experience for learning. Note that I have not mentioned that a hex editor would be needed in this example!

Anyway, fingers are tired from typing lol

May 29, 2016, 09:30:09 pm - (Auto Merged - Double Posts are not allowed before 7 days.)

5
LOL. just wanted to chime in and say i agree with that these type of replies are the cuntiest, most pretentious and ignorant possible replies that this person's questions about hex editing could possibly merit.

1. to the person who is starting out with hex editing: ignore everything said here and google subjects like character control codes in hexadecimal editing; what are character codifications, what is shift-jiss, etc. start there and you'll naturally continue exploring these matters.

2. the short of it is that hexadecimal values are a mid-way representation of machine-readable and human-readable code. yes, if you open a snes rom with a hex editor you are indeed seeing the game's entire code but not in the way you're thinking, you're looking at a hexadecimal INTERPRETATION that nevertheless is very much code.

3. if you want to see much more detailed interpretations of a game's code download the program IDA Assembly software and... have fun :)

4. starting out i would say the easiest way to start translating game text is something like the following scenario:

- like you, i knew absolutely nothing about rom hacking and i also did not have a single clue as to what hex editing was. i have never coded or programmed anything before this point so i was as new as new can get. I had in my hands a japanese windows PC game called wizardry empire 1: the ancient princess, and i was desperate to play it but the game was in japanese.

- looking inside the game's folders i found that inside a folder there were some files that were named things like: ITEM.DAT, SPELL.DAT, MONDATA.DAT, etc.

- a friend adviced me to simply open those files with a hex editor and start carefully scrolling through the entire .DAT file(s) until I spotted anything I recognized, like a spell name or _anything_.

- I opened first the SPELL.DAT file, and immediately realized i was looking at all of the game's spells in japanese. and they were perfectly organized. I copied (by hand lol cos windhex32 doesn't support copy/paste) the japanese characters and then downloaded a japanese dictionary software and inputted the characters and then copy/pasted that into a google search and sure enough: it was indeed a spell and i knew immediately then that the one following after it in the file would be the level 2 spell, so on and so forth.

- armed with that knowledge I began doing what I now know is termed "corruption", but i didn't know that at the time: basically i just overwrote the japanese characters with english letters and then launched the game to see if the spells had changed. And sure enough, they had indeed!

so that was how i got into rom hacking. there is no better way to start than via pure hex :)

btw, i'm almost done translating that wizardry game. just waiting for some remaining NPC conversations to get translated by someone who is fluent and then hex-edit that in and done.

also, you're going to hear a lot about pointers: the best way to solve the pointer problem is learning to use IDA, hehe. IDA just tells you where shit is, it's great. But focus first on finding a game file that is clearly labeled, like the spell.dat one in my example, opening that in a hex editor (I RECOMMEND DOWNLOADING CRYSTALTILE2) and first do corruption, see if shit changes in-game, then go back and start replacing japanese characters with english words.

it's easier with PC games as console games always require a lot of hoops to jump through in order to extract the relevant files from out of the game.

oh, one pro tip i learned: go to google translate site and set it to ENGLISH -> JAPANESE, and then write in english the word you want to find inside the game. for example, i was recently busy working on wizardry empire 3 for PSP and the very first thing i did to quickly find where the list of weapons/items were was:

- go to google translate, set it to eng->jap, write "Sword", copy the japanese characters that google spit out.

- open the game using a PSP-specific tool for extracting the files out of the PSP game iso's (UGMD or however it's called) and extracted every file and then I went through every single file and did a CONTROL + F with the japanese characters for Sword that I'd copied earlier.

- sure enough i hit the jackpot in less than 10 minutes and found the japanese characters for Longsword very quickly in a .BIN file titled UMDDATA.BIN. using that as a starting point everything else fell into place.

if you need to find where the chunks of japanese dialog is i have actually had a lot of success by utilizing the game's japanese wiki/faq fan-site. set google search to language->jap, and region->jap, and search for any sort of wiki/faq fan site for the game you want to translate. if you're lucky you'll find one! and once you find one these type of sites are great because they have lists of items and sometimes a lot of DIALOG QUOTES.

copy the japanese dialog quotes and then control + F through the game files in your hex editor (crystaltile2) until you find it! this has worked for me very, very well. of course it depends entirely on the game having these type of wiki-faq fansites ^_^

<('   '<)

oh! btw, regarding the thing about changing the FF3 life spell's spell level? one easy way to do it is to move around the CONTROL CHARACTERS that are normally invisible and are hidden inside ordinary looking text.

i don't really have the energy right now to explain this step by step but basically, and i'll just use another wizardry example for quickness:

- in all 3 of the wizardry empire games, AND also in the elminage games, (both series are made by the same dev, STARFISH) you will always find a section in the hex data where in the TEXT FIELD to the right you will see the Wizardry game's spell schools abbreviated and in double-width letters (i.e. letters that are wider than "regular" ones, which are single-width)

- so in this example the alchemy spell school double-width abbreviated word they used was of course, A L C, but the kicker is that if utilize CrystalTile2's "CODE CONVERSION" option it flips shit and you can see the invisible control characters. What you THOUGHT was a "C" is in reality a lower-case "h", and this lower-case "h" is a control character.

- if you then go to M A G, which is of course next/near to A L C since you're in the "magic section" of the file, you'll see that the "G" is actually a lower-case "f". switch those two around and now your mage learns alchemy spells and your alchemist learns mage spells. weee!

- these control characters are directly related to the topic of pointer tables and to pointers themselves! they won't help you FIND the pointer, as you need to do either math utilizing addresses and whatnot or use IDA or an ASM program that is similar to IDA. I also want to stress that a lot of people talk endless about pointers but in my opinion they are not that important to keep track of in terms of a game translation... why?

- if the text don't fit, ain't no pointers gonna help you there. not REALLY. only way is to hack the game's font (i.e. the game functions that order the game code to call the font, to then have the font function begin "drawing" letter-glyphs from the "physical" font-bitmap file and then eventually placing the glyphs into character codification supported by the game, which is shift-jiss usually if it is a japper game.

- so basically don't worry too much about pointers and instead i recommend that after you are comfortable hex editing shit or whatever to start learning how to really use the debuggers in the emulators. they are invaluable. Google terms like "how do i backtrace a function", "what is a stack", hehe. if the emulator debugger is not good enough download CHEAT ENGINE it's pretty good especially for listing DLLs, functions and strings.

basically what i'm trying to say to you is that yes, you can ABSOLUTELY do what you want to do in your OP WITHOUT SPENDING 10 YEARS IN A HYPERBOLIC CHAMBER RELIGIOUSLY STUDYING PROGRAMMING LANGUAGES THAT WILL NOT HELP YOU IN ANY WAY BECAUSE:

SPOILER ALERT! HEX IS NOT A PROGRAMMING LANGUAGE!

you can probably do the life spell switchero by moving around the control characters for the spell and spell levels. obviously this requires spending time familirizing yourself with the FF3 game rom and how it ticks. it just pisses me off when people treat other people like dirt and dismiss their search for improvement by insulting them and trying to get them to lose hart and stop pursuing something.

like what has happened in this thread. I recommend that everyone who posted that this person needs to just unplug his laptop ASAP and never again use a computer until he has spent 100 years training in C++ to go fuck themselves and to not post at all if they are not going to at least provide a helpful reply.

6
Well one very important link i left out is the website i used to read up on text encoding and its history all the way from the 60's up to today. I didn't include that link because I figured it was stuff "everyone knew". Basically It helped me learn what all those invisible characters I was living in fear of writing over when translating a string in fear of messing up a pointer or something of that nature. I no longer fear that scenario.

relevant link: https://www.cs.tut.fi/~jkorpela/chars.html#control

the information in the links is universal: the legend of heroes forum post explaining vertices and letter drawing and font table positioning and all that is 100% universal. the only aside really is when he goes into detail about prim triangles and rectangles: that's a PSP thing. Not the triangles/rectangles, mind you, that is also universal in drawing on-screen objects but rather in terms of that specific post.

and then in the Mother 3 blog post you will find the fundamental explanation of what the hell is a font, what is a letter, what is a bit, what is a pixel (in terms of fonts and letters), and the dude provides the most elegant explanation of half/full/variable are really just a matter of pixels.

from reading that blog post it dawned on me that VALUEx445254 is a thing. obviously those numbers are made up.

the mine sweeper link gives a greatly detailed step-by-step on using an assembly/debugger program, in this case IDA, to trace a function and then modify it.

that is the single most essential thing one needs to learn to do.

As for elminage 2 I already hacked in the battle menu / character menu texture buttons from psp Elminage 1 onto psp Elminage 2. It works flawlessly. Since I don't (yet) have image editing skills that was the fastest, and coincidentally the most ELEGANT solution possible. As always this sort of stuff is for my own personal use really... i konw a lot of people have a lot of very weird attitudes about "project quality" and would say that hacking in a prequel game's menu textures onto its sequel is something unforgivable and that the "project was of extremely poor quality"; i think that attitude is ridiculous. I plan to learn how to edit images eventually. the important thing is I have reliable scripts for extracting and inserting textures into and out of elminage 2 on psp.

i've done prelimenary translation of elminage 2 but i stopped because i realized i first need to hack into the game the variable width font (routines/opcodes/tables/lists/etc) take it from elmiange 1 psp iso and slap it onto elmiange 2 psp iso. But before THAT i first need to sped some time studying ASM.

eventually once that is done then i have to learn how to go about either making a script for extracting the text from elminage 2 scp.bin or acquiring an existing program for another game and modifying it. the game simply has too much text to translate it without a text-extraction/insertion tool...

i tend to focus on low-text rpgs like dungeon crawlers, such as the wizardry empire games. i think there's probably less than 100 dialogs in wizardry empire 1 for pc. i did them all in a hex editor. there's actually LESS dialog than that in the 3rd game for PSP; however once again i first need to hack the font otherwise there is no use translating anything, be it little or large amount.

7
I've been spending the past week and a half or so simply studying/reading all I can about the different character codifications until i can say i'm pretty much well-versed now in the stuff... I finally, *FINALLY* fully understand what the following things are:

- pointers
- pointer tables
- control characters
- what a font glyph set is (font.bmp)
- how the font glyphs are encoded with positional x/y/d values (give or take one thing or another; hardware/formats differ)
- the difference between variable width, half-width, full-width and monospacing.
- what a texture vertex is, what it does and what you can do with it|
- some very, VERY small knowledge of the most basic opcodes for both x86 and the PSP MIPS architecture

...aaaaand I finally get what most of you guys were telling me about knowing what tools are needed and the importance of either acquiring them or modifying existing tools for your own needs; and the fact that the hex-editor is only the ENTRY into the translation attempt and it lacks the many and varied utilities and/or functions that one will need throughout the course of a translation attempt.

right now the only tool i've been using for the last week is simply IDA program. I swear guys, if IDA's built-in hex editor supported shift-jis I would *ONLY* use IDA, except for when I need to compare files or do something graphics related or simply need a robust hex editor like 010 to do some weird shit.

right now i'm knee deep in poring over  the wizardry empire 3 eboot.bin for PSP; i've been using the past few days to just stop trying to translate the game and instead spend some time per day reading about IDA usage and MIPS opcodes and the like... I realize that unlike wizardry empire 1 (windows PC) I'm 100% going to need to implement some font hacks into Wiz-Emp-3 or the translation will be very, very poor.

At the moment I'm going over trace dumps of the game to see if I'm correct in where I think the value that controls the amount of pixels that are inserted by the game between each monospaced full-width letter-segment (the game uses canned text output); once I've found it and successfulyl changed it I then want to eliminate the kanji sets and make a pointer table / display list / character-width table / etc, of my own and see if i can put in some variable width letter drawing.

I do believe that will require to also hack the game's routine for prim triangles and i have to google how to change them into prim rectangles.



anyway what i also wish i can learn to accomplish is to write a tool to extract all of the text from the game. I've yet to actually google this subject in any depth as reading the ASM/encoding stuff has had me pretty floored.

Elminage 2 for PSP is the 2nd game I want to translate and attempt to implement a VWF hack... on the plus side I have Elminage 1 (one) for PSP and that game was officially localized into ENGLISH language so I have been spending a few hours (heh) poring over very carefully the way the Elminage 1 publisher/devs hacked in their VWF routines, opcodes, lists, tables, etc; EVERYTHING.

Just studying how they did it on Elminage 1 and then contrasting/comparing with the fixed-width monospaced font(s) in elminage 2 has been very illuminating.




it's all still very much a mystery really but i can sort of understand what it is i'm staring at, which is more than I can say for two weeks ago.

------------------

relevant links for anyone who might find them useful, just as i did:

This forum post from one of the persons involved in the translation project of one of the lost heroes games on PSP went and completely explained and detailed what prim triangles, rectangles, vertex addresses and vertices are and they are used by the psp hardware and how you can utilize this knowledge for your own needs; the post author also goes into detail explaining how letter drawing is done on the psp (and in general, of course, but specifically on PSP):
- http://www.heroesoflegend.org/forums/viewtopic.php?t=197

This forum post didn't really illuminate *much* BUT it did provide one unbelievably helpful step-by-step tutorial on expanding or extending a PSP game's "space" for the later introduction of your own code:
- https://gbatemp.net/threads/psp-asm-hacking-for-variable-width-font.374967/

This forum post about a guy asking for help on how to use IDA to pin down a specific function is... very, very good. The amount of detail the person who replied to him went into should be the same gold standard every single perosn should aspire to when replying to any question, from anyone, at any time. Always. No excuses!
- http://stackoverflow.com/questions/931257/how-can-i-find-the-data-structure-that-represents-mine-layout-of-minesweeper-in

This link is to part of the blog of the mother 3 translation team; in this entry the author goes into detail on the basics and the core concepts, or rather, the fundamentals, of just what exactly is going on when a letter is drawn from its glyph and onto the game and why/how half/full widths are a thing and how to work around such things:
- http://mother3.fobby.net/blog/2008/03/29/welcome-to-text-welding-102/

-------------------

Now... much less useful but still these are posts/links that i found helpful. This following link is for someone's personal wiki page where he archived how he achieved a VWF hack for the psp release version of persona 2:
- http://bbq.xyz.is/Persona_2:_Batsu#Make_font_variable-width

8
THANKS. Yeah i am, i was/am using 1.3.x. downloading now.

9
Can't you just hex edit a save file and locate the character's name and his abilities and use trial-and-error (i.e. corruption) to find the hex values you need?

10
I don't think that was "unhelpful" at all! That was a great reply. Thanks. And you're right about everything you said.

Just for the sake of keeping my post short I'll give one example: after my success at stumbling across Wizardry Empire 1's SetPixel / GetPixel functions I thought it would allow me to do the same with Wizardry Empire 3, which is a PSP game and not a PC game like the 1st gmae is; however it's been about a week so far and I've yet to successfully replicate my success on WE 3 on PSP.

HOWEVER! I have slowly, day by day, been learning more and more about WE 3's code and I understand what I'm seeing outputted in the PPSSPP's debugging log more and more. I firmly believe that EVENTUALLY I'll change the Pixel size on WE 3's font. In the 1st WE game, which as I said is on PC all you need to do is change a single value: "08" -> "10" and voila, the game's font is now 8 pixels instead of 16. Of course the real work is debugging the game and coming through its ASM routines until you find WHERE that value is.

On the PSP it is much, much, much more complicated. For starters the game has a buffer where it stores a lot of values, such as I suspect the pixel size one, and that buffer is continually read from and therefore it is not enough to change the pixel value(s) in one place; I think I have to do something like creating some sort of routine that makes the buffer redirect to newly created rom data containing the pixel values I want.

To be honest I've come to find out I enjoy trying to achieve these things much more than I do playing the actual game. I don't think I even like Wizardry Empire 3 that much; it is not as good as the 1st or 2nd games in the series. But I absolutely love coming home and spending a few hours trying to learn what it is I have to do to translate the game. I don't think I'll even play it after I translate it; I've already got 2-3 other japanese Wizardry-clones (dungeon crawlers) that I want to translate.

I started work on Wizardry XTH: Unlimited Students on the PS2 and while hex editing the executable is no problem (i.e. my changes reflect in-game without any problems), AND the game features latin-character support (no immediate need to figure out how to change the font)-- I've hit a complete roadblock with the game's only other game file, a file called DATA.DAT which is 800mb's and contains basically everything else that isn't in the PS2 executable, i.e. dialog and "lists of stuff" like items/spells/etc.

However unlike with the executable where I've already translated all of the menus and the like without any problems whatsoever when I translate something from the DATA.DAT file, say the spell "Cure"... IT DOESN'T SHOW UP IN GAME. The spell is still selectable and it works but THE WORDS ARE INVISIBLE.

So yeah, that's the first time this specific problem has happened to me as I've had a lot of stuff "go weird" in game when I edit a string but so far I've never had the in-game text go invisible but still functional LOL.

And like I said, I've come to find I enjoy this puzzle-solving. I have _zero_ clue why this is happening with this PS2 game but by god I swear I will I will find out. Unfortunately I can't find a good debugger for PS2 as the PCSX2 doesn't feature any... so I've been having to use Cheat Engine and while it "works" It isn't as intuitive as using the emulator-specific one, such as the PPSSPP debugger which has taught me a LOT about what I need to do and what I was doing wrong.

EDIT: I'm thinking the invisible text in the PS2 Wizardry game probably has something to do with pointer tables / or perhaps I have written over some invisible control character, or "pointer" character, maybe an invisible @ or B or whatever. You know what I mean. The problem is I've compared the original japanese DATA.DAT with my edited one using 010 Hex Editor and the only changes I've made are to simple japanese characters. I'm thinking I'll end up learning more than I ever wanted to know about pointers by the time I solve this puzzle.

11
Sounds like a lot of work. The reason I somewhat dislike editing images that function as menus and the like is that no matter what I do the end result will never look as good as the original game tile/texture.

It's frankly another facet of rom hacking where there is need of yet another skill set... i guess that just menas i'm going to have to learn. :)

BTW, i learned today how to implement a command, so to speak in the cheat engine debugger, to change a game's font to 8 pixels instead of 16! now i'm trying to do the same on a psp game and the referenced functions are all mostly the same, and infact they even use the same GDI32 but for whatever reason i ain't getting any joy.

Anyway i'll stop now since this thread isn't about this stuff. So yeah, a small section on basic step-by-step editing of common images, like rpg menus, how to do it / what to use / etc, and how to re-insert them on a per-console basis. That'd be sweet.

also it'd be amazing if you included as well in that section a detailed breakdown of the many things you can do with a game's font image once you've extracted it, all the way from stuff like shaving off the rough edges, so to speak, off each letter to stuff like replacing fonts and even maybe you could include a section on making your OWN fonts to serve a game's needs best and how to do it, ttf to per-console formats and the like and how to make it work.

now that would be awesome.

EDIT: oh, you should also make sure to include a detailed section on how to use debuggers, what they can do and what they are for and what you can accomplish with them and you CANNOT accomplish with them; debugging is quickly becoming one of the most important facets of rom hacking and almost no one knows how to use a debugger, myself included.

and you know? also a section on what tools are needed for what task, more specifically concentrating on whether you should use a rom hacking oriented hex editor or whether perhaps what you need to do at that moment could be better served with notepad++ and using its powerful macros and ability to set automatic line breaks. that's just a specific example but it's stuff that comes up. most people don't even know about macros for text/script editing.

also, i just thought... you really need a section all about tables and the crazy stuff you can do with them. and even more so what you can do with two tables at once. a lot of people don't know about DTE's and how you can stuff a table with them.

12
MAJOR update! I've been getting my feet wet learning about how ASM is "understood", and how you can use it and what you need to do beforehand... and anyway as an experiment I decided I wanted to track down the function that sets the Pixel size of the Wizardry Empire 1 (One) font, even though the game doesn't really need it as it has full latin-ascii support and ample text-length; but the point was to practice on a game I know literally inside and out before moving on to ASM'ing half-size pixel font on the 3rd Empire game on PSP.

Long story short: success!





!!!

Now I just need to buckle down and start hunting dowm the same shit I just spent half the day figuring out but on the third game on PSP. Should be fun! Especially since Wiz-Emp-3 does not use GDI32.DLL as the backend for it's font. It's going to be much harder to find. But I will. And once I do...

13
well, how would one go about translating a texture? like, open up MSPAINT and pick Letterbox tool and write letters on top of the tile? I honestly have no clue.

15
It expands on the clickety-click? Postcount +1?


16

May 02, 2016, 01:02:08 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Quote
STANDARD HEADER
By far the simplest and quickest pointers to work with.  These pointers appear somewhat frequently. Calculating them is simple (Final Fantasy 1 uses Standard Header pointers, and the procedure for calculating them is stepped through above).

 In brief, you:
1) Determine the position of the text. (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Drop the Ten-thousands digit, if there is one. (SAMPLE: 2335)
4) Break the number into two pairs.  (SAMPLE: 23 35)
5) Swap the number pairs. (SAMPLE: 35 23)
Voila, you have your value.

 SetOff X000
By far the most common type of pointers on the NES, SetOff X000 pointers are pointers whose position within the ROM is modified for whatever reason.  This amount of modification is indicated by the X000 value.  Why they're modified isn't important, only how.  To calculate them, you follow the same procedure as STANDARD HEADER pointers with one additional step.  (IMPORTANT NOTE: You can only calculate X by locating the pointer table, which is described below)  If you know X000, they are calculated by:
1) Determine the position of the text.  (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Add X000 to your value. (SAMPLE: if you found X000 to be 3000,
then 12335 + 3000 = 15335)
4) Drop the Ten-thousands digit, if there is one.  (SAMPLE: 5335)
5) Break the number into two pairs. (SAMPLE: 53 35)
6) Swap the number pairs. (SAMPLE: 53 35)
Voila, you have your pointer value.
If you do not know X000, you need to follow the steps outlined below in LOCATING THE
POINTER TABLES.  When you get there, read through that section below, and then come back
up here and re-read this one to gain the best understanding of SetOff X000 pointers.

 Some notes about SetOff X000:
1) Standard Header pointers are just SetOff 0000 pointers, but usually it's just easier to think of them as their own type.
2) For whatever reason, X seems to be 8 more frequently than most other values.
3) Although it is theoretically possible to have SetOff 1500 pointers, or some other value which is not a multiple of 1000, I have never encountered them.

 SEQUENTIAL TEXT
Technically, this is not even a pointer system, but I include it here because being able to recognize this when it's used can save you hours of headache and searching.  Sequential text systems don't use pointers, making their widths adjustable without any problem.  Customizing a YES/NO prompt would actually be as simple as changing:
Yes#No#texttexttexttext...
into:
Sure.#No way!#texttextt...
This way of doing things isn’t used very often, but some games do use it.
One important note: With this system, the game knows it's done when it reads enough strings. Therefore, you cannot simply remove a string you don't like without messing things up. If one string is slightly longer, the string to the right of it will have to be slightly shorter.
 FIXED-LENGTH SLOTS
Rarely used for text, this system is frequently used for item and spell listings.  The game knows that the text string is a certain amount of characters long, and reads them accordingly. You don't really need to do anything with these, and you can’t re-map them.

 SEQUENTIAL POINTERS
If you see these, run. I've only encountered them once, in Wily & Light no RockBoard, a Mega Man monopoly spinoff, and I'm still trying to figure out how on Earth they know everything they're supposed to.  Anyway, I'm including them here for completeness, but you're free to skip ahead.

 Sequential pointers are stored in their simplest form, two digits (as opposed to the 4 used in SetOff X000 and Standard Header pointers).  They're calculated as follows:
1) Determine the position of the text.  (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Drop all digits except for your ones and tens places.  (SAMPLE: 35)
Voila, you have your pointer value.
Seems easy, doesn't it?  Well, it's not, because as you try to remap these pointers, all kinds of weird things can happen without the hundreds and thousands place.  Space constraints are worse here than anywhere else.  Trust me, you really don't want to deal with these unless you absolutely have to.

 Well, there you have your standard pointer systems.  Are there others?  Probably.  I've only encountered one other system on the NES, and I'm STILL trying to figure that one out...

 So now do we finally get to learn about how to find the pointer table?
Yes, you deserve a reward for having read this far! Let’s pull all of this together.

 FINDING POINTER TABLES and your pointer system (the easy way):
(if you need an example, refer back up to the Final Fantasy one above)
Does the ROM use a SEQUENTIAL TEXT or FIXED WIDTH systems?
1) To determine if you've got a FIXED WIDTH system, just look at your text strings.  If you've got a bunch of strings that are exactly the same length (especially if they use 00s, FFs, or some kind of <NULL> code at the end to guarantee that), odds are you've got a fixed width system at work.
2) To test and make sure, locate a string you can easily get the game to display.  Overwrite it with something longer which runs into the next string in your hex editor.  Save, then load the ROM and bring up that item.  If it cuts off after a certain length, you've got a fixed width system.
3) To test if you've got SEQUENTIAL TEXT, find some dialog in the game and locate in in your hex editor.  Now, go into the string BEFORE it and add an <END> code right in the middle of it.
Save, and load up your ROM again.  Bring up the dialog you used before.  If it's different, then you've got a SEQUENTIAL TEXT system.

 If your ROM does not use the SEQUENTIAL TEXT or FIXED WIDTH systems:
1) Open the ROM in your hex editor.
2) Locate a string of dialog which you can bring up in the game easily or quickly. (Don't use menu options/other types of text, as these tend to be stored differently than main text)
3) Write down your offset.
4) Subtract 10 for the header.
5) Note down the last two digits of your value.  (If your adjusted position was 12345, the 45)
6) In your hex editor, Hex Search UP for this value. (the 45)
7) Usually, you'll find one close to where you started your search.  Regardless, write down the two digit number to the right of the one you just found (Let's say, searching for 45, your search returned the 45 in the following line:  AB 45 FE. Write down the FE)
8) Check the second digit in the number you just wrote down (In FE it'd be the E).  Compare that value with the hundreds place in your adjusted position (In 12345 it'd be the 3).  If they match, you may have found the
beginning of your pointer table.  In they don't, continue searching UP.
9) Since the values didn't match, we continue searching upwards until they do.  (Finally, we find it a line that includes 82 45 83)
10) Now that we've found a potential pointer, we need to test it.  Change the pointer so that the text will display differently (my favorite is to just set the ones place ahead by 1, so that the first letter doesn't display.  When that happens, you know it worked.)  Then you'll need to remap the pointer to another string of text, and see if it changed like you expected it to (What we did in the Final Fantasy example above.)
11) If the text changed as you expected it to, CONGRATULATIONS, YOU'VE JUST
LOCATED THE POINTER TABLE.
12) To figure out which pointer system you're dealing with, you only need to look at the pointer you've found and compare it to the position of the text you're using.  Take the first digit of the right number in your pointer (45 83 -> Take the 8) and compare this to the thousands digit of your position (12345 -> The 2).  If they're the same, you're dealing with STANDARD HEADER pointers, otherwise you've got SetOff X000 pointers (where X is the number you just examined from the pointer -8- less the number you just took from the position -2-, or 6 => Therefore, you're dealing with SetOff 6000 pointers)

 That's it.  This system will work for most of the games that many of you will want to hack out there.  Now that you know the pointer system, you can calculate the pointer for the first item in your text block.  Find that, and you've found the beginning of your pointer table.


 FINDING THE POINTER TABLES (a harder way):
If the above system didn't work for you, make sure you don't have any control codes (described below).  If you're sure that there aren't any in there, try this system:
1) Test for SEQUENTIAL TEXT and FIXED WIDTH systems, as outlined above.
2) Locate a string you can bring up easily in the game.
3) Using its position, calculate the pointer using the STANDARD HEADER system.
4) Search the hex for this value. Change every occurrence slightly (remember, change the second digit of the first number pair for minor changes).
5) Save, and load the ROM.  If the change you wanted occurred, then you know it's a STANDARD HEADER system.  Change matches one by one until you find the right one.
6) If that doesn't work, then calculate the pointer using the SetOff 8000 system.  Search for this value in the ROM, changing every occurrence slightly.
7) Save, and load the ROM.  If the changes you wanted occurred, then you know it's a SetOff 8000 system.  Change matches one by one until you find the right one.

 If these don't work, then I'm afraid you'll have to resort to:

 BRUTE FORCE POINTER LOCATION (the tedious way):
The system of last resort.  This will work 99.9% of the time if you know that you're not dealing with a FIXED WIDTH or SEQUENTIAL TEXT system, and that there are no control codes throwing you off (and in that 0.1% of the time, you don't want to deal with it anyway).
1) Locate a dialog string you can bring up easily in the game.
2) Locate that string in the hex editor.
3) Calculate the pointer for it using the SEQUENTIAL POINTER system (no, don't worry. Using this method does not automatically mean you've got sequential pointers).
4) From the beginning of the ROM, Hex Search through for that value.  One by one, change that value slightly.  Saving and testing to see each time if you've found the right one.
5) EVENTUALLY you'll come across it (hey, if you're lucky, the Hex Search will only turn up 500 or so occurrences).  FROM there you can figure out the pointer system as described under the easy way above.

 ...or you could go on datacrystal.romhacking.net and see if somebody else has found it and written it up.

 So, is that it?  Do I now know everything I need to about pointers?
Almost. There are only two more minor things for me to address.

 Empty Space in the ROM:
Fitting all of the text you need to into the ROM can be quite troublesome.  Fortunately, there is a solution...sometimes.  On some occasions there will be blank space within the ROM which you can use to fit some extra text in.  You can recognize this space because it's usually just a large block of 00s or FFs.  Look at the end of your ROM and you'll probably see a good example of it.
Now, because NES pointers are limited to four digits, so too are your pointers.  Therefore, your range will be limited to whatever bank the existing text is in.

 Control codes:
These are codes that programmers put in with the text which tell the NES how to display it.  Much like the font tables themselves, these vary from game to game.  End of Line, End of Page, and Pause are two common examples.
I bring these up because if the text string you're using has one (or more) of these at its beginning, the pointer will point to THAT instead of the first actual letter of the message.

 COULD IT BE?  YOU ARE FINISH!!

SOURCE: chrome-extension://gbkeegbaiigmenfmjfclcdgdpimamgkj/views/app.html

17
Yeah I ran into my lack of knowledge of editing images with Elminage 2. I ended up editing the Elminage 1 (which was localized into english)-- the Elminage 1 menus and then replacing the japanese Elminage 2 menus with the english ones from the first game. It worked like a treat because there were no menu changes from E1 to E2. However for Elminage 3 they changed it a lot so I won't be able to do that again.

Screenshot:

18

May 02, 2016, 01:03:11 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
Quote
STANDARD HEADER
By far the simplest and quickest pointers to work with.  These pointers appear somewhat frequently. Calculating them is simple (Final Fantasy 1 uses Standard Header pointers, and the procedure for calculating them is stepped through above).

 In brief, you:
1) Determine the position of the text. (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Drop the Ten-thousands digit, if there is one. (SAMPLE: 2335)
4) Break the number into two pairs.  (SAMPLE: 23 35)
5) Swap the number pairs. (SAMPLE: 35 23)
Voila, you have your value.

 SetOff X000
By far the most common type of pointers on the NES, SetOff X000 pointers are pointers whose position within the ROM is modified for whatever reason.  This amount of modification is indicated by the X000 value.  Why they're modified isn't important, only how.  To calculate them, you follow the same procedure as STANDARD HEADER pointers with one additional step.  (IMPORTANT NOTE: You can only calculate X by locating the pointer table, which is described below)  If you know X000, they are calculated by:
1) Determine the position of the text.  (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Add X000 to your value. (SAMPLE: if you found X000 to be 3000,
then 12335 + 3000 = 15335)
4) Drop the Ten-thousands digit, if there is one.  (SAMPLE: 5335)
5) Break the number into two pairs. (SAMPLE: 53 35)
6) Swap the number pairs. (SAMPLE: 53 35)
Voila, you have your pointer value.
If you do not know X000, you need to follow the steps outlined below in LOCATING THE
POINTER TABLES.  When you get there, read through that section below, and then come back
up here and re-read this one to gain the best understanding of SetOff X000 pointers.

 Some notes about SetOff X000:
1) Standard Header pointers are just SetOff 0000 pointers, but usually it's just easier to think of them as their own type.
2) For whatever reason, X seems to be 8 more frequently than most other values.
3) Although it is theoretically possible to have SetOff 1500 pointers, or some other value which is not a multiple of 1000, I have never encountered them.

 SEQUENTIAL TEXT
Technically, this is not even a pointer system, but I include it here because being able to recognize this when it's used can save you hours of headache and searching.  Sequential text systems don't use pointers, making their widths adjustable without any problem.  Customizing a YES/NO prompt would actually be as simple as changing:
Yes#No#texttexttexttext...
into:
Sure.#No way!#texttextt...
This way of doing things isn’t used very often, but some games do use it.
One important note: With this system, the game knows it's done when it reads enough strings. Therefore, you cannot simply remove a string you don't like without messing things up. If one string is slightly longer, the string to the right of it will have to be slightly shorter.
 FIXED-LENGTH SLOTS
Rarely used for text, this system is frequently used for item and spell listings.  The game knows that the text string is a certain amount of characters long, and reads them accordingly. You don't really need to do anything with these, and you can’t re-map them.

 SEQUENTIAL POINTERS
If you see these, run. I've only encountered them once, in Wily & Light no RockBoard, a Mega Man monopoly spinoff, and I'm still trying to figure out how on Earth they know everything they're supposed to.  Anyway, I'm including them here for completeness, but you're free to skip ahead.

 Sequential pointers are stored in their simplest form, two digits (as opposed to the 4 used in SetOff X000 and Standard Header pointers).  They're calculated as follows:
1) Determine the position of the text.  (SAMPLE: 12345)
2) Subtract 10 for the header. (SAMPLE: 12335)
3) Drop all digits except for your ones and tens places.  (SAMPLE: 35)
Voila, you have your pointer value.
Seems easy, doesn't it?  Well, it's not, because as you try to remap these pointers, all kinds of weird things can happen without the hundreds and thousands place.  Space constraints are worse here than anywhere else.  Trust me, you really don't want to deal with these unless you absolutely have to.

 Well, there you have your standard pointer systems.  Are there others?  Probably.  I've only encountered one other system on the NES, and I'm STILL trying to figure that one out...

 So now do we finally get to learn about how to find the pointer table?
Yes, you deserve a reward for having read this far! Let’s pull all of this together.

 FINDING POINTER TABLES and your pointer system (the easy way):
(if you need an example, refer back up to the Final Fantasy one above)
Does the ROM use a SEQUENTIAL TEXT or FIXED WIDTH systems?
1) To determine if you've got a FIXED WIDTH system, just look at your text strings.  If you've got a bunch of strings that are exactly the same length (especially if they use 00s, FFs, or some kind of <NULL> code at the end to guarantee that), odds are you've got a fixed width system at work.
2) To test and make sure, locate a string you can easily get the game to display.  Overwrite it with something longer which runs into the next string in your hex editor.  Save, then load the ROM and bring up that item.  If it cuts off after a certain length, you've got a fixed width system.
3) To test if you've got SEQUENTIAL TEXT, find some dialog in the game and locate in in your hex editor.  Now, go into the string BEFORE it and add an <END> code right in the middle of it.
Save, and load up your ROM again.  Bring up the dialog you used before.  If it's different, then you've got a SEQUENTIAL TEXT system.

 If your ROM does not use the SEQUENTIAL TEXT or FIXED WIDTH systems:
1) Open the ROM in your hex editor.
2) Locate a string of dialog which you can bring up in the game easily or quickly. (Don't use menu options/other types of text, as these tend to be stored differently than main text)
3) Write down your offset.
4) Subtract 10 for the header.
5) Note down the last two digits of your value.  (If your adjusted position was 12345, the 45)
6) In your hex editor, Hex Search UP for this value. (the 45)
7) Usually, you'll find one close to where you started your search.  Regardless, write down the two digit number to the right of the one you just found (Let's say, searching for 45, your search returned the 45 in the following line:  AB 45 FE. Write down the FE)
8) Check the second digit in the number you just wrote down (In FE it'd be the E).  Compare that value with the hundreds place in your adjusted position (In 12345 it'd be the 3).  If they match, you may have found the
beginning of your pointer table.  In they don't, continue searching UP.
9) Since the values didn't match, we continue searching upwards until they do.  (Finally, we find it a line that includes 82 45 83)
10) Now that we've found a potential pointer, we need to test it.  Change the pointer so that the text will display differently (my favorite is to just set the ones place ahead by 1, so that the first letter doesn't display.  When that happens, you know it worked.)  Then you'll need to remap the pointer to another string of text, and see if it changed like you expected it to (What we did in the Final Fantasy example above.)
11) If the text changed as you expected it to, CONGRATULATIONS, YOU'VE JUST
LOCATED THE POINTER TABLE.
12) To figure out which pointer system you're dealing with, you only need to look at the pointer you've found and compare it to the position of the text you're using.  Take the first digit of the right number in your pointer (45 83 -> Take the 8) and compare this to the thousands digit of your position (12345 -> The 2).  If they're the same, you're dealing with STANDARD HEADER pointers, otherwise you've got SetOff X000 pointers (where X is the number you just examined from the pointer -8- less the number you just took from the position -2-, or 6 => Therefore, you're dealing with SetOff 6000 pointers)

 That's it.  This system will work for most of the games that many of you will want to hack out there.  Now that you know the pointer system, you can calculate the pointer for the first item in your text block.  Find that, and you've found the beginning of your pointer table.


 FINDING THE POINTER TABLES (a harder way):
If the above system didn't work for you, make sure you don't have any control codes (described below).  If you're sure that there aren't any in there, try this system:
1) Test for SEQUENTIAL TEXT and FIXED WIDTH systems, as outlined above.
2) Locate a string you can bring up easily in the game.
3) Using its position, calculate the pointer using the STANDARD HEADER system.
4) Search the hex for this value. Change every occurrence slightly (remember, change the second digit of the first number pair for minor changes).
5) Save, and load the ROM.  If the change you wanted occurred, then you know it's a STANDARD HEADER system.  Change matches one by one until you find the right one.
6) If that doesn't work, then calculate the pointer using the SetOff 8000 system.  Search for this value in the ROM, changing every occurrence slightly.
7) Save, and load the ROM.  If the changes you wanted occurred, then you know it's a SetOff 8000 system.  Change matches one by one until you find the right one.

 If these don't work, then I'm afraid you'll have to resort to:

 BRUTE FORCE POINTER LOCATION (the tedious way):
The system of last resort.  This will work 99.9% of the time if you know that you're not dealing with a FIXED WIDTH or SEQUENTIAL TEXT system, and that there are no control codes throwing you off (and in that 0.1% of the time, you don't want to deal with it anyway).
1) Locate a dialog string you can bring up easily in the game.
2) Locate that string in the hex editor.
3) Calculate the pointer for it using the SEQUENTIAL POINTER system (no, don't worry. Using this method does not automatically mean you've got sequential pointers).
4) From the beginning of the ROM, Hex Search through for that value.  One by one, change that value slightly.  Saving and testing to see each time if you've found the right one.
5) EVENTUALLY you'll come across it (hey, if you're lucky, the Hex Search will only turn up 500 or so occurrences).  FROM there you can figure out the pointer system as described under the easy way above.

 ...or you could go on datacrystal.romhacking.net and see if somebody else has found it and written it up.

 So, is that it?  Do I now know everything I need to about pointers?
Almost. There are only two more minor things for me to address.

 Empty Space in the ROM:
Fitting all of the text you need to into the ROM can be quite troublesome.  Fortunately, there is a solution...sometimes.  On some occasions there will be blank space within the ROM which you can use to fit some extra text in.  You can recognize this space because it's usually just a large block of 00s or FFs.  Look at the end of your ROM and you'll probably see a good example of it.
Now, because NES pointers are limited to four digits, so too are your pointers.  Therefore, your range will be limited to whatever bank the existing text is in.

 Control codes:
These are codes that programmers put in with the text which tell the NES how to display it.  Much like the font tables themselves, these vary from game to game.  End of Line, End of Page, and Pause are two common examples.
I bring these up because if the text string you're using has one (or more) of these at its beginning, the pointer will point to THAT instead of the first actual letter of the message.

 COULD IT BE?  YOU ARE FINISH!!

SOURCE: chrome-extension://gbkeegbaiigmenfmjfclcdgdpimamgkj/views/app.html

19
Make a save state and then open that save state in a hex editor (I recommend CrystalTile2: http://www.romhacking.net/download/utilities/818/); once you have it open I want you to try and see if the english names of your party members appear _anywhere_ on the far-right side of the editor: i.e press CONTROL + F and enter any of your characters names and if we're lucky then they will appear.

I'm going to assume they do, because I have never edited this particular game so I don't know if they do or not; once you find one of your characters you are going to look hard at each of the hexadecimal values on the middle-portion of the editor. Under each name all of that characters data will be represented except it won't look recognizable. The fastest way to find the two-number value (example: 01, AF, etc) that represents a character level is to find one of your characters who is (EXAMPLE) at level 5, and then make sure you have at least 2 or 3 more characters ALSO AT LEVEL 5. Why? Because you're going to eye-ball all of the hexadecimal values, i.e. the "pairs" of numbers in the middle of the editor and you will try to find which hex value is REPEATED ON ALL THREE of the imaginary characters I'm describing in this example.

It might look something like this:



May 02, 2016, 12:06:33 pm - (Auto Merged - Double Posts are not allowed before 7 days.)


That screenshot is from the save file of the game Elminage: Gothic, an RPG, and I have it open in CrystalTile2. As you can see the file begins immediately with my party members, and below their names, i.e. below "aweigh", is the entirety of that characters makeup: level, age, gold currently in hand, race, class, LEVEL, skills, etc, etc. Some examples...

On the hexadecimal portion of the editor try to eye-ball/find the value "0A" in the offset address (the numbers on the FAR LEFT) 000000A0. You'll find it in the hexadecimal column "0C". It is not surrounded to the left/right by anything else other than 00's, so you should find it easily. That value, 0A, is the character aweigh's Defense number. On another chracter (if you were to keep scrolling down) it might be say 0A but whatever their Defense hex-value is it will be in the same place, generally speaking. It might differ somewhat in Tactics Ogre of course, but the principle is the same.

The only way to find out what value means what when you don't know and there is no online tutorial for your specific game (as in your case, and as it was with me with Elminage) is to pick some values that look "likely" and change them. For example I recommend picking lone values, or at least in the case of Elminage the ones off by themselves usually represent stuff like what Skill the character uses, or what level he is, or what CLASS he is, etc. The long rows of many hex values usually represent other stuff. Think about it this way: how many values would a programmer need to represent.... Level 5? Exactly.

Now you go and change them, load up the game, and see if something changed. Eventually you'll find the value (or values) you want.

Sometimes there are some things that can be self-evident. I want you to eyeball that image again and spot where there is a long row of 03 03 03 03 03 03. Saw it? Ok, that represents the amount of spells the aweigh character has on six spell schools in the game. He has a max 3 spells on schools 1-6. I'm not saying it's hard to miss a detail like that but, I want you to keep in mind that sometimes the hex value is represented "naturally" and is self-evident. I.e. if you are looking for level 10 it just might very well be a 10. Who knows!

Post again and take screenshots if you need any more help with this.

20
The trouble comes in that you are slowly reinventing either programming or something already done by more dedicated tools.
You spoke of junking work because you had a mistake caused at an early stage, that should never happen unless you made a fundamental error in figuring out the file format.

Correct! I have never programmed or edited anything or even knew that you could "open an .exe" before a few months ago. No one is teaching my anything and the bits I have learned have been through trial and error and by reading guides, FAQs and forums. For example today I learned  the difference between sequential text and fixed length text strings.

Yesterday I learned the "hard way" that by spacing out (0x20) some strings I didn't like that it would cause the game to crash. Allow me to explain: in the three Wizardry Empire games (japanese) the developr 3 to 5 %s (operators? commands? strings? I do not even know what they are called. I used to think they represented percentages, and that %d represented a dice roll!)-- 3 to 5 of these:

%s"JapaneseCharacterMeaningITorTHE"%, and the others are usually words like "BODY" (for when a character uses the HIDE command); and the like. However the problem is that since they always get called during a lot of actions sometimes they won't "match" the translation of the ACTION itself. I was seeing stuff like "aweigh THE received damage!". See the problem? So when I translated Wizardry Empire 1 I "solved" this problem by replacing each of those variables with %s"8145"% which is a cute little square. So now during that example it would read: "aweigh * received damage!". It is the best possible solution I can think of because you can't name that variable to "awiegh HAS..." because it is also utilized by 20 other actions that won't FIT NATURALLY with the word HAS. Using a simple square looked nice and problem solved.

Yesterday while translating Wizardry Empire 3 (Three) I did the same thing with its battle strings but 8145 caused the game to crash. Now I need to find something else to use there. My whole point with this example is to let you know that you are correct: I have no idea what I'm doing. I absolutely learning this as I go and am getting better every day. I did not even know about "invisible" characters until I spent hours one day, weeks ago, translating via hex editor the NPC dialog of a character in Wizardry Empire 1 and when I went to go check it out in-game all of his dialog boxes were empty. It turned out the period at the end of his sentences, 8142 which is the FW kana period, was a substring (is that the correct term?) that when erased affected the hex value immediately following, 5C, the slash sign, which then in turn affected the lower-case "n". Boy was there egg on my face.

I was SHOCKED that there could be invisible characters! That there was stuff I wasn't seeing! Instead of being dismayed I kept learning.

"engender curiosity for hexadecimal"
It is a numbering system useful for computers, or at least humans working in them. Other than maybe a quick extra for floating point/decimal points and signed numbers and there is really not a lot more that needs to be said or done. If you meant hex as shorthand for abstraction and data representation then maybe, I still prefer to teach it as a separate concept.

I think it's incredibly exciting and fun to learn. A few days ago I spent a whole day building several tables just for fun and for practice. I love reading websites that detail what controlling characters are, what line-breaks are, hell I've even spent hours reading the history of the evolution of the encoding systems. It allowed me to translate Wizardry Empire 1, and it is allowing me to translate 3 other games; it also allows me to edit save states and it also allows me to the OPPORTUNITY to learn. I have never been interested in any programming language but this has become a gateway and now I plan to learn many others.

Did I mention that I used to think %d represented a to-hit dice roll? :)

A lack of flat out explicit answers being a problem? It is not that folks here take some perverse delight in sitting there thinking "haha I know the exact answer to the question they seek, however I shall not answer, not even reply, and they will never know".
I should also say that your chosen game is for the PSP, not something a lot of people around here have the most experience in and though things do cross console barriers (anybody here probably can learn it, give or take some of the stuff involved with 3d coordinates and maths possibly being a bit new if you have only ever really done 2d and isometric) it still involves learning a new system, using the sub par debuggers that it has (PPSSPP is doing good stuff but it is somewhat recent and still not a patch on fceux, no$gba or PC debugging), downloading in this case you got lucky and it is only 130 meg iso (though that is still not a 4 meg SNES game and most that know the PSP see things routinely hover around the 1 gig mark) and then spending a few hours to figure out which of the dozen or so ways of doing that particular thing that a dev did.

Working PSP games is an absolute nightmare. Wizardry Empire 1 I did the Windows PC version and it was paradise compared to what I'm going through working on Elminage 2, 3 and on Wizardry Empire 3 for PSP. Also: I fucking love it. Know why?

I didn't even know what debugging was until 2 weeks ago. Sure I knew what it was in a general sense, and in the literary definition of the word; but I didn't know absolutely ANYTHING WHAT-SO-EVER about things like: viewing hexadecimal data in memory; inspecting and "fixing" ASM functions/code, and of course I had never before extracted textures from a game. Because of all the obstacles the PSP games presented me... starting with the fact that the .BIN files are impossible to extract shit from, that is of course until I FORCED MYSELF TO LEARN HOW TO DO IT, and hell also the fact that I didn't even know that games came encrypted!

I spent at least a good 3-4 days trying to "troubleshoot" why my game-edits were not showing up in-game... imagine my surprise when I learn that I have to do this thing called decrypting the EBOOT.BIN. Oh, and you know what else? Thanks to that experience I was also forced to learn about headers! I didn't know what headers were, or what headers are used for even until I had to decrpy some .BIN files. Are you starting to understand my point about how jumping in blindly and with enough enthusiasm and the right attitude you can always keep learning and improving?

Now I am using programs like Cheat Engine to locate strings, functions, calls; to view the hexadecimal data in memory; etc. I LOOOOOVE doing stuff like that. And again, it was only due to the unbelievably problematic nature of PSP game hacking that I had to learn how to trace and how to dump and what the fuck a freaking break-point even is. I distinctly remember a GBATemp poster telling me to "set a breakpoint and trace where the text is being pulled from". Now, do you think that helped me? It did, but not in the way he intended. It helped me in the sense that I thought: Well since absolutely nobody wants to explain anything I'll have to do it myself. And there is merit to this!

But there is also merit in sharing as much as humanly possible _just because you CAN_. Not everyone is as enthusiastic as me.

And besides: how else am I supposed to learn ANY of this stuff? The guides on any site, be it here or on GBATemp are either flat-out subpar or they only cover stuff detailing NES/SNES games. THAT IS COMPLETELY USELESS TO ME. PSP games do not deal with lo-rom and hi-rom stuff. I sure do wish there was a guide on PC GAME headers or what is the best way to create an entirely new font width table and insert it into a PC or a PSP game! Unfortuantely I havne't found any detailed ones so far; but I devote some hours every day to learning more until eventually I'll know what to do and how to do it.

Guess what? Today I actually expanded the Wizardry Empire 3 EBOOT.BIN successfully and added 3000 bytes. I did this because I know I'm going to need space for when I insert the ASM code that changes the game's font to 8 pixels. Do I know ASM? No. Will I know what I need to know to do what I want in a few weeks? I guarantee it. Now to be fair this is the very first time one of the many "tutorials" I read actually detailed everything perfectly concerning what expanding was and how to handle the headers and whether to add new spaece or to extend, etc. Everything I just mentioned... 100% new to me, learned it today, loved doing it, and I feel awesome that it is done using a hex editor!

Your one thread with the slightly dramatic title however can be answered quickly from memory by a large portion of the active forum users without so much as doing a single search.
Anyway back on the PSP then if I lack the desire to do the things in the list then I get to instead cover font representation as a whole and hope something is applicable. I do this in various replies from time to time, probably why I have a bit of a reputation for long winded replies, where I probably should have just said that will likely take you learning assembly so good luck with that. You might also be reading the wrong thing into such a reply -- most of the time I would read it as something like "I have this problem, is there are quick or existing way of sorting it or is it going to be a long winded affair?" and "yes mate, asm is in your future if continue down this path" is the way of saying yeah it is.

I completely agree mate and I love long posts too. I mainly post at the rpgcodex and I treat forums as verite communque: you should write about your stuff as detailed as humanly possible and provide even perhaps what some would call unnecessary extrapolation. A long post that contains actual communication is rare and should be prized.

I detest people who say "lol tldr".

http://s7.photobucket.com/user/aweigh01/library/

Feel free to peruse my photos as they are all specifically about the Wizardry games (and others) I've been working on. If you go back to the very first photos you can look at the in progression and see the actual "learning" thing I've mentioned a few times. Also I feel very proud :)

The entire reason I even started hex editing files is because I am probably the biggest Wizardry fanatic you'll ever meet. And when I ran out of Western-developed Wizardry games and got tired of pining hopelessly for the JP Wizardry games to get localized I simply decided that I would translate them myself. Why? Because I want to play the games myself. For literally no other reason. Although I will release I could not possibly care less who else plays whatever I work on. I'm doing this because these are games I want to play in english myself and enjoy them. If it means learning 10 different things that will probably have me screwing up over and over until I "get it right" then great, and if it so happens I've found out I massively enjoy editing hexes then don't you think that's a good thing and something worth being enthusiastic about? I do.

I update the thread ( http://www.rpgcodex.net/forums/index.php?threads/all-in-1-thread-about-wizardry-clone-translations-right-here-new-info-been-working-on-wiz-xth-2.108462/ daily even though almost no one replies because not only does it serve as a way to arouse discussion in that forum (it's an RPG forum but there is a surprising amonut of dedicated rom hackers there who post regularly, and it was one of them who got me started) and to be honest: that thread made in a non-rom hacking website has yielded a lot more education for me concerning hacking, debugging, extracting files and graphics editing and all that good stuff than any romhacking forum has so far.

One person even explained to me step by step how to find the value that dictated the FW_FONT's "FW_WIDTH" value when I was doing the grunt work on Wizardry Empire 1: he went step by step and in the end I learned that surprisingly enough, the Microsoft website has a ton of information that is very useful for rom hacking! Needless to say I immediately changed that game's default FW_WIDTH value from its default "500" (8403 hex value) to a cool, slim and skinny and beautiful looking 2C01.

I had no idea how to "write" the number 300 for the value I wanted instead of 500 so... once again, guess what? I had to learn via googling and look up hexadecimal FAQs/websites until I learned alllll about a lot of stuff that went way beyond how to write 100 in hex. One problem once again leads to completely new avenues and they are ALL very interesting!

Do you honestly think that would have happened to me by making a post about the font being too bulky in a hacking site? Be honest with yourself... you know the answer is no.

Pages: [1] 2