News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: "Never use hex-editors for translating a game". Isn't hex-editing the only way?  (Read 7539 times)

DougRPG

  • Full Member
  • ***
  • Posts: 144
    • View Profile
You can use hex editor to translate games with few texts, and it's usually done by everyone starting in romhacking.
For any serious translation (like Rpgs) you need to create your own tools.
Nobody translate a game with, say, more than 100 dialogs, using an hex editor. There are some reasons for that:

1-) You cannot replace the original string with a bigger string, otherwise you will overwrite the next dialog or game data. In these cases you'll need to manually change the pointers, getting the new addresses yourself.
With a custom tool you extract the dialogs to a nice format, and after that your tool reinserts the text and do all the grunt work automatically, so you can translate the strings to any size without worries. If the final block doesn't fit in the original place just expand the Rom and put the text block in the end (not always works of course). Each game is different, but with custom tools you can always do more stuff.

2-) It's absurdly boring and time comsuming. Romhacking is not this masochistic act. Romhacking is a fun exercise to apply your knowledge. If you have the intention to translate a big game with an hex editor, go do something better with your time. The time you will waste translating a big game with an hex editor you can do a programming course to learn how to program your own tools, and probably will take less time. If with state of art techniques it takes months to translate most of translations, with a single hex editor it will take decades.

3-) It's absurdly error prone. After changing a few pointers and some dialogs you'll find a lot of errors. It's almost impossible to change a single dialog without an error. You'll have to check each dialog as you progress, and you'll be crazy very soon.
With custom tools this is exponentially less error prone.

4-) It only works for games with no compression. Most Rpgs and bigger games has some type of compression.

5-) Even if you finish the translation, the quality will be very, very poor, at least compared to using better techniques. Doing something poor that noone will play is waste of time in my opinion. You can do good a translation with an hex editor only if you are translating a game with few texts, or if you have a huge amount of patience and free time. You can probably mention several games translated only with hex editors, but probably these games should have a much better quality if they had used custom tools.

But translating using an hex editor is the natural course of the romhacking learning curve. Probably everyone starts changing stuff with an hex editor. I'm only saying that because you wrote like absurd things in your first messages.
Hex editor is probably one of the most used tools in romhacking, but I only use it for visualize data, and I never change a single byte using an hex editor. For that I use patchers where I can generate the final Rom with a single command, always using the original Rom.

If you want to translate a game but don't want to learn programming, then try to find someone to do the romhacking for you and focus only on the text translation. It's difficult to find a good translator.
If you want to learn programming, then you should start studing programming and creating your own tools. Hex editor is only the beginning.

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
I've been spending the past week and a half or so simply studying/reading all I can about the different character codifications until i can say i'm pretty much well-versed now in the stuff... I finally, *FINALLY* fully understand what the following things are:

- pointers
- pointer tables
- control characters
- what a font glyph set is (font.bmp)
- how the font glyphs are encoded with positional x/y/d values (give or take one thing or another; hardware/formats differ)
- the difference between variable width, half-width, full-width and monospacing.
- what a texture vertex is, what it does and what you can do with it|
- some very, VERY small knowledge of the most basic opcodes for both x86 and the PSP MIPS architecture

...aaaaand I finally get what most of you guys were telling me about knowing what tools are needed and the importance of either acquiring them or modifying existing tools for your own needs; and the fact that the hex-editor is only the ENTRY into the translation attempt and it lacks the many and varied utilities and/or functions that one will need throughout the course of a translation attempt.

right now the only tool i've been using for the last week is simply IDA program. I swear guys, if IDA's built-in hex editor supported shift-jis I would *ONLY* use IDA, except for when I need to compare files or do something graphics related or simply need a robust hex editor like 010 to do some weird shit.

right now i'm knee deep in poring over  the wizardry empire 3 eboot.bin for PSP; i've been using the past few days to just stop trying to translate the game and instead spend some time per day reading about IDA usage and MIPS opcodes and the like... I realize that unlike wizardry empire 1 (windows PC) I'm 100% going to need to implement some font hacks into Wiz-Emp-3 or the translation will be very, very poor.

At the moment I'm going over trace dumps of the game to see if I'm correct in where I think the value that controls the amount of pixels that are inserted by the game between each monospaced full-width letter-segment (the game uses canned text output); once I've found it and successfulyl changed it I then want to eliminate the kanji sets and make a pointer table / display list / character-width table / etc, of my own and see if i can put in some variable width letter drawing.

I do believe that will require to also hack the game's routine for prim triangles and i have to google how to change them into prim rectangles.



anyway what i also wish i can learn to accomplish is to write a tool to extract all of the text from the game. I've yet to actually google this subject in any depth as reading the ASM/encoding stuff has had me pretty floored.

Elminage 2 for PSP is the 2nd game I want to translate and attempt to implement a VWF hack... on the plus side I have Elminage 1 (one) for PSP and that game was officially localized into ENGLISH language so I have been spending a few hours (heh) poring over very carefully the way the Elminage 1 publisher/devs hacked in their VWF routines, opcodes, lists, tables, etc; EVERYTHING.

Just studying how they did it on Elminage 1 and then contrasting/comparing with the fixed-width monospaced font(s) in elminage 2 has been very illuminating.




it's all still very much a mystery really but i can sort of understand what it is i'm staring at, which is more than I can say for two weeks ago.

------------------

relevant links for anyone who might find them useful, just as i did:

This forum post from one of the persons involved in the translation project of one of the lost heroes games on PSP went and completely explained and detailed what prim triangles, rectangles, vertex addresses and vertices are and they are used by the psp hardware and how you can utilize this knowledge for your own needs; the post author also goes into detail explaining how letter drawing is done on the psp (and in general, of course, but specifically on PSP):
- http://www.heroesoflegend.org/forums/viewtopic.php?t=197

This forum post didn't really illuminate *much* BUT it did provide one unbelievably helpful step-by-step tutorial on expanding or extending a PSP game's "space" for the later introduction of your own code:
- https://gbatemp.net/threads/psp-asm-hacking-for-variable-width-font.374967/

This forum post about a guy asking for help on how to use IDA to pin down a specific function is... very, very good. The amount of detail the person who replied to him went into should be the same gold standard every single perosn should aspire to when replying to any question, from anyone, at any time. Always. No excuses!
- http://stackoverflow.com/questions/931257/how-can-i-find-the-data-structure-that-represents-mine-layout-of-minesweeper-in

This link is to part of the blog of the mother 3 translation team; in this entry the author goes into detail on the basics and the core concepts, or rather, the fundamentals, of just what exactly is going on when a letter is drawn from its glyph and onto the game and why/how half/full widths are a thing and how to work around such things:
- http://mother3.fobby.net/blog/2008/03/29/welcome-to-text-welding-102/

-------------------

Now... much less useful but still these are posts/links that i found helpful. This following link is for someone's personal wiki page where he archived how he achieved a VWF hack for the psp release version of persona 2:
- http://bbq.xyz.is/Persona_2:_Batsu#Make_font_variable-width
« Last Edit: May 11, 2016, 12:22:21 am by aweigh »

BlackDog61

  • Hero Member
  • *****
  • Posts: 784
    • View Profile
    • Super Robot Wars A Portable translation thread
I've been spending the past week and a half or so simply studying/reading all I can about the different character codifications until i can say i'm pretty much well-versed now in the stuff... I finally, *FINALLY* fully understand what the following things are:

- pointers
- pointer tables
- control characters
- what a font glyph set is (font.bmp)
- how the font glyphs are encoded with positional x/y/d values (give or take one thing or another; hardware/formats differ)
- the difference between variable width, half-width, full-width and monospacing.
- what a texture vertex is, what it does and what you can do with it|
- some very, VERY small knowledge of the most basic opcodes for both x86 and the PSP MIPS architecture
What could have been documented differently to help there? From the links you posted, I take it you were essentially happy to get PSP-specific info. Anything else?

Elminage 2 for PSP is the 2nd game I want to translate and attempt to implement a VWF hack...
Interesting! If you see that you're getting somewhere, it would be nice if you would be  ok to create a topic about it. What do you say?

This forum post from one of the persons involved in the translation project of one of the lost heroes games on PSP
Did you mean "Legend of Heroes" or really "Lost Heroes"? (I'm interested in Lost Heroes.)

aweigh

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
Well one very important link i left out is the website i used to read up on text encoding and its history all the way from the 60's up to today. I didn't include that link because I figured it was stuff "everyone knew". Basically It helped me learn what all those invisible characters I was living in fear of writing over when translating a string in fear of messing up a pointer or something of that nature. I no longer fear that scenario.

relevant link: https://www.cs.tut.fi/~jkorpela/chars.html#control

the information in the links is universal: the legend of heroes forum post explaining vertices and letter drawing and font table positioning and all that is 100% universal. the only aside really is when he goes into detail about prim triangles and rectangles: that's a PSP thing. Not the triangles/rectangles, mind you, that is also universal in drawing on-screen objects but rather in terms of that specific post.

and then in the Mother 3 blog post you will find the fundamental explanation of what the hell is a font, what is a letter, what is a bit, what is a pixel (in terms of fonts and letters), and the dude provides the most elegant explanation of half/full/variable are really just a matter of pixels.

from reading that blog post it dawned on me that VALUEx445254 is a thing. obviously those numbers are made up.

the mine sweeper link gives a greatly detailed step-by-step on using an assembly/debugger program, in this case IDA, to trace a function and then modify it.

that is the single most essential thing one needs to learn to do.

As for elminage 2 I already hacked in the battle menu / character menu texture buttons from psp Elminage 1 onto psp Elminage 2. It works flawlessly. Since I don't (yet) have image editing skills that was the fastest, and coincidentally the most ELEGANT solution possible. As always this sort of stuff is for my own personal use really... i konw a lot of people have a lot of very weird attitudes about "project quality" and would say that hacking in a prequel game's menu textures onto its sequel is something unforgivable and that the "project was of extremely poor quality"; i think that attitude is ridiculous. I plan to learn how to edit images eventually. the important thing is I have reliable scripts for extracting and inserting textures into and out of elminage 2 on psp.

i've done prelimenary translation of elminage 2 but i stopped because i realized i first need to hack into the game the variable width font (routines/opcodes/tables/lists/etc) take it from elmiange 1 psp iso and slap it onto elmiange 2 psp iso. But before THAT i first need to sped some time studying ASM.

eventually once that is done then i have to learn how to go about either making a script for extracting the text from elminage 2 scp.bin or acquiring an existing program for another game and modifying it. the game simply has too much text to translate it without a text-extraction/insertion tool...

i tend to focus on low-text rpgs like dungeon crawlers, such as the wizardry empire games. i think there's probably less than 100 dialogs in wizardry empire 1 for pc. i did them all in a hex editor. there's actually LESS dialog than that in the 3rd game for PSP; however once again i first need to hack the font otherwise there is no use translating anything, be it little or large amount.
« Last Edit: May 11, 2016, 03:05:56 am by aweigh »