Editing graphics in DAT format (Falcom Related)

[apraxiumRum]

Another topic by me on my quest to translate Dragon Slayer: New Heroes Legend for PC Windows, will I ever organize my questions??
So...The game shows promises so far. The texts is sorta separated from the program and the coding, and it was categorized so I can mess around without destroying too much.
But I just keep wanting more to improve and do my best. And now I've hit a new hurdle: Translating texts in graphics. Namely the Party's HUD:

I noticed that nothing in the game executable file changes their Japanese name in the HUD, but then I saw screencaps of the game..in Chinese??


So I figure that there are image files for the HUD, that were changed through some means to make the names appears in the HUD in Chinese.
What I want to know:
1- Is there a way to extract images such as (Interlude cutscenes, monsters sprites, characters sprites) from the data files? I tried falcnvrt but it doesn't support this game.
2- Anyone knows a way to edit said images? Everything from the conversations to monster battle messages is in dat format. Yet I can edit the Japanese characters in these files in a HexEditor no problem. I think I need to sorta convert the image files into actual "images", edit them, then convert them back to DAT files.
I tried searching the subject but I have no idea, not to mention that I have zero knowledge on decoding a file of a game so I am asking for help.
I await your constructive inputs. 

[Her-Saki]

Usually PC games store their images in DAT-like files and read from there. In order to edit that images, yeah, you need to "extract" the images, then edit and bring it back to dat format. This is the case with games like visual novels; in this case there's a need for reverse engineering tools that detect what type of compression engine uses (most on VN goes on engines), and decompress it properly. Sometimes is compression and other is encryption, but is not that usual. One case is huffman encoding that saves about 40% of space, there's a couple of programs that can decompress files associated with it, then recompress it. From this way you have two options: find proper software what makes all for you, or create it. The latter requires you to study file from a debugger (Ollydbg for PC), and try to figure out, for example, what kind of compression game uses, what image's format, palettes, sizes, images that cannot be edited/extracted with your actual method, etc, then apply correct algorithms. For recompression stuff, how that kind of compression is made in the language you're using for your own software, check if you must somewhat modify instructions on the engine...
TL;DR: you must use other people's reverse engineering software, or made it with your hands, or ask others to do it for you. I saw that on a VN forum once. Otherwise you cannot edit images, unless game is from 1995 Virtua Fighter PC-esque (hell, I'm surprised this game didn't give up providing source code, having all that free images for edit).

[apraxiumRum]

I stumbled upon this while I was messing around the data/mons file with HxD editor. Now that I am looking at it, I think the issue isn't about decryption. Look at that!

Apparently the picture files are coordinates for colors. I think it closes a bit on the problem, but I still need some sort of tool to make the picture materializes as an image file I can view.

[Jorpho]

Have you tried just opening the files in Tile Molester, and see if anything appears in any of the image modes?

[apraxiumRum]

Tried Tile Molester, but it didn't work. Couldn't make a head or a tail from what I saw.
However I managed to make it despite odds, and a heavy strain of headaches to me.
See for yourselves:

Though I might reconsider editing some more, I don't think that I'll redo the ending screen. It just....was more trouble than its worth.

[weissvulf]

Looks nice!

Do you have a small data sample of an actual text graphic? The 'silhouette' you posted earlier seems more likely to be a mask than the actual image data.

Compression aside, there are only a few common ways to encode images. I made a tool for my own use that will extract/insert image data from a few common raw formats, but I have no idea if it would work for your needs without seeing actual image data.

[apraxiumRum]

Here is the original file that have the party HUD:
Link

[weissvulf]

The image is in 8 BPP (aka 256 color) format. My tool will extract/insert it, but the color palette for the image is missing so the colors will be off. There is a bit of data at the end of the file which may be a palette of some sort, but the length doesn't match what would be expected for a 256 color image. Example:

OBSOLETE IMAGE REMOVED

You probably already know the following, but I'll mention just in case.

Spoiler

An 8 bits per pixel image uses 8 bits (aka 1 byte) of data to define a single pixel. That gives each pixel 256 different possible colors (aka 0-255). In the case of such images, each byte refers to an entry in the color palette. For example, the byte 00 references 'palette entry 1', byte 01 references 'palette entry 2' and so on.

The palettes for such images usually store the color values as 3-byte per color (aka 24bit) where the first byte sets the Red value, second sets the Green value and third sets the Blue value. So a palette entry of 000000 would be black since the RGB values all have an intensity of 0. FFFFFF would be white, FF0000 would be pure red and so on.

[close]

My tool will extract this image to a TIM format (Playstation) and generate a generic color palette. You can then use a tool like TimViewer (available here) to convert to BMP, or the TIM plugin for Photoshop (available here) to open the TIM directly in a supported photo editor.

Edit the image and palette as you like in the photo editor, but my tool only re-inserts the image, so any palette changes would only be to make the image easier to edit. Once you're done, just save/convert back to 8-BPP TIM, and my tool will insert it.

NANA (available here) works well as a reference for my tool to determine 'start location' and 'width'. It scans images in various modes 'on the fly'. Just drag the file onto nana.exe. Use the < and > buttons to change image width, 1-6 to set the BPP mode, and left-right arrow keys to slide the image sideways. Nana's 'raw:byte' mode (6 on the keyboard) is 8-BPP.

HERE is my tool. Included is the TIMproject file (just a renamed txt) which has the settings for the file you posted. It's a work in progress and may have some bugs. Any feedback would be appreciated. 
November 25, 2017, 10:05:14 PM - (Auto Merged - Double Posts are not allowed before 7 days.)
I just looked at the image data in a hex editor. While it's technically encoded as 8bpp, it only uses 16 colors (all pixels are in the 00-0F range). The 48 bytes of non-image data at the end of the DAT is indeed a 16 color palette (16 colors x 3 bytes per color = 48 bytes total).

Each entry is standard, but their order is scrambled. In other words, black in the image data is 00, but that is the third entry in the palette (01 01 01). Normally 00 would refer to the first palette entry. Still, you should be able to edit these palettes if needed in your translation.

This image shows the palette data in HxD set to 3 wide rows. Red labels show the expected order. Also, here's 8bpp BMP copy of the image from the DAT file with the 16 colors of the palette edited to match your earlier screenshots. The image data is completely unchanged.

[apraxiumRum]

I...I don't even know what to say. 
Just to be clear, everything you said about the whole color palette and your TIM tool is pretty much foreign to me. I never got that deep in the whole image file editing with HEX editor. I don't even know what do I do.
The procedure I used relied heavily on copying and matching hex values with addresses and sticking to 3 colors for the text. 
I am completely lost. Impressed still, but lost.
Would you please teach me how did you make that BMP image from scratch? I could benefit greatly in my work on the game's files If I can see what they look in the game.

[weissvulf]

Ugh! I ran into complications. My tool extracts the image into TIM format which I open directly in Paint Shop Pro using the plugin HERE?. I tried the free TIM converter I know of and none of them seem to work as needed.

TIMviewer doesn't seem to convert odd sized TIM files well. Tim2View, seems to totally alter the image and palette when converting a PNG back to a TIM. The TIM plugin FOR GIMP won't work on my system.

In other words, I don't know a good option for converting from TIM to BMP other than the Photoshop plugin I'm using. Do you happen to have a photo editor that can use that plugin? It's possible someone here knows how to get the GIMP PLUGIN working instead.

I'm assuming there are only a few text-graphics in your game. I can convert some for you if needed.

I put together a step by step tutorial showing the steps I use, but it relies on the PSPro editor so i don't know if it will be much help. It will tell you how to extract the images as TIMs if you want me to convert them for you. It's in RTF (rich text format) here.

[weissvulf]

In case you are still working on this project, I noticed that the version of nana on RHDN was an old one that didn't play nice with modern systems. Neil's site seems to be gone now, but I submitted a newer version of his tool that should work. You might check the nana listing again in a few days to see if the update was approved.

[apraxiumRum]

Didn't get to reply for too long because I am busy with the project. 
I tried nana viewer, probably the old version, and I gotta say, it rocks!:




If I find some free time; maybe I'll try the method you described.
Thanks!
December 13, 2017, 08:14:19 AM - (Auto Merged - Double Posts are not allowed before 7 days.)
Could you please reupload Your tool: pS1mple Extractor again?
It was deleted from Dropbox.