[PSP] How do I unpack/pack this bfs file?

[Ajbjc]

Recently, I wanted to try and mod Flatout Head On for the PSP to add more music, at least for now, and when using UMDGen to unpack the ISO, I get a folder and file within the USRDIR folder: DATA (contains 20 .at3 files for each song and the opening video) and FLATOUT.bfs (which contains everything else with the game). I know there are bfs unpackers online but they don't really work: the one for Flatout 1 I don't exactly know how to use and doubt it would work, and bfs2pack has one or two modes (depending on whether you get it from SourceForge or GitHub) only partially unpacks the file for some reason (it gives the main data folder then a few of the folders and files I am meant to be getting).


bfs2pack is open source and written in C++, source code is made available though the GitHub version requires you to compile it yourself in case anyone wants to look at that and see what might be causing the issue, whether it's getting filenames, unpacking them, or something else. As for messing around with the files to get the unpacker to work correctly, I don't know where to start or what to even do. Though I tried downloading all the files to add some breakpoints then run the code without compiling to a .exe file just yet, however it's proving to be a lot more difficult than I thought


I don't know much about hex editing and nothing about how bfs files work, but putting FLATOUT.bfs into a hex editor gives basically an entire file filled with gibberish with the only plaintext that could be read as actual English, and not random characters, is "bfs1" at the very beginning and many instances of *.wav where * is a three-digit number, though none of the files as far as I know are .wav. The BOOT.BIN file also makes a reference to this .bfs file along with file paths to all the files within the .bfs file, and a few references to lua variables.


Replacing the existing .at3 files with new files with the same filenames just causes those files to be skipped for some reason. The game can be modded and it doesn't require editing the BOOT.BIN file (unless you were to add files to the .bfs file itself) because I came across files to mod the game by replacing the opening video, adding 27 songs and translating the entire game, which isn't great for me. As for how they did it, I have no idea but I doubt it was by hex editing as it's a 324MB file that contains track data, car data, textures, etc. Using FlexHex to compare files, I tried modifying mine by implementing changes to the hex code from the modded version into the normal version, however it's mostly crashes and I don't know which changes do what. There are chunks of hex code that have been added, either large chunks, around 2k bytes, or many small chunks in a row, and there is also chunks that have been changed with some being small and many while others being huge (sometimes into the several thousand, or even 10-20k bytes of code changed) and scattered.

[FAST6191]

Hmm. Don't often see all in one archive things for PSP games.

Anyway yeah archive formats can change between games in subtle ways. If you have a resource like source code, even partially working, then I would probably start there to see if I want to use that to figure out what goes (if it is C++ though and the devs went all basically code minimisation/obfuscation on you rather than more plaintext or no comments to note what goes then that would be cause to skip something).
Archive formats for games don't tend to get that radical. In their basic form you need something to detail the file locations or file lengths (length1+length2 = location of file 3 after all). Better formats will have names, better still with directories. You tend not to get full OS level file permissions or flags as it is a game and ROM is kind of key, though sometimes you see compression ones.
Said file location detail if it is in the same file (sometimes they are separate -- easier/quicker to grab a list of names/locations and possibly keep that in memory than it is to grab a chunk of file, figure out how large the list section is, do the maths, and then read the relevant data) will tend to be near the start (bundled with any header) but could be at the end (see footers) but I have also seen [length][file][length][file]... and segmented areas (think of it maybe as directory by directory).
If you have existing files extracted then see if you can find them within the archive, and then what might indicate a location or list of locations for a list of files. You can also then figure out what might have happened to trip up the files you are missing.

Figuring out archive formats is pretty much a fundamental skill of the ROM hacker not dealing with GBA or older or 16 bit and older consoles, and even those will be able to take to it very quickly.

Replacing causes skipped track (somewhat impressive that they did that -- most games tend to crash when "bad" data is encountered). That will depend upon how you replaced the track. Usual suspects for reasons are improper insertion (if you are using tools you noted as broken then maybe) and improper/different standards to what is expected (higher frequency, bit depth, bit rate, stereo/mono where original was not...)*, along with maybe not stripping any metadata or whatever that a not specific for the game encoder might have included (at3 is not necessarily a popular format outside Sony's game uses but it is more than some as some phones and other things went there).

*even if the system can handle it elsewhere the game might not, and might not be for space saving reasons (rare on the PSP as UMD is something of a fixed size and nobody cared about the PSP street for the purposes of downloads) as much as pushing that many polygons might not leave room to handle audio qualities for the golden ear set. Sometimes it is laziness, stupidity or cheapness though and the hacker gets to source better things.

[Ajbjc]

Anyway yeah archive formats can change between games in subtle ways. If you have a resource like source code, even partially working, then I would probably start there to see if I want to use that to figure out what goes (if it is C++ though and the devs went all basically code minimisation/obfuscation on you rather than more plaintext or no comments to note what goes then that would be cause to skip something).

There have been some comments in the code, and using the function name I know at least some code I should be looking at however I'm not sure how to get the breakpoints activated and how to run the code at the same time without compiling into an .exe file. That at least would potentially narrow down the search to a potential function or two that would need changing.

Archive formats for games don't tend to get that radical. In their basic form you need something to detail the file locations or file lengths (length1 length2 = location of file 3 after all). Better formats will have names, better still with directories. You tend not to get full OS level file permissions or flags as it is a game and ROM is kind of key, though sometimes you see compression ones.
Said file location detail if it is in the same file (sometimes they are separate -- easier/quicker to grab a list of names/locations and possibly keep that in memory than it is to grab a chunk of file, figure out how large the list section is, do the maths, and then read the relevant data) will tend to be near the start (bundled with any header) but could be at the end (see footers) but I have also seen [length][file][length][file]... and segmented areas (think of it maybe as directory by directory).
If you have existing files extracted then see if you can find them within the archive, and then what might indicate a location or list of locations for a list of files. You can also then figure out what might have happened to trip up the files you are missing.

Here's the problem, let's say I have a file with the file path "data/folder1/folder2/test.bed" and put it through bfs2pack, it will give me a .bfs file and when I put it through a hex editor, amongst all the gibberish I get, I also get the contents of the .bed file given in plaintext. However, if I were to have a data folder with multiple folders and files within it, the contents end up looking completely different. I don't know exactly why this is happening but within the bfs.cpp source code file, there is a function called getFilenameIndex which uses a lua hash function, which is also given in the same file, which is a pretty weird way of hashing, considering what I just mentioned earlier. This is assuming the packing method in this app, will be compatible with this other game. Though I might need some help compiling the files or just debugging them, it's very annoying and getting different errors usually relating to not finding files such as zlib.h or tchar.h, even though the source files don't include them, and that's without getting into compiling which I'm not sure how to do. I was told to use Make but not sure what I'm meant to be doing, I followed a guide on StackExchange to use the Makefile.win file through the Visual Studio command line but got some error about an illegal '{' character in a macro.

The BOOT.BIN at least stores file paths though that seems to just be about it, as the new .bfs file has a different file size, with who knows how many files have had their file sizes changed. The .at3 files are handled by just the .bfs file as there is no mention of them in BOOT.BIN.

Replacing causes skipped track (somewhat impressive that they did that -- most games tend to crash when "bad" data is encountered). That will depend upon how you replaced the track. Usual suspects for reasons are improper insertion (if you are using tools you noted as broken then maybe) and improper/different standards to what is expected (higher frequency, bit depth, bit rate, stereo/mono where original was not...)*, along with maybe not stripping any metadata or whatever that a not specific for the game encoder might have included (at3 is not necessarily a popular format outside Sony's game uses but it is more than some as some phones and other things went there).

*even if the system can handle it elsewhere the game might not, and might not be for space saving reasons (rare on the PSP as UMD is something of a fixed size and nobody cared about the PSP street for the purposes of downloads) as much as pushing that many polygons might not leave room to handle audio qualities for the golden ear set. Sometimes it is laziness, stupidity or cheapness though and the hacker gets to source better things.

Used .wav for the music files, as converting the original .at3 files to .wav worked unlike .mp3, and all I did was use audacity to change the audio levels as apparently the PSP plays them a bit louder than expected, then converted the files to .at3 with a 128kbps bit rate. Apart from starting with .wav it was the same method I used with another game and that worked perfectly fine, and that time the new files would've had different file sizes. Though fixing that issue will likely have to wait until I unpack this file first so I can look at the actual files to see what is going on.