PSX MDEC/bitstream trouble

[Raccoon Sam]

EDIT: SOLVED!
The answer is complex and needs a proper writeup, but I figured I'd let you all know not to potentially waste your time.
I'll update this thread sometime hopefully soon 

The problem:
London Seirei Tanteidan is a game we are translating for the Playstation 1.
It uses a bizarre method for many of its graphics. While we have pinpointed (and edited!) most of the typical graphics on the disc like your everyday 4bpp/8bpp TIM images, some backgrounds of the game are stored in a bitstream format which seems to be somewhat documented, but I still have trouble understanding how to work with it or if there are any tools for working with them.

When the first chapter of the game comes to a close, a title card is drawn over the scene with a fade-in effect. See this video in its entirety: (video link)
The image is not drawn into VRAM and glued over the framebuffers like a typical image, but instead pushed in straight away as 16x118 and 16x122 slices with the CpuToVram command.

Looking into RAM at that point, the slices indeed do exist there. They can be visualised with Tile Molester by dicking around with the width settings a bit.


Here's the graphics in isolation:


Now, usually when dealing with any kind of graphics data, compressed or not, my gut instinct is to either search for the bytes on-disc or look into the debugger and see where the data comes from,  figure out the origin, step through the routine and learn the compression. In this case, though, these small steps can't be taken because the process of the graphics transforming from their compressed state into the framebuffer is nearly instantaneous because it's a bitstream.

I have pinpointed the exact origin of the data. It is in the file called SM.. You can download a copy of the file here.

The file begins with a header listing of 538 items, 16 bytes each.
The later parts of an entry are not important here, but the first 4 bytes are, so I've converted them to little endian 64-bit form for readability's sake:
00000005 6F 01 21 01 00 05 80 03 80 80 80 00
00000096 6F 01 21 01 00 05 80 03 50 50 87 00
00000127 7D 02 21 01 60 04 00 03 80 80 80 00
000001C7 7D 02 11 01 60 04 00 03 50 50 87 00
00000257 68 01 11 01 0D 04 44 02 80 80 80 00
000002D1 68 01 11 01 0D 04 44 02 50 50 87 00
0000034B 30 01 21 01 E0 01 00 02 80 80 80 00
...
0000EBA3 68 01 00 01 60 04 00 03 80 80 80 00
0000EC0C 28 01 00 01 20 03 00 01 80 80 80 00
0000EC35 30 01 00 01 58 02 5E 01 80 80 80 00
0000EC66 10 01 00 01 80 02 E0 01 80 80 80 00


The 0000EC35 is the pointer to the intermission title's data, and the true address is the value shifted left 11 times; 0000EC35 << 11 = 761A800.
I have confirmed this is the correct address by editing the entry to something else and having the scene in-game display some other, irrelevant backgrounds instead of the title card. Below I have replaced the
35 EC 00 00 30 01 00 01 58 02 5E 01 80 80 80 00
with
66 EC 00 00 10 01 00 01 80 02 E0 01 80 80 80 00
which is the entry immediately after it. The result is as follows (video link)
It is evident that the data at 0000EC66 (or 0x7633000, to be exact) is the image used for the ending.

Going to the address 0x761A800 in the file will present us with this data:

I've highlighted the first eight bytes because they're irrelevant(?) right now. Instead, pay attention to the bytes after that:
A0 06 00 38 01 00 03 00

That is a tell-tale sign of this being a bitstream, as explained not only in the nocash docs here.

BS v1/v2/v3 header
  000h 2    MDEC Size/4 (after huffman decompression) (rounded to 80h/4 bytes)
  002h 2    File ID (3800h)
  004h 2    Quantization step/factor (0000h..003Fh, for MDEC "DCT.bit10-15")
  006h 2    Version (1, 2, or 3) (2 is most common)
  008h ...  Huffman compressed data blocks (Cr,Cb,Y1,Y2,Y3,Y4, Cr,Cb,Y1,Y2..)

This makes a ton of sense - not only does the File ID look right, the quantization step and version (version 3 in this case) match. I would've expected a some sort of a PSX Bitstream reader/writer to exist, but I can't find anything that really helps. The only things that gave me hope were:
· BS2PNG, which only appears to support version 2 bitstreams. Even so, it would only be a decoder, not an encoder.
· This github gist by cyxx where they masterfully decode the Heart of Darkness bitstreams and provide important context with images and code snippets, but unfortunately the game uses version 2 bitstreams and ultimately is only a decode process too. It does make use of ffmpeg, which in my mind could be helpful for encoding too but I am not an expert on the subject so I can't really say.
· psxavenc which does seem to support bitstreams, but only for v2.
· PSXpress, which does support version 3, but I do not understand how to work with it at all for either decoding or encoding, and I am unable to even compile anything related to it.

I feel powerless because the game/console makes it seem so easy and instant, just reading the data and pushing the un-huffmanned MDEC data into its proper place for decoding, but me being unable to replace it let alone even read it without the console's help.

This is the last piece of the puzzle in translating the game's assets, so I really hope someone can help with the issue. I am only a graphic designer and have a new title card graphic ready for insertion, but without a tool to pack it into its proper v3 bitstream form I feel powerless. I can program in Python, for what it's worth.

If there indeed is a general purpose bitsream decoding/encoding tool with v3 support, please let me know! If you are a programmer with intimate knowledge of the MDEC/Bitstream data format and especially that of version 3, please help me to figure out an elegant method to replace the intermission title card graphic.



One last thing to consider is that I only assume that the intermission title card is the last piece of the puzzle. There might be some other bitstreams hidden in the SM. file besides the data at 0x761A800, but the only way to confirm that is by churning through the entire entry array, visiting all of the pointers and doing the decoding song and dance. That's where a general purpose bitstream tool would come in handy, but as it stands, I feel dumbfounded because everywhere I look I feel a lot of people smarter than me know and understand the bitstream format, but working with the data (both reading and writing) is super obtuse or just impossible and that there are zero tools for working with it directly.

[Nezz]

Have you tried JPSXdec?

[Raccoon Sam]

Have you tried JPSXdec?

Yes, I have. Unfortunately jPSXdec can only detect bitstreams in proper STR containers or if they are proper stream files. It is unable to read bitstream data from inside individual files. I am under the impression that the source code has functionality for this, after all it is needed for reading v3 bitstream videos. I don't know any Java though, so I can't dig into it at source level.