I think there is a lot that could be said given the comprehensive nature of the program. The primary barrier would be in my ability to adequately convey the information to the audience. I have never posted at length on a regular basis, so I am not sure if I would be good or bad at it.
The other issue would be the irregular posting schedule. I would only be posting when I am actually working on the project, and when interesting issues occur.
(As this primarily involves code problem solving, I have posted here. But if this better belongs in Personal Projects, feel free to move the thread.)
I have been running into an issue with the original ROM running out of free space when all data records have been marked as modified, without changing the data. This is unusual because, in theory, if none of the data has actually been changed, it should not take up any more space than it did originally. The cause appears to be several things.
The first was somewhat easy to find. Several of the record types lived in space not marked as usable to the program. Even worse, some of the record types were only partially marked as usable. For example, I had only marked the primary group of Location Maps as usable, but there are two or three smaller groups of maps stored in other places in the ROM. Tracking this down manually would have been time consuming and error prone, comparing the list of usable space to the entire list of offsets (which contains records, unused space, and code). Fortunately, this process could be automated. After each of the records retrieved its data, it then checked the free space list to see if its home address was available. If not, it popped up an error message.
Unfortunately, this does not appear to be the only problem, as even after correcting for all of the error messages, the ROM still runs out of space. It is not immediately clear why, but I suspect it has something to do with new records being created. Flux can split an overlapping record into two new ones, or create entirely new ones in cases where there is room for more of that type. There is little point into looking into this because it is not falsifiable: new records will always take up more space. I need to make sure there are no other causes before I try to eliminate this.
Maybe Flux is not saving the records as efficiently as the original ROM, taking up more space. This is the avenue I am currently investigating. Hampering my progress, Flux has never saved compressed data in a matching one-to-one manner as the original game. While it still works, the compression addresses never matched up. The fix here appears to be surprisingly simple. Instead of looking for matching patterns to compress from the beginning and moving forward, it seems the original tool started at the end and worked backward. Merely changing the end I started at appears to have made it sync up and post one-to-one results. I was so surprised that was all it took that, at first, I figured the game was not actually saving the data and was just dumping the ROM back out to file again. I had to test it several more times and even actively debug the program and physically see it doing the work to believe it. This will help the investigation immensely.
That is all for now. Let me know if there is any interest in more.