I'm kinda shocked your's is 3GB, the one I have is 4.37GB haha.
Wii games use full disc encryption, even on the junk areas between files.
The gold standard for ISOs then includes this encryption. This will be the full size single layer DVD (we can skip the slightly different sizes there for certain bad dumpers as that was solved early on) for those games using single layer discs (a handful of them used double layer, which occasionally caused some problems but enough of that for now).
A slightly lesser but practically the same is a basic scrubbed iso. As the Wii (and anything that officially runs Wii games) does not check this random (and thus incompressible) padding you can blank it and gain often quite considerable compression gains.
There are a few flavours of scrubbing beyond that (mostly "safe" and ones that shuffle everything to the front of the iso and thus require properly modded IOS modules rather than a DVD chip or equivalent) but we can skip that for now.
People running things from USB drives will however often convert them to something like the WBFS format which often both decrypts the files and gets rid of any padding. Various sites would upload these, much to the chagrin of both the DVD running set, ROM preservation efforts and in option to then current Scene sensibilities (though given the massive annoyance that was going on at those times with the xbox 360's backwards compatibility with the xbox and lack of clean rips there it is not an unfounded position).
You can strip files from games to reduce size further (a process confusingly known as ripping a game) but those tended to be more techniques people did rather than isos they spread. Not to mention there was less call for it as hundreds of gig USB drives were the norm by then.
Ignoring stripped files then these days you can regenerate a full iso from a WBFS actually (previously it was taken as a lossy process so most of the Scene did not permit releases in this format, and even looked down upon basic scrubbed isos).
Anyway sounds like you are making progress on the problem itself. Still
"And as to dealing with the 12 bit encoding sorry dude you lost me, I wouldn't even know how to start by merging the results and not quite sure what you mean by the "upper bits"."
If you assume everything is 16 bits rather than end to end 12 or whatever you have now then you gain a bunch of junk that will mess things up, presumably in the higher values aka upper bits on new 16 bit values.
shift is a fairly basic operation done in binary. In its purest form just shuffles numbers to and fro (you have left shift and right shift available) just like multiplying by 10 does in decimal and pads things out with 0000, chopping off anything else in the process. It can be used then to clear bits if you want, and historically was used for it (consoles like the GBA even being noted for rapid shifting capability). Or if you prefer shuffle it 4 bits one way, shuffle it 4 back and the bits that are surplus to what you need are now harmless 0s which your relative search options should deal with more happily. If you are more used to maths then if you have done integration and differentiation then this is kind of like the plus c thing you are told to add upon integrating a function as you don't know what was there before, in this case though the devices assume it was a 0 and just go with that rather than random values.
As that technically breaks at least every other character in the game you would get back the original file, add a few bits such that you could do the shift thing a second time and gain the two halves of the whole, at which point you would have to merge it.
For the second part I meant.
Take the standard game, otherwise untouched by you. Go to the location of the "third character talking" or whatever and edit how you would from there onwards. Doing this will hopefully help figure out whether it is memory issues causing it as the previous stuff you know to work is unchanged -- while the ISO itself likely has a gigabyte or more of space available the file format itself or if the game tries loading the whole file in memory and your making it bigger caused an overflow.
If it still goes blank though then chances are it is a pointer issue or a syntax issue.
That said it looks like we are beyond that now so I will skip it.