Managed to read Star Fox SNES ROM using VL-Tone guides. I need more info.

[dyson132]

Hello everybody!
I'm new here but I think you guys might like this.
A few weeks ago I tried to read all the information that was ever posted by VL-Tone about the subject and managed to find a lot... just no all of it.
VL-Tone was the guy that made this http://homepage.mac.com/qubedstudios/StarFoxObjectDecoder.htm
I had to use WayBack Machine to find some stuff and I still have some color/texture and lighting information missing (here is one with the missing palettes that might I need http://web.archive.org/web/20050517170102/http://board.acmlm.org/thread.php?id=6876 ).
I've attached what I did reconstruct, it was programmed in c++ and openGL. The model coordenates were obtained using VL-Tone guides, all that is missing there is the correct colors which I have not been able to find the 16 color palette used all over starfox inside the rom, nor have I found the decoded 256 RGB palette used by VL-Tone in the Star Fox Object Decoder. That information seems to be missing.

I'd like to be able to get more info on the subject, specifically find out where in the rom is the 16 color palette mentioned and the correct lighting settings.
Any help would be great and I can asure you this test will end up in the public domain as (as least) a 60fps full HD playable Corneria Stage <--- you should be able to add all the fan music covers you like to this.

Thanks!




April 13, 2012, 12:28:00 PM - (Auto Merged - Double Posts are not allowed before 7 days.)
Here is a video a captured using fraps. The Airwings Lighting and coloring is wrong, that is why I need more info or maybe to contact VL-Tone.
https://vimeo.com/40300915

[MathOnNapkins]

That is pretty cool! And let me be the first to say







DO A BARREL ROLL!

[FinS]

The palettes are built in ram at $7e178b before being uploaded to cgram. The palette for 3d models specifically is the 8th palette so it is first loaded to $7e186b in ram before being sent to cgram. There are actually 3 palettes, each of 16 colors, for 3d. They are located complete in rom at $18acb unheadered addressing in order:

Space Palette

Code Select Expand

00 00 51 04 3A 15 BD 22 7F 37 45 54 2B 6D 8F 7E B6 7F A3 0C 88 25 4E 42 F5 56 9A 6B FE 7F 00 03
Night Time Palette

Code Select Expand

00 00 B5 00 9E 11 DF 1A 7F 2F A5 5C 4C 6D F1 76 B7 7F 47 08 ED 1C D0 39 74 4E 38 6B FE 7F 20 02
Daytime Palette

Code Select Expand

00 00 35 20 7E 35 BF 36 5F 4B 40 6D 2C 7E 6D 7F F5 7F C3 24 89 39 0E 4E D3 62 78 77 FD 7F 20 02

This is what they look like in no particular order.


My best guess is that the screen is set to full brightness but I can't be sure because that seems to be controlled by the FX chip which I don't understand very well. Someone here probably will  have some knowledge about it.

[dyson132]

@MathOnNapkins:
xD I'm definitely doing barrel rolls right now!

@Fins:
Thank you so much!, I'm really going to try out that program. I'm guessing the greens at the ends are supposed to be the "alphas" in the game maybe (According to VL-Tone one of the hex values in the palette was used for alpha)
I do have a question, is anybody else so into this that they managed to get the correct light source info? I have the normals for every model so openGL is ready for correct lights!
I'd like to be able to replicate the lighting used in the Star Fox Object Decoder!

Thanks again!

Edit->
@FinS:
Well VT-Tone found out that the palette of 16 colors is actually used to make a kind of grid over polygons using the 16 colors as a basis, creating the illusion of 256 colors for any poligon. When he made the SF object decoder he first decoded the colors and with that info he generated the 256 color palette that he used.
But of course it would be great to have as much information as posible and your input is just what I needed. With it I can decode the 256 palette.

[FinS]

Here is an example of what you are talking about. You can make a zsnes save state and open it with vsnes to find out almost anything you need to about the game except for fx processor stuff.


You can see here that this is background 1 which contains all of the wire frame objects and some other things such as the portrait, shield health bars and some text. You can see here how the grey shade pixels are meshed with other colors to cause the illusion of multiple colors even though it only uses the 16 colors of the 8th palette which you can see almost directly in the middle of the palette editor. The first color in each palette is always transparent so it is actually only 15 colors after the fact and I don't believe the fx chip is changing that.  The green color is only being used as the background of the portrait. The shield health bars which I cut off both use the 3rd palette which is the red, blue, pink and grey one.


This is what is displayed in background 2 which is almost everything else. The only sprites are just a few pieces of text, lives display and such. These graphics use the 4th, 5th, 6th and 7th palettes.

[dyson132]

 
This is definitely usefull!
Thank you very very much! xD
By the way, how did you export the palette to use it in ImPalEd?
I'm really into this right now!

[FinS]

ImPalEd can read a multiple variety of 256 color palettes. Here is one way to make a palette using geiger's debugger.

Run the game in the debugger up to a point you want to get the palette then dump it, highlight and copy the numbers, then open up a hex editor like HxD, create a new file and paste the palette numbers in.


Then after you save the file with a ".pal" extension you can open it in ImPalEd. Impaled is able to open 15-bit, 16-bit, 24-bit, 32-bit, Jasc and Riff palettes.

Alternatively, you can also just find the palette entries you want to use and place them in a file then pad the rest with zeros in a hex editor up to a point it will make the file the correct size with regards to the size of the palette entries. For example, say you are using 16-bit per color entries, then you would need to pad a total file size of 512 bytes to make 256 colors.

[dyson132]

Nice!

Now I have a huge problem, seems the colors in the models are references to other references in hex values so I'm kinda stuck until I manage to get that to work, although with your information I did manage to find the palete inside the rom.

[FinS]

I am not sure but I think you are referring to the different formats that palettes come in. The format dumped by Geiger's Debugger is not the same format which the Snes uses naturally. It is the regular 24 bit hex format just like the color format used in web pages.  That is a reason ImPalEd comes in handy. You can load a 24 bit palette with it and save it as BGR 15 bit which is the format SNES uses.

[dyson132]

Well I'm not sure, I'm very new at this. Here is the info.
VL-Tone says that for every 3D model in the Star Fox ROM the 28 byte object definition has a reference to a certain set of color/texture (I have verified this):

There are about 9 of these sets and the 28 byte
object definition has a pointer to tell which one is used, for example 13 82 is the most commonly used palette (refer to my other post to know where the 28 bytes object definition table is.)
That 13 82 number translate to $18213+$200=$18413 in the ROM. At this address you can find 256 2 bytes color/texture references.

Source: http://web.archive.org/web/20050517170102/http://board.acmlm.org/thread.php?id=6876

Now when you look at all the hex values that make up the airwing you end up with vertex data, a BSP (for rendering I believe), a list with triangles formed by the vertex data for said BSP, and more importantly, the values with which to build the 3d Model which is like this:

Ns Fn Co NX NY NZ V1 V2 V3... Ns is the number of sides for this face. Fn is face number or ID.
-------------------------- Co is color/texture number, refer to other doc about the color/texture sets.
NX NY and NZ is the vector of the calculated normal for this face normalized to 128 (or is it 127?)
Vx refers to the vertex numbers as defined by the vertex lump list just like the triangle list (see above).
The number of Vx vertex references is equal to the number of sides, in this case three.
Supported number of sides are 1 (point),2 (line),3,4,5,6 and 8 sides.
Ns Fn Co NX NY NZ V1 V2 V3
--------------------------
[03 06 10 A6 5A F7 03 02 0B] ->This face is a triangle with a gray tint ($10), with a normal vector of (-90,90,9), connecting vertex 3, 2 and 11.
[FF] End of faces group($FF)
[14] Start of Face group (convex shape) ($14)
[03 0E 07 BF 64 D5 0A 0E 07]
[FF]

Source: http://pages.infinit.net/voxel/Arwing_Deconstruction.html

When you look at the Co value for the first triangle it says that it should be gray tint. The value which it references is supposed to be a value in a certain set inside the 13 82 "template", or something that ends up being a grey color made up of a checkerboard pattern of colors in the 16 color palette.
This is where I get lost with all the values and references.
=(

[FinS]

I am assuming this is Super FX related material and I am lost on it as well.

[dyson132]

I am assuming this is Super FX related material and I am lost on it as well.

Yeah, I think this is where finding VL-Tone would be awesome! Although your information points me in the right direction so thank you very much! =D
April 15, 2012, 01:35:55 AM - (Auto Merged - Double Posts are not allowed before 7 days.)
In the meantime, here is another video. The palette is more accurate but I set them by hand, not really perfect and not what I really wanted. 
https://vimeo.com/40384833

[Mattrizzle]

Here's what I have so far after dissecting the table at $18213:

Most values beyond 0x2F are either additional animated colors, or textures (which I'm too lazy to rip due to the interleaved format they use).
Space and sunset versions of this image can be found here and here, respectively.

The meanings of the values in the second column are in the initial post of the archived Acmlm's Board topic you linked to (it can also be found here). However, I noticed that VL-Tone made a couple of mistakes, which I will try to address now.

Paragraph 5:

At this address you can find 256 2 bytes color/texture references.

There are actually only 109 color/texture references in the table at $18213, because another table begins at $182ED. This means the valid values are from 0x00 to 0x6C.  Also, I counted way more than 9 of the sets when scanning through the object definitions (there's actually close to 50!).

Going into the format descriptions:

XX 00 to XX 08= Nine light sourced colors, the different variations are stored at $19000 in ROM.

The number is actually ten, from 00 00 to 09 09, as can be seen in my image above. The data for them begins at $18E0A in the ROM ($1900A headered). Pointers to these can be found at $18B2A-$18B89.

Here are the 4 consecutive shade variations of these colors, in the space, sunset, and daytime palettes. Each 8x8 square is equivalent to a single byte in the ROM.

I will post images with the numerous variants of the 32 solid colors when I finish them.

[dyson132]

@Mattrizzle:

S**T!
This is AWESOME!!! xD
THANK YOU! =D

I had read that thread over at Aclmm but I have little experience with HEX and the pictures VL-Tone posted are offline, that is why I used Way Back Machine for the same thread. I had come across stuff but I didn't know they were mistakes, again, I have little experience.

This is turning out better than I expected! thank you so much!

Edit---------------------------------------------------------------------------------------->
Palettes match up perfectly. =D
Now to get the correct lighting. xD
May I ask what software you used in order to get the palettes printed in such a way?

Edit 2-------------------------------------------------------------------------------------->
I will post all links to all the info I have managed to find in case anybody else wants to remake the game themselves or wishes to make a bigger project:

Starfox Level data found, and hidden new weapon
http://acmlm.no-ip.org/archive2/thread.php?id=6066&ppp=20&page=0#120061

Starfox Polygon Viewer, Now in Color!
http://acmlm.no-ip.org/archive2/thread.php?id=6876

Annotated Deconstruction Of Low Polygon Arwing Object From The Original StarFox
http://pages.infinit.net/voxel/Arwing_Deconstruction.html

Preliminary Stage Hacking (older post by VL-Tone inside another post)
http://acmlm.no-ip.org/archive2/thread.php?id=4730&ppp=20&page=0#96583

More info!-------------------------------->
When mirroring X-axis vertex data as mentioned in the Arwing Deconstruction, it should be done immediately for every vertex. Meaning that the first vertex on the list is followed by its mirrored vertex, once for each vertex (there are 6 mirrored). This took me some time to understand since the document isn't too explicit about it so I just went ahead and mirrored the 6 vertex values AFTER the first normal ones so my first model was badly rendered.

SFXedit 0.93b has a function in which it can export 3d models to .obj files. I never got it to work either on mac or on PC

SFXedit 0.93b has a complete (?) stage analisis for most (?) stages. It has (what I believe) every single bit of information I will need to remake a level. But I cannot export that information out in any way, so it has to be done some other way.

VL-Tone used to have a text file with all the model object pointers (I believe). The file does not exist anymore.

I documented almost all of these objects in this document: http://pages.infinit.net/voxel/STARFOX_OBJECTS.TXT

[Mattrizzle]

You're welcome, dyson!

May I ask what software you used in order to get the palettes printed in such a way?

I created an indexed color image in Paint Shop Pro version 4, which is little more than your average paint program, aside from having better handling of indexed color palettes than most.
I retrieved the colors needed for the palettes by opening the ROM in TiledGGD, going to the offset where they start, setting the image format to 16 bits per pixel, changing the panel size to 16x3, and copying/pasting the image to Paint Shop Pro. This yields a bitmap image with all three 16-color palettes. I edit the palette in the indexed image to match one of the palettes in the copied image, then with a hex editor (XVI32, although I usually import the hex string to a text editor and add split them into multiple lines so that they are easier to read) opened to the appropriate location in the ROM, use the data to lay pixels of the correct color index down. The text is added to the image using Paint Shop Pro as well.

For the animations, I made multiple copies of the completed image to use as frames, modified the colors to be animated in each one, and then imported these frames to a program that creates animated GIFs (Ulead GIF Animator 2.0).

In short, I don't have a single program that is specialized to do the whole task.

Also, as promised, here are the variants of the "solid" colors:
Space palette
Sunset palette
Daytime palette

[dyson132]

That is awesome work Mattrizzle.
I do have another question regarding all these topics. I've come accross a lot of problems while trying to do this and I only have like a year of c++ experience, even less in openGL (I'm learning that now). I was wondering how I should go about proposing this "little project" of mine to other people, who to send messages to, which other forum would be appropiate for this type of work. I know I can go further but with other people I may go faster and actually get somewhere. So what do you think? I'm actually a newby here, I've only made and posted in this thread and I would like to get people interested in this Star Fox HD in order to see it work in the future.

In the mean time, does anybody have any idea how the lighting works in this game? I've found little to no information directly addressing this issue but VL-Tone seems to have figured it out!
I can handle the palettes and all, program the correct use of them, but everytime I look at the game I get more and more confused.

[Stef]

Hi Dyson,

Exciting project ! It looks really good so far, i wonder if you are still working on it ?
I plan to do a sort of Starfox port but not for PC I collected some informations and so far i have successfully got the palette data and figured almost the most important about the format (lighting, animations...). I also ripped some backgrounds but i still miss the most important which is the models. I found only the arwing model but i would like to rip others 3D models too. I tried to use SFXEdit but the export to OBJ function don't export anything :-/ I wonder if you still have your models as you seems to have successfully exported them.
I can help you in figuring the light stuff

[dyson132]

Hi Dyson,

Exciting project ! It looks really good so far, i wonder if you are still working on it ?
I plan to do a sort of Starfox port but not for PC I collected some informations and so far i have successfully got the palette data and figured almost the most important about the format (lighting, animations...). I also ripped some backgrounds but i still miss the most important which is the models. I found only the arwing model but i would like to rip others 3D models too. I tried to use SFXEdit but the export to OBJ function don't export anything :-/ I wonder if you still have your models as you seems to have successfully exported them.
I can help you in figuring the light stuff

Awesome! I had left the project on permanent pause due to lack of information. I did find VL-Tone in a forum that he seems to moderate frequently but at some point in the past he posted that he had a horrible hard drive crash in which he lost most of the work and wouldn't get back to it, I can understand that. Here is the forum I just mentioned:
http://jul.rustedlogic.net/forum.php?id=41
If there was anything I really wanted to read was the source code for the Macromedia Shockwave file for SFXEdit but it seems to have been destroyed.
I'm sorry to say Stef that I have had little to no progress with this project. What you see in this thread is what I know. At most I can tell you that VL-Tone was able to read Super Mario 64 models because (it seems) the 3D model system appears to be based to that of Star Fox for the SNES (something worth checking). Other companies wanted to copy the system because it was very efficient. Also, the ROM file has no encryption, it is just an awesome and very efficient use of the limitations of the SNES.

I was also never able to rip any 3D model using SFXEdit. The Airwing model is the same as in the info posted by VL-Tone. What I was able to do was verify that it existed where it was supposed to and I parsed it with MS Excel (really!) so it is really easy to program once the info is read correctly.

I've got some important personal stuff these next two weeks, after that I can look into this again but I don't think I can go further than I had already without help.

Good luck!

[Stef]

Awesome! I had left the project on permanent pause due to lack of information. I did find VL-Tone in a forum that he seems to moderate frequently but at some point in the past he posted that he had a horrible hard drive crash in which he lost most of the work and wouldn't get back to it, I can understand that. Here is the forum I just mentioned:
http://jul.rustedlogic.net/forum.php?id=41
If there was anything I really wanted to read was the source code for the Macromedia Shockwave file for SFXEdit but it seems to have been destroyed.
I'm sorry to say Stef that I have had little to no progress with this project. What you see in this thread is what I know. At most I can tell you that VL-Tone was able to read Super Mario 64 models because (it seems) the 3D model system appears to be based to that of Star Fox for the SNES (something worth checking). Other companies wanted to copy the system because it was very efficient. Also, the ROM file has no encryption, it is just an awesome and very efficient use of the limitations of the SNES.

I was also never able to rip any 3D model using SFXEdit. The Airwing model is the same as in the info posted by VL-Tone. What I was able to do was verify that it existed where it was supposed to and I parsed it with MS Excel (really!) so it is really easy to program once the info is read correctly.

I've got some important personal stuff these next two weeks, after that I can look into this again but I don't think I can go further than I had already without help.

Good luck!

Thanks for replying that quickly !
Ok, you got the arwing model the same way i got it : just grabbed data on the VL-Tone format description example :p
Indeed you are right, i can directly rip the data model from the rom, just need to know where they are located (and SFXEdit could help for that). If you were able to parse the model data with Excel, what was missing you ?
If you plan to get back in the project i would be pleased to help you in figuring a bit more of the format

[Friedslick6]

Just making a list of every link used to post information about VL-Tone's StarFox viewer:
Download SFXedit 0.93b
StarFox Level Editor - QuickTopic free message board hosting
Starfox Object Decoder 0.2a
Starfox Object Decoder 0.5a - Now in Color!
StarFox Object Viewer
Starfox Level Editor 0.8b
SFXedit 0.8 online documentation
SFXedit 0.93 Documentation.
Acmlm's Board -- Rom Hacking: Starfox Polygon Viewer, Now in Color!
Is there any way to decompile the Shockwave executables used on the online editors?