News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Help for music hacking in Ninja Gaiden (NES) game  (Read 3396 times)

NiO

  • Jr. Member
  • **
  • Posts: 81
  • Let's bring order to chaos
    • View Profile
    • https://www.facebook.com/groups/retrogamersmx/
Help for music hacking in Ninja Gaiden (NES) game
« on: February 26, 2017, 02:03:35 am »
Well, as some of you may know, we are a group of guys working on a Ninja Gaiden hack, turning the game into a Deadpool game, changing graphics for stage, enemies, bosses and the protagonist itself.

We have managed to do more stuff, like new habilities on the game, but we have a big problem.

We don't know how to add music from other games or new music (done for .nes files).

Someone here knows how to do that?
It's possible to add music for any other game or are limitations that just allow certain games to be edited?
It's someone interested to join in the music department?

We really, reaaaaaally need help with that, because the game is already looking as a new game, not just an easy hack, but having the same old music doesn't help that feeling.

You can see what we have done here (we even make a demo release in Reply #44 if you are interested)
http://www.romhacking.net/forum/index.php/topic,23318.0.html




Googie

  • Hero Member
  • *****
  • Posts: 835
  • I'd like to solve the puzzle... Jiggers!
    • View Profile
    • My work in progress ROM Hacking site
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #1 on: February 26, 2017, 11:39:13 am »
I'll be honest with everyone here, I helped edited two levels this team put my cartoon character as an enemy, it really looks cool.

Working with these guys gave me the motivation I needed to pick up on my projects, now that I have my fiance's laptop.

This hack is so awesome, everybody working on this hack has put sweat and blood into making this hack great. If a person with musical know can help us out with the music changes, we'd really appreciate it. :thumbsup:

DANGER X

  • Full Member
  • ***
  • Posts: 166
  • Nes hacker & player
    • View Profile
    • Deadpool Nes
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #2 on: February 26, 2017, 02:54:53 pm »
As far as I know certain nes games have the same nsf header (if is called like that), so this means we can have other games music (only certain games), but is better to compose new music let's say like the deadpool pc game.
By the way I'm also in the project, I'm the lead hacker  ;D

nesrocks

  • Hero Member
  • *****
  • Posts: 609
    • View Profile
    • nesrocks.com
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #3 on: February 26, 2017, 03:24:53 pm »
I took a quick look and couldn't find much. I'll take another look later. I don't have time to do what you're asking but a kickstart could encourage people to help. It's basically a matter of finding where and how music is stored.

edit:
Here we go:
Music for stage 1:
header: 0xC83B - 0xC873
square1 part1: 0xC937 - 0xC967
square1 part2: 0xC875 - 0xC905 (this is broken into a few smaller parts)
?????: 0xC968 - C97C (this is a small section that produces no sound that I can notice)
triangle: 0xC97D - 0xCA24
DPCM: 0xCA25 - 0xCB17

So, the way I see it, the header must be the tracker that plays each section of the song, how many times, when, etc (it should have pointers to each section, I haven't looked).
As for the song sections themselves, each note seems to be 2 bytes long. The first byte does something I can't figure out what is, and the second one is the note.

edit#2: the header has two parts:
first one, starting at 0xC83B:
00 43 88
01 43 88
02 43 88
03 43 88
04 43 88
05 43 88
06 43 88
07 43 88 FF

then part2 starting at 0xC854:
04 54 88 (square 1)
05 58 89 (square 2)
02 6D 89 (triangle)
07 15 8A (DPCM)
06 43 88 FF (mute. it is the same pointer as the template)

The first byte of each 3 byte sequence seems to choose a channel. The other two bytes are pointers to where the notes are. Part 1 in this case seems to be an unused template (all pointers are the same). Part 2 indicates the song. The header has more info after this, probably to the next sections of the song.

Note that there are two pointers above at 0xC78F that indicate where these two header sections are. This is probably the place that tells what song to play.
2B 88
44 88

This is all rough still. What needs to be done is to test these bytes and see what each one does exactly.

edit#3: I think the DPCM data is stored at 0x1FC10 - 0x1FF10
« Last Edit: February 27, 2017, 01:22:14 pm by nesrocks »

DANGER X

  • Full Member
  • ***
  • Posts: 166
  • Nes hacker & player
    • View Profile
    • Deadpool Nes
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #4 on: February 27, 2017, 02:04:27 pm »
Thanks for the info, I will try to play a bit with these values.

February 28, 2017, 02:55:09 pm - (Auto Merged - Double Posts are not allowed before 7 days.)
This is really serious Asm stuff, and is not for me, still searching for people who might wanna help the hack looks awesome, it really deserve a new music.
« Last Edit: February 28, 2017, 02:55:09 pm by DANGER X »

nesrocks

  • Hero Member
  • *****
  • Posts: 609
    • View Profile
    • nesrocks.com
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #5 on: March 01, 2017, 12:20:35 pm »
There isn't really a lot of asm going on (if at all, really). It is all data, even the pointers are treated as data (some code somewhere else in the ROM looks for this list to find the addresses). Here is how to read the pointer:
Take this one for example: 43 88
1 - You switch them so it is now 88 43
2 - Then you add 10 because that's the NES header size, so now it is 88 53
3 - Now in this case you switch the first 8 with a C, so now it is C853. Why replace the 8 with a C? This is because of bank switching. In this case, the ROM range from C000-FFFF is "loaded" (not really loaded, just memory address points to this range), so it becomes 8000-BFFF. This is better explained by snarfblam here, but all you really need to know for this is that the 8 becomes a C. Note that snarfblam's explanation was about transforming a ROM location to a CPU address, but in our case we are doing the opposite. The pointer is in CPU address format and you want to find the ROM location.
4 - If you go to 0xC853 in the ROM you will find the first byte that the pointer points to and reads.

I'd say it is more of a job to someone with musical understanding than a coder. Someone needs to make a list of all byte values and what notes they represent. Also, the header has some bytes that do stuff (probably amount of loops, speed, etc). Just keep testing and feeding a list, like I did on my hack with help from FCandChill. Like this:
(the following list is for Super Pitfall, not ninja gaiden, and both games have different features and format. It's just an idea)
Code: [Select]
Square and Triangle:
Byte Note
0F REST
00 A-1
01 A-2
02 A-3
10 A#1
11 A#2
12 A#3
20 B-1
21 B-2
22 B-3
30 C-2
31 C-3
32 C-4
40 C#2
41 C#3
42 C#4
50 D-2
51 D-3
52 D-4
60 D#2
61 D#3
62 D#4
70 E-2
71 E-3
72 E-4
80 F-2
81 F-3
82 F-4
90 F#2
91 F#3
92 F#4
A0 G-2
A1 G-3
A2 G-4
B0 G#2
B1 G#3
B2 G#4
F0 REPEAT

Noise:
00 F-#
01 E-#
02 D-#
03 C-#
04 B-#
05 A-#
06 9-#
07 8-#
08 7-#
09 6-#
0A 5-#
0B 4-#
0C 3-#
0D 2-#
0E 1-#
0F 0-#

With the ninja gaiden list in hand (not the one I posted) it's a matter of writting a program that converts one of the various formats exported by famitracker, copying the bytes and inserting them on the rom, editting the header to point to the sections correctly, or do it by hand (which is how I did on my hack). I painstakingly looked at the finished song in famitracker and "translated" note by note, byte by byte to a txt file and then just copied it and pasted in fceux's hex editor in the ROM, preferably in the same place as the song to be replaced. If you're confortable with bank switching you can paste it on some other place and switch it just to read the notes. Or you can paste it anywhere in the 0xC000 0xFFFF range if is unused space and not need to bank switch to read it (just update the pointer to the new address).

Keep in mind that this game's songs have headers, and that may seem more complicated than simply a sequence of notes for each channel (which is how it is on Super Pitfall), but this allows to really save on ROM space because identical note sequences only exist once, the header tells the music engine to play that section as many times as desired.

Also you can really trust the pointers. If you look at the pointer list I originally posted, you will see that I located "square 1 part 1" and "square 1 part 2" on a different place than what is on the pointers list. You can trust that I made a mistake because the pointer is pointing some other place. For instance, I got the triangle and dpcm ones correctly.

square1 part1: 0xC937 - 0xC967 wrong guess, look at the list below
square1 part2: 0xC875 - 0xC905 wrong guess, look at the list below
triangle: 0xC97D - 0xCA24  I guessed correctly, look at the pointers list below
DPCM: 0xCA25 - 0xCB17  I guessed correctly, look at the pointers list below

54 88 (square 1) becomes 0xC864
58 89 (square 2) becomes 0xC968
6D 89 (triangle) becomes 0xC97D
15 8A (DPCM) becomes 0xCA25

I made the guess based on what I was seeing as updated bytes read using fceux's data logger as the music was loaded, and it was before I had found the pointers list.
« Last Edit: March 01, 2017, 04:51:57 pm by nesrocks »

DANGER X

  • Full Member
  • ***
  • Posts: 166
  • Nes hacker & player
    • View Profile
    • Deadpool Nes
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #6 on: March 01, 2017, 01:24:00 pm »
Your efforts will not be in vain, I will ask the guys in the group and we will change it together somehow, we won't give up.

NiO

  • Jr. Member
  • **
  • Posts: 81
  • Let's bring order to chaos
    • View Profile
    • https://www.facebook.com/groups/retrogamersmx/
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #7 on: March 02, 2017, 05:15:51 am »
Thanks a lot for that info, I dont understand it at all, Im more of an artist but I really see the effort there ti explain and I'm sure Danger knows what you wrote

Woodman

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #8 on: March 02, 2017, 11:20:16 am »
Hey dude. I am in the same position as you except with Mega Man 2, but recently (like just now) I found what looks like a REALLY promising guide on this site that's written in english. Here's the link: http://www.romhacking.net/documents/39/

I haven't actually tried editing the music in the way that it is described, but it sounds really easy. You just find the NSF (NES music) files for ninja gaiden online and load it into an NSF editor and proceed with editing. Now to me this sounds just a little too good to be true but I say it's still worth a look and a try.

NiO

  • Jr. Member
  • **
  • Posts: 81
  • Let's bring order to chaos
    • View Profile
    • https://www.facebook.com/groups/retrogamersmx/
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #9 on: March 03, 2017, 05:18:06 am »
I think Danger already use that program because I was able to hear music with it and remove some parts, but not sure on how to edit (just muted some stuff)

Thanks for the info, we will take a look at it

nesrocks

  • Hero Member
  • *****
  • Posts: 609
    • View Profile
    • nesrocks.com
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #10 on: March 03, 2017, 03:48:30 pm »
I too thought it was a little too good to be true, so I went on to read it. It seems to me that the only advantage of using a NSF file is that most of the data there is music, so it is easier to "find" it than in the ROM (this is debatable because I could locate the music on the ROM very fast using fceux's amazing debugging tools).

Maybe I misunderstood something from the guide, but he seems to be doing the same thing here. Brute force try each byte to figure out what it does, and then hex edit the new music in the NSF (that is, after figuring out the whole format and then inserting note by note, byte by byte), and then after satisfied, copy it to the ROM, but all the while having to deal with pointers. Is that so?

The way I did on Super Pitfall is I created a music call to the title screen (the original game had silent title screen) and every time I wanted to test a new song I changed the title screen music pointer to point to the new song I was testing. This way I could listen to the whole song and see if I did everything correctly before calling it done.

Also, I looked for a "NSF Editor" and I couldn't find one. Is there one? I think that even if you export a NSF from famitracker it won't be in the same format as the ninja gaiden sound engine, so you wouldn't be able to copy paste the music that way.

Sliver X

  • Hero Member
  • *****
  • Posts: 1038
  • EX•CINERIBVS•IGNIS•EXCITABITVR
    • View Profile
    • Panicus - A Fragment of Memory
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #11 on: March 03, 2017, 09:27:06 pm »
Yeah, I wrote that document in 2001, I think. This was before there were decent debugging tools available for the NES.

Editing music isn't going to be easy, though I guess you could RE the music data format and code a custom utility for that, which is just a whole bunch of additional work up front with the payoff of making the actual composition part faster and easier.

Having some kind of passing knowledge in how music is actually constructed is a huge benefit: The only way I was able to do the crap I did pre-FCEUD was because I play guitar: All my music was composed on it first, then put into the game as best as I was able given the limitations of whatever sound engine I was dealing with.

HertzDevil

  • Jr. Member
  • **
  • Posts: 8
  • 星の鼓動は愛
    • View Profile
    • GitHub
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #12 on: March 04, 2017, 01:13:25 am »
I have dumped all the music from Ninja Gaiden as MML files which are much closer to the underlying music format than tracker files. Right now I cannot insert multiple custom songs without destroying each other, but it will happen later. Note that binary formats resembling tracker output are much rarer than MML-like stream formats. Contact me if you are interested in the details.



certain nes games have the same nsf header (if is called like that), so this means we can have other games music (only certain games)
This is not how the NSF header works at all.

first one, starting at 0xC83B:
00 43 88
01 43 88
02 43 88
03 43 88
04 43 88
05 43 88
06 43 88
07 43 88 FF
...
Part 1 in this case seems to be an unused template (all pointers are the same). Part 2 indicates the song. The header has more info after this, probably to the next sections of the song.
This is not a header, this is the internal music track used to mute all channels. It appears 8 times on the master song list. "43 88" is simply the pointer to the "FF" that appears last; it is used in many other songs as well. At certain points the game presumably mutes all playing sounds before starting a new one. Immediately following stage 1's true header is the stream data for the Pulse 1 track.



The master song table is located at $877D, containing 85 music tracks and sound effects (with some duplicates); stage 1 is track 3, located at $8844, which is pointed from $8781 - $8782.

A song header is simply a list of track indices followed by pointers to the track data stream, terminated by a FF. Indices 04 to 07 are for Pulse 1, Pulse 2, Triangle, and Noise respectively. 00 - 03 are the same, except they have higher priority, for use by sound effects. DPCM is entirely done by effect commands. In the case of stage 1, track 02 handles the triangle, 07 handles noise and DPCM, and 06 is completely silent (it has the "43 88" seen above), but in some other songs track 06 is used for DPCM so that the triangle channel can share its patterns with other tracks.

The command table is as follows:
  • 00 - 7F (Pulse / Triangle): Inputs a note. High nybble is the octave, low nybble is the pitch name (024579B corresponds to CDEFGAB). 0C is used for note rest.
  • 00 - 10 (Noise): Inputs a note. 00 is highest, 0F is lowest. 10 is used for note rest.
  • 80 - AF: Sets the note duration using the tick count table at $8586. The sound engine does not have the concept of tempo, all time control must be done through ticks. The following length values are actually used:
  • E0 xx: Uses the volume envelope selected from the table at $85B1. Each volume envelope is defined by length-value pairs; the last item has a length of FF. 23 volume envelopes are defined.
  • E2 xx: Sets the duty cycle of the pulse tracks. 00 / 40 / 80 / C0 corresponds to 12.5% / 25% / 50% / 75% respectively.
  • E3 xx: Reduces the output volume of the track by (xx + 1).
  • E4 xx: (used in sound effects only, controls the pulse hardware sweep unit)
  • E5 xx: Detune, operates by adding xx, unsigned, to the period register. Higher value means more downward detune; there is no way to detune upwards.
  • E8 xxxx: Unconditionally moves the track pointer to xxxx. Used for looping a song or sharing entire streams between tracks.
  • E9 xxxx: Invokes the pattern at xxxx. Patterns can be nested once within other patterns.
  • EA: Returns from a pattern.
  • EB xx: Starts a local loop that plays the contents xx times. Like patterns, they can be nested once.
  • EC: Ends a local loop.
  • ED: Enables vibrato.
  • EF: Disables vibrato.
  • F8: (used in sound effects only)
  • F9: (used in sound effects only, seems to silence the DPCM channel)
  • FA xx yy zz: Plays a DPCM sample by writing xx, yy, zz to $4010, $4012, and $4013 respectively; does not interrupt currently playing samples. Unless you are planning to add your own samples, the only parameters that appear in the game are 0F F0 10 for kick, and 0F F4 20 for snare.
  • FB xx yy zz: As above, but will interrupt playing samples.
  • FE xx: Causes the track to fade out by reducing the final volume by 1 per xx ticks.
  • FF: Ends the current track.
An example of volume envelope: Instrument 0x12 is used only in the noise track of one of the ending themes. Its pointer is located at $85D5, and the actual envelope stored at $86B3 reads:
Code: [Select]
Original: 01 0F 02 0D 01 0B 02 09 02 07 02 06 02 05 01 04 01 03 01 02 01 01 FF 00
Unfolded: 0F 0D 0D 0B 09 09 07 07 06 06 05 05 04 03 02 01 00
Because of a E3 05 command, it is never heard in full, resulting in "9 7 7 5 3 3 1 1 0".

More about 80 - AF: Only entries up to AA are actually defined, and not all length values are used:
Code: [Select]
    +00 +01 +02 +03 +04 +05 +06 +07 +08 +09 +0A +0B +0C +0D +0E +0F
+80 ---   1   2   3   4   5   6   7   8   9  10  12  14  15  16  18
+90  20  21  24  27  28  30  32  36  40 ---  48  54  56  60  64  72
+A0  80  84  96 108 112 128 144 --- --- ---  13

Also, apart from command ED, some songs create vibratos manually, usually when the current instrument is blank (E0 05). An example from Pulse 2's first pattern of the intro theme, at $9CFF:
Code: [Select]
EB 24                                        ; loops the content 36 times (instrument is blank here)
   81                                        ; set note length to 1 tick
   18 E5 02 18 E5 04 18 E5 02 18 E5 00       ; plays G# for five ticks as if the envelope { | 0 2 4 2 0 } is used
EC                                           ; 36 * 5 = 180 ticks = 3 seconds
EB 24 16 E5 02 16 E5 04 16 E5 02 16 E5 00 EC ; F#
EB 24 14 E5 02 14 E5 04 14 E5 02 14 E5 00 EC ; E
EB 24 13 E5 02 13 E5 04 13 E5 02 13 E5 00 EC ; D#
EA                                           ; end of pattern
« Last Edit: March 04, 2017, 10:53:20 am by HertzDevil »
Constructing Chiptune; Construing 8-Bit. Makes a generic MML compiler in Lua.

Woodman

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #13 on: March 06, 2017, 12:17:46 am »
HertzDevil, that guide you have there, especially the command table, looks like it will be VERY useful once I'm done composing the music and start putting it into the game. I cannot thank you enough.

And to Silver X, your guides on music have been really helpful, but I want to know if you recommend writing music the old way your guide describes or via FCUEX. I'm kind of in the middle at the moment as I haven't really started putting the music to code yet but I want your opinion.

Sliver X

  • Hero Member
  • *****
  • Posts: 1038
  • EX•CINERIBVS•IGNIS•EXCITABITVR
    • View Profile
    • Panicus - A Fragment of Memory
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #14 on: March 06, 2017, 12:32:50 am »
The most surgical way to do it would be to watch the registers used for sound generation in a debugger then tracing the data being read from there.

Without assembly, you can use FCEUX's "Code Data Logger" to find music data and also edit it in real time. Take a save state right before a song starts and start logging in the hex editor. Let it play for a few seconds then stop logging. Look through ROM in the hex editor.

Most likely the music data is going to show up as blue bytes (Ignore yellow, as that's code. Green would indicate bytes being read as code AND data, which I do seem to recall being the case in a few music formats I looked at years ago?). Start corrupting shit until you find a piece of music data, then reverse engineer its format from there.

« Last Edit: March 06, 2017, 12:40:20 am by Sliver X »

Woodman

  • Jr. Member
  • **
  • Posts: 20
    • View Profile
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #15 on: March 06, 2017, 08:41:07 am »
So FCUEX? Got it. I'll look into using that method once I get to the coding. Sounds a lot more consistent than hacking away at the nsf, even if the nsf method is simpler.

elmer

  • Full Member
  • ***
  • Posts: 122
    • View Profile
Re: Help for music hacking in Ninja Gaiden (NES) game
« Reply #16 on: March 06, 2017, 10:12:33 am »
I have dumped all the music from Ninja Gaiden as MML files which are much closer to the underlying music format than tracker files. Right now I cannot insert multiple custom songs without destroying each other, but it will happen later. Note that binary formats resembling tracker output are much rarer than MML-like stream formats. Contact me if you are interested in the details.

Nice analysis!  ;)

Yep, an interpreted bytestream-per-channel is what I'm used to in old games.

I don't know of people using pure tracker data in games, but I wouldn't be surprised to hear that some crazy folks did it.

It's not difficult to convert tracker data into a regular bytestream-per-channel format ... but that's more for homebrew developers rather than romhackers.