News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - HertzDevil

Pages: [1]
I did.

The entire inserter is available on my GitHub. (I also made a thread somewhere here, I just have not updated it in months.) If you have Lua, you can readily insert custom music into the ROMs listed here. The only thing lacking here is the documentation for Castlevania's MML engine definition, but those who have done a bit of music hacking before should be able to figure out how to write the MML files for Castlevania.

So I loaded it up, messed around with the title theme a little bit, and exported the result as an nsf and loaded it and the original nsf into translhextion for a comparison. Immediately, I noticed that a large chunk of the substance of the original file was gone, replaced with random repeating digits. This was probably due to the fact that Famitracker can only open 1 "song" from an nsf file at one time, overwriting the space that the other songs occupied with garbage data.

Now I have tried scrolling through, trying to find bits that look like music, copying and pasting into the Hex editor in FCEUX, and nothing of note has happened besides game corruption, but I feel like I've barely scratched the surface. Either I'm just dumb and Famitracker has reformatted the nsf to where it's basically useless, or I have no idea what I'm doing or where I'm putting the code or what code I'm putting in so the desired result has still yet to be achieved.
This is full of dangerous assumptions. You are not scratching the surface any more than other methods of memory corruption can achieve.

The FamiTracker NSF Importer mod does not reconstruct any input NSF file. The moment you import an NSF file, its internal structure is already lost; the resulting FTM file only attempts to replicate the song down to every single register write. All that much of data in the re-exported NSF is used for these granular events, usually bigger than the whole original NSF. If you have ever seen a raw VGM file in a hex editor, this is more or less the same thing. It also never "[overwrites] the space that the other songs occupied"; you can manually merge multiple such FTMs into a single file, and still the result is the same. FamiTracker can write unused data, but never garbage data under normal usage.

I have dumped all the music from Ninja Gaiden as MML files which are much closer to the underlying music format than tracker files. Right now I cannot insert multiple custom songs without destroying each other, but it will happen later. Note that binary formats resembling tracker output are much rarer than MML-like stream formats. Contact me if you are interested in the details.

certain nes games have the same nsf header (if is called like that), so this means we can have other games music (only certain games)
This is not how the NSF header works at all.

first one, starting at 0xC83B:
00 43 88
01 43 88
02 43 88
03 43 88
04 43 88
05 43 88
06 43 88
07 43 88 FF
Part 1 in this case seems to be an unused template (all pointers are the same). Part 2 indicates the song. The header has more info after this, probably to the next sections of the song.
This is not a header, this is the internal music track used to mute all channels. It appears 8 times on the master song list. "43 88" is simply the pointer to the "FF" that appears last; it is used in many other songs as well. At certain points the game presumably mutes all playing sounds before starting a new one. Immediately following stage 1's true header is the stream data for the Pulse 1 track.

The master song table is located at $877D, containing 85 music tracks and sound effects (with some duplicates); stage 1 is track 3, located at $8844, which is pointed from $8781 - $8782.

A song header is simply a list of track indices followed by pointers to the track data stream, terminated by a FF. Indices 04 to 07 are for Pulse 1, Pulse 2, Triangle, and Noise respectively. 00 - 03 are the same, except they have higher priority, for use by sound effects. DPCM is entirely done by effect commands. In the case of stage 1, track 02 handles the triangle, 07 handles noise and DPCM, and 06 is completely silent (it has the "43 88" seen above), but in some other songs track 06 is used for DPCM so that the triangle channel can share its patterns with other tracks.

The command table is as follows:
  • 00 - 7F (Pulse / Triangle): Inputs a note. High nybble is the octave, low nybble is the pitch name (024579B corresponds to CDEFGAB). 0C is used for note rest.
  • 00 - 10 (Noise): Inputs a note. 00 is highest, 0F is lowest. 10 is used for note rest.
  • 80 - AF: Sets the note duration using the tick count table at $8586. The sound engine does not have the concept of tempo, all time control must be done through ticks. The following length values are actually used:
  • E0 xx: Uses the volume envelope selected from the table at $85B1. Each volume envelope is defined by length-value pairs; the last item has a length of FF. 23 volume envelopes are defined.
  • E2 xx: Sets the duty cycle of the pulse tracks. 00 / 40 / 80 / C0 corresponds to 12.5% / 25% / 50% / 75% respectively.
  • E3 xx: Reduces the output volume of the track by (xx + 1).
  • E4 xx: (used in sound effects only, controls the pulse hardware sweep unit)
  • E5 xx: Detune, operates by adding xx, unsigned, to the period register. Higher value means more downward detune; there is no way to detune upwards.
  • E8 xxxx: Unconditionally moves the track pointer to xxxx. Used for looping a song or sharing entire streams between tracks.
  • E9 xxxx: Invokes the pattern at xxxx. Patterns can be nested once within other patterns.
  • EA: Returns from a pattern.
  • EB xx: Starts a local loop that plays the contents xx times. Like patterns, they can be nested once.
  • EC: Ends a local loop.
  • ED: Enables vibrato.
  • EF: Disables vibrato.
  • F8: (used in sound effects only)
  • F9: (used in sound effects only, seems to silence the DPCM channel)
  • FA xx yy zz: Plays a DPCM sample by writing xx, yy, zz to $4010, $4012, and $4013 respectively; does not interrupt currently playing samples. Unless you are planning to add your own samples, the only parameters that appear in the game are 0F F0 10 for kick, and 0F F4 20 for snare.
  • FB xx yy zz: As above, but will interrupt playing samples.
  • FE xx: Causes the track to fade out by reducing the final volume by 1 per xx ticks.
  • FF: Ends the current track.
An example of volume envelope: Instrument 0x12 is used only in the noise track of one of the ending themes. Its pointer is located at $85D5, and the actual envelope stored at $86B3 reads:
Code: [Select]
Original: 01 0F 02 0D 01 0B 02 09 02 07 02 06 02 05 01 04 01 03 01 02 01 01 FF 00
Unfolded: 0F 0D 0D 0B 09 09 07 07 06 06 05 05 04 03 02 01 00
Because of a E3 05 command, it is never heard in full, resulting in "9 7 7 5 3 3 1 1 0".

More about 80 - AF: Only entries up to AA are actually defined, and not all length values are used:
Code: [Select]
    +00 +01 +02 +03 +04 +05 +06 +07 +08 +09 +0A +0B +0C +0D +0E +0F
+80 ---   1   2   3   4   5   6   7   8   9  10  12  14  15  16  18
+90  20  21  24  27  28  30  32  36  40 ---  48  54  56  60  64  72
+A0  80  84  96 108 112 128 144 --- --- ---  13

Also, apart from command ED, some songs create vibratos manually, usually when the current instrument is blank (E0 05). An example from Pulse 2's first pattern of the intro theme, at $9CFF:
Code: [Select]
EB 24                                        ; loops the content 36 times (instrument is blank here)
   81                                        ; set note length to 1 tick
   18 E5 02 18 E5 04 18 E5 02 18 E5 00       ; plays G# for five ticks as if the envelope { | 0 2 4 2 0 } is used
EC                                           ; 36 * 5 = 180 ticks = 3 seconds
EB 24 16 E5 02 16 E5 04 16 E5 02 16 E5 00 EC ; F#
EB 24 14 E5 02 14 E5 04 14 E5 02 14 E5 00 EC ; E
EB 24 13 E5 02 13 E5 04 13 E5 02 13 E5 00 EC ; D#
EA                                           ; end of pattern

ROM Hacking Discussion / Journey to Silius music hacking notes
« on: January 09, 2017, 10:32:50 am »
Some notes I made while adding support for Journey to Silius to my generic MML inserter. They will be sent to the database or the wiki later.

Command List
  • 00 xx, 01 xx, ... 7F xx: Plays a note for xx ticks; 00 is C2 (MIDI note 36). The high nybble does not seem to have any purpose on the noise track. xx must be omitted if fixed note duration is enabled (see F6).
  • E5: Finishes a group of slur notes. E0 to E4 are all unused aliases.
  • E6: Disables the current pitch envelope.
  • E7 xx yyyy: Sets a pitch envelope; The envelope pointed by yyyy starts xx ticks after each new note.
  • E8 xx yyyy: (unused)
  • E9: Begins a group of slur notes; the next note plays as usual, and then all new notes do not retrigger the instrument envelopes until the next E5 command.
  • EA xxxx: Sets a volume/duty envelope; the envelope pointed by xxxx starts with all new notes.
  • EB xx: Writes xx to the 2A03 pulse sweep register ($4001 / $4005) for all new notes.
  • EC xx: On the pulse or noise channel, disables the volume envelope and writes xx to the volume/duty register ($4000 / $4004 / $400C) for all new notes. On the triangle channel, writes xx to the linear counter register ($4008) for all new notes.
  • ED xx yy: Waits for yy ticks, decreasing the track volume by 1 every xx ticks.
  • EE xx: Forces all new notes to trigger the release envelope xx ticks before it finishes. Unused.
  • EF: Disables the release part of the volume/duty envelope.
  • F0: Halts the current track.
  • F1 xx: Note rest; plays nothing for xx ticks. xx must be omitted if fixed note duration is enabled (see F6).
  • F2 xx: Extends the duration of the previous note by xx ticks. xx must be omitted if fixed note duration is enabled (see F6).
  • F5 xx: Gate time; behaves like EE, but the release time is at most half the note duration. F3 and F4 are unused aliases.
  • F6 xx: Enables fixed note duration; all future note events (00-7F, F1, F2) use xx as the number of ticks.
  • F7: Disables fixed note duration.
  • F8 xx: Detune; adds (xx - 80) to the channel's period register. Values less than 80 will therefore raise the pitch.
  • F9 xx: Transpose; shifts all new notes by xx semitones.
  • FA: Leaves the current pattern, if there is one.
  • FB xxxx: Invokes the pattern pointed by xxxx.
  • FC xxxx: Unconditionally moves the track pointer to xxxx.
  • FD xx yyyy: Loop; moves the track pointer to yyyy for xx times.
  • FE xx yyyy: Loop; same as FD, but uses different variables, permitting nested loops.
Volume/Duty Envelope Format
  • Byte pointer to the release point, relative to the beginning of the envelope, or 00 if there is none;
  • One byte for each tick, where the entire byte is written verbatim to the volume/duty register;
  • The byte 00;
  • Byte pointer to the sustain envelope's loop point, relative to the beginning of the envelope (must be positive). If this is negative, decreases the instrument volume by 1 every (xx - 80) ticks, or holds the volume if it is exactly 80;
  • If there is a release envelope, the 3 items above are repeated.
Pitch Envelope Format
  • One byte for each tick, where the most significant bit is the direction, raising the pitch if clear, and the remaining 7 bits are the offset amount for the period register;
  • The byte 00; (80 must be used for parts of the envelope with zero pitch offset)
  • Byte pointer to the envelope loop point, relative to the beginning of the envelope.
Song Format
  • Track index, may be 00 to 08, higher values have higher priority (used by sound effects);
  • Channel index, may be 00 (Pulse 1), 01 (Pulse 2), 02 (Triangle), or 03 (Noise), omitted for the DPCM track;
  • Pointer to the beginning of the track data;
  • After all track definitions are placed, the byte FF terminates the song definition.
Lookup Tables
  • $862F - $86F0: Period lookup table, starting with 97 high bytes followed by the corresponding low bytes. All period values are subtracted by 0x80.
  • $BEFB - $BF50: Song / sound effect pointer table, 43 entries.
  • $BF51 - $BFC0: DPCM assignment table, 28 entries, where each entry consists of 4 bytes; DPCM pitch, initial delta counter value, sample index (real address is C000 + xx * 40), sample length (real length is xx * 10 + 1).
  • Envelopes may appear anywhere within the music data, so every time a track needs to use a new instrument envelope it must repeat the envelope pointer.
  • There are no instrument definitions for the triangle / noise channel's percussion. They are controlled entirely by separate notes and pattern invocations.
  • Command FA does nothing if there is not a current pattern in use, which means patterns may be defined within the main track data itself, or even defined on another track (Stage 1 does this for almost the entire Pulse 1 track's echo data). Example taken from the title's triangle track:
Code: [Select]
$AD3C: 1B 01 1A 01 19 01 F1 07 FA                   [ (kick8)[o2 d+%1 d%1 c+%1 r%7]
$AD45: FB 3C AD                                       (kick8)
$AD48: F6 01 21 20 1F 1E 1D 1C 1B 1A 19 F1 F7 FA      (snare8)[L1 o2 ag+gf+fed+dc+ r L0]
$AD56: FB 3C AD                                       (kick8)
$AD59: FB 3C AD                                       (kick8)
$AD5C: FB 3C AD                                       (kick8)
$AD5F: FB 48 AD                                       (snare8)
$AD62: FB 3C AD                                       (kick8)
$AD65: FE 02 3C AD                                  ]3
$AD69: F6 01 25 24 23 22 21 20 1F 1E 1D F1 F7 FA    (tomhi8)[L1 o3 c+c<ba+ag+gf+f r L0]
$AD77: FB 69 AD                                     (tomhi8)
$AD7A: F6 01 1D 1C 1B 1A 19 18 17 16 15 F1 F7 FA    (tommid8)[L1 o2 fed+dc+c<ba+a r L0]
$AD88: FB 2E AD                                     (tomlo8) ; defined earlier
$AD8B: FB 12 AD                                     (tomhi16) ; defined earlier
$AD8E: FB 12 AD                                     (tomhi16) ; defined earlier
$AD91: F1 05                                        r16
$AD93: FB 12 AD                                     (tomhi16) ; defined earlier
$AD96: FB 20 AD                                     (tommid16) ; defined earlier
$AD99: FB 20 AD                                     (tommid16) ; defined earlier
$AD9C: FB 2E AD                                     (tomlo8) ; defined earlier
  • Command FC may point to the middle of a loop. This may be used to implement loop breaks, however it does not seem to be used on local loops, only the global loop (the last command of the title's Pulse 1 track does this, skipping over a rest on looping).

I have a trace logger Lua script specially designed for NSF files, that displays the first time each byte in the ROM is read by each song; the output looks like this (warning: large text file) on a modified JoshW NSF rip of Moon Crystal. The script saves me a lot of time figuring out the music format for any NES game whatsoever, since the timing information is included in the logs and reflects exactly what the song sounds like in virtually all cases.

The next step I do is to place a read breakpoint on any convenient music data byte, e.g. the first byte of track 3's Pulse 1 data (where it says "$000EC1:A8 first accessed on frame    0" in the log), after which the effect dispatcher can be easily found:

Code: [Select]
; $F9EC
lda ($5D), y
sta $0701
bpl @NoteCommand
inc $0708
jmp @EffectDispatch
; ...

; $FA7F
cmp #$A0
bcs :+
jmp @LengthCommand
: asl
lda $FA55, x
sta $5F
lda $FA56, x
sta $60
jmp ($5F)

; $FAD5
and #$0F
lda $FF16, x
; ...

The command lookup table resides in $FA95 - $FAD4, so bytes A0 to BF all correspond to commands (but there are duplicates); bytes 80 to 8A use a note length table at $FF16 - $FF20. It suffices to directly inspect the code for each individual command first, checking the number of bytes they consume. The log also reveals that the global song pointer table starts from $8100, and that each song definition simply consists of the four pointers to each track's data stream. Even though the effects of the commands are still unknown, there is already enough information to build a music compiler where controlled trial-and-error music data can be inserted. There is no need to edit the NSF "by hand in a hex editor" from this point.

All of this took around 20 minutes. A full analysis of the sound engine may take 2 or 3 hours. The disassembly part is unnecessary, but saves a fair chunk of time writing the interim music compiler.

A few months ago, someone asked me about the idea of making a generic ROM hacking tracker, since I already maintain one using Music Macro Language instead. I made some comments that would apply whether targeting a single sound engine or many of them:
  • Not all note lengths are necessarily representable by the sound engine, but a tracker almost always makes no assumptions about them because tracker modules trigger and halt notes using separate entities;
  • Trackers usually synchronize all the tracks available, and the ones that don't (e.g. Goat Tracker, LSDj) have a much higher difficulty curve for the end users;
  • Said desync can occur because the tempo settings can be local to each track instead of global, and some games rely on this to access more note lengths (e.g. triplets by locally tripling the tempo);
  • Said desync can also occur because many music engines have richer control flow than tracker modules, such as track-local loops with breaks, nested loops, inline patterns, and global patterns with variable length. More importantly it is difficult to represent them in conventional tracker interfaces;
  • The insertion tool must be able to represent and insert any well-formed music data; in particular, it should be able to recreate exactly the contents of the original music, except possibly for pointer locations. This is the reason such a tracker cannot optimize away the commands that desynchronize tracks;
  • Commands can take any number of parameters, but trackers rarely implement effect commands that take more than one;
  • Sometimes instruments do not exist as an abstraction at all, and different aspects are controlled by different commands, in which case the tracker ought to also be able to use each of them individually.
Those quirks do not happen with Music Macro Language dialects because they are designed to map directly to the compiled data; anything not supported by the target sound engine is simply a syntax error of the MML dialect.

Personal Projects / Re: MGCInts (MML Generic Compiler Interfaces)
« on: December 31, 2016, 09:34:38 pm »
I pushed the Journey to Silius engine definition, this is an example of what it can do (source).

It is still not exactly ready for ROM hacking because currently inserting this track would destroy the stage clear theme. Feel free to suggest other games / sound engines though.

Personal Projects / MGCInts (MML Generic Compiler Interfaces)
« on: December 20, 2016, 04:17:11 pm »
I have been making a generic MML (Music Macro Language) toolkit entirely in Lua, which allows users to write their own grammar definitions for a target sound engine. It aims to unify binary music hacking tools for almost any conceivable target, under a uniform MML syntax scheme.

MML is chosen instead of tracker formats or sequenced formats, since text files are inherently portable, and MML scripts are very transparent about the underlying data format. There is usually a one-to-one correspondence between MML streams and the data they compile to; this allows hackers to quickly inspect music data from existing ROM images (there is a utility module especially for this purpose, which reconstructs from binary data MML streams that are mostly compatible with the grammar definition it produces at the same time). Other developers may incorporate MGCInts into their toolchain and write specialized converters from other formats to MML.

Currently, it supports the following games: (see here for more details)
  • Castlevania (NES / NSF; Example)
  • Mega Man 3 (NSF)
  • Zelda II: The Adventure of Link (NSF, not published yet; Example)
  • Shatterhand (NSF, not published yet)
  • Journey to Silius (NSF, not published yet)
  • Super Robot Wars Compact 2 (WS, not published yet)
This project is licensed under the Mozilla Public License Version 2.0. It is still in very early alpha; it does not support multisong insertion yet (which means anything compiled may corrupt other music data), most engines only support JoshW NSF files, and there are almost zero examples for both the sound engines and the framework other than the source code itself. Stay tuned for updates in the coming months or, if you cannot wait to try out the MML compiler, report issues at my own bug tracker.

Download: (download as ZIP from the master branch or the Releases tab)

Newcomer's Board / Re: Introduction Topic
« on: November 23, 2016, 03:39:20 pm »
Hello RHDN!

I make chiptune music chiefly on a non-hacking-related tracker fork I maintained. I am well-versed in Lua / C++ / 6502, and can also read x86 and Z80 disassembly. My aim here is to make music hacking accessible to the general ROM hacking community.

Pages: [1]