Emu Accuracy analysis: FCEUX vs. NEStopia

[Disch]

obscurumlux01 in another thread was curious about accuracy comparisons between FCEUX and NEStopia.

The link is here:
http://www.romhacking.net/forum/index.php/topic,20530.msg288751.html#msg288751

In my reply I made some bold claims and pretty much said "zomg NEStopia is the greatest thing ever omgomgomg", but I realized that opinion came largely from the state of the emulators ~7 years ago.  In that time, FCEUX has been in active development, while NEStopia has largely been idle.

So did FCEUX catch up?  Was I talking out of my ass?  I decided to take a look into a lot of "gotchas" that plague NES emus which try to cut corners on accuracy.

This is far from an exhaustive list, and I am grossly oversimplifying by just assigning points to individual aspects.  But this might be fun and interesting nonetheless.


In this post, I am comparing Win versions of FCEUX 2.2.2 and NEStopia 1.40, which to my knowledge is the latest official release of each emu.

DMC IRQs and DMC Timing

Reference:
http://wiki.nesdev.com/w/index.php/APU_DMC#Memory_reader

DMC timing is very tricky, and a handful of Camerica games rely on reasonably strict DMC IRQ timing for split screen effects in some games, specifically Fire Hawk and the dreaded Mig-29 Soviet Fighter (one of the harder games to get working)

Both emus run the games correctly, but looking at the source, FCEUX cheeses it with a hack.  Rather than emulating the buffered sample fetch, it runs the DMC ahead a bit, then backtracks after the IRQ fires:

FCEUX | sound.cpp | 564

Code Select Expand


   /* Unbelievably ugly hack */
   if(FSettings.SndRate)
   {
    soundtsoffs+=DMCacc;
    DoPCM();
    soundtsoffs-=DMCacc;
   }

(I did not add that comment, that is actually in the source)

FCEUX also does not appear to halt the CPU (or "steal cycles") for the DMA performed by the DMC.  NEStopia on the other hand, not only steals cycles, but properly steals a variable amount depending on the CPU state at the time of the DMA, in addition to properly buffering the DMA'd value:

NEStopia | NstApu.cpp | 2113

Code Select Expand


if (!readAddress)
{
cpu.StealCycles( cpu.GetClock(cpu.IsWriteCycle(clock) ? 2 : 3) );
}
else if (cpu.GetCycles() != clock)
{
cpu.StealCycles( cpu.GetClock(3) );
}
            ...
dma.buffer = cpu.Peek( dma.address );
cpu.StealCycles( cpu.GetClock() );
dma.address = 0x8000 | ((dma.address + 1U) & 0x7FFF);
dma.buffered = true;



I'm going to give NEStopia 2 points for this, since IRQ timing and cycle stealing are two different things.

2 Points go to NEStopia


OAM Reads during Rendering

Reference:
http://wiki.nesdev.com/w/index.php/PPU_sprite_evaluation

This is an edge case exposed by the game Micro Machines.  I touched on this in the original thread.

Reading from $2004 during rendering exposes the internal sprite fetches.  Micro Machines abuses this to find HBlank in order to do raster tricks.  FCEUX clearly glitches due to incorrect behavior here (screen shakes at the character selection screen)... but only when "old ppu" is selected from the Config menu.  When "new ppu (slow!)" is selected, the shaking goes away.

Looking at the source, "old ppu" FCEUX doesn't even try to emulate this behavior:
FCEUX | ppu.cpp | 648

Code Select Expand


} else {
FCEUPPU_LineUpdate();
return PPUGenLatch;
}


HOWEVER... "new ppu" FCEUX does in fact emulate this behavior.  It's on line 526 of ppu.cpp, and I won't paste the code here since it's massive.


NEStopia, while it plays the game properly, appears to only be emulating the behavior half-way:

NEStopia | NstPpu.cpp | 870

Code Select Expand


if (!(regs.ctrl[1] & Regs::CTRL1_BG_SP_ENABLED) || cpu.GetCycles() - (cpu.GetFrameCycles() - (341 * 241) * cycles.one) >= (341 * 240) * cycles.one)
{
io.latch = oam.ram[regs.oam];
}


This is almost what it should be doing -- but isn't quite right.  It's not exposing internal fetches as much as it's just exposing internal RAM.  I don't know if I'd call this a hack, as much as I'd call it a misrepresentation of the behavior.  Though ultimately is appears to be a fluke that Micro Machines is working.


Since "old ppu" is the default in FCEUX, I don't know how many people actually use the new ppu.  But the emulation is there for anyone that wants it --- just have to flip a switch.

So yeah, NEStopia kind of fucks this up, and FCEUX does it right.   Point goes to FCEUX



PPU Reads during Rendering

Reference:
http://forums.nesdev.com/viewtopic.php?p=62827#p62827
http://wiki.nesdev.com/w/index.php/Reading_2007_during_rendering#Output

Another edge case exposed by The Young Indiana Jones Chronicles.  I have no idea why, but the game apparently reads from the PPU I/O register ($2007) during rendering, which mangles the PPU address.  This is one of the more recent discoveries and was discovered after NEStopia 1.40 was released, so not surprisingly, NEStopia fails and glitches up during this game (near the end of level 1, when a cannonball hits the ground).

FCEUX, somewhat surprisingly, emulates the game just fine without any glitching.


NEStopia source indicates that it does not treat in-rendering PPU reads any different than out-of-rendering
NEStopia | NstPpu.cpp | 967

Code Select Expand


NES_PEEK_A(Ppu,2007)
{
Update( cycles.one, address );

NST_VERIFY( IsDead() );

address = scroll.address & 0x3FFF;
scroll.address = (scroll.address + ((regs.ctrl[0] & Regs::CTRL0_INC32) ? 32 : 1)) & 0x7FFF;
UpdateScrollAddressLine();

io.latch = (address & 0x3F00) != 0x3F00 ? io.buffer : palette.ram[address & 0x1F] & Coloring();
io.buffer = (address >= 0x2000 ? nmt.FetchName( address ) : chr.FetchPattern( address ));

return io.latch;
}


'IsDead' checks to see if rendering is disabled.  NST_VERIFY just seems to log something -- it doesn't actually change flow.  Proper behavior here would be to put everything after the NST_VERIFY line in a 'if( IsDead )' block, and 'else' that with code addressing the weird behavior.


FCEUX is employing a hack... sort of.  It's questionable to say the least:

FCEUX | ppu.cpp | 243

Code Select Expand


if (rendering)
{
//don't do this:
//if (by32) increment_vs();
//else increment_hsc();
//do this instead:
increment_vs();  //yes, even if we're moving by 32
return;
}


FCEUX, unlike NEStopia, does in fact acknowledge that behavior is different when rendering, but it doesn't even try to get the proper behavior right.  The logic here is a lot more complex than simply incrementing the PPU addr by 32 (as illustrated on the wiki page http://wiki.nesdev.com/w/index.php/Reading_2007_during_rendering#Output ).  So it's kind of a fluke that this works at all.

It would be interesting to date this code to see if it was written before or after the post on the nesdev forums outlining this behavior.  But just from looking at it, I can't help but think the developer just said "whatever, this fixes that game so it's good enough".

On one hand you could argue this is a hack.  On the other hand you could argue it's just incomplete.  I'll be kind and give FCEUX devs the benefit of the doubt and just say it's incomplete.  And FCEUX does in fact run the game where NEStopia chokes.  So while neither emu does it right, FCEUX at least sort of does it.

Point goes to FCEUX



Rendering when Rendering is disabled

(Can't find a reference for this on the wiki -- sorry)

When rendering is disabled, the PPU will usually draw the backdrop color ($3F00) to the screen.  However, if the PPU address lies in palette space, then whatever palette entry it points to will be drawn instead.

Micro Machines relies on this behavior to draw bars with a fancy gradient.  FCEUX does not draw these bars properly, and also has splashes of black scanlines on the title screen that shouldn't be there.  NEStopia properly renders the game without issue.

In the source, NEStopia is properly checking to see if the address lies in palette space, and uses the pointed to value if so -- otherwise defaults to the backdrop.

NEStopia | NstPpu.cpp | 2627

Code Select Expand


HActiveOff:
{
const uint pixel = output.palette[(scroll.address & 0x3F00) == 0x3F00 ? (scroll.address & 0x001F) : 0];


To be honest, I am having all sorts of difficulty finding the relevent code in FCEUX.  The only rendering code I can find makes no distinction between whether or not rendering is enabled -- which I assume means the emu checks that before the routine is called, but I can't find where it actually does that -- the rendering code is all over the place and pretty unorganized.  I've been looking for ~20 minutes and I can't find it.


Regardless, NEStopia is doing it right, and is emulating the game properly.  FCEUX has glitches, so whatever it's doing, it must be doing it wrong.

Point goes to NEStopia




I think that leaves it at NEStopia: 3, FCEUX: 2.

There are other things I want to check but I need a break.  So I'm going to stop for the time being.  I'll come back to this later.

[Pennywise]

I thought this was a moot point because puNES is the most accurate NES emulator. Also, I think Bizhawk's NES emulation core is also better than FCEUX.

[Disch]

Shows how long I've been gone.  I don't know anything about puNES.


EDIT:

Yeah puNES pretty much looks like the Bee's Knees.

[Chpexo]

.

[dougeff]

Also (I think), neither one uses all of MMC5 features correctly. Though I can't think of any games this effects.

[Jorpho]

The most obscure thing I can think of is a glitch used in a Mega Man TAS for skipping a stage that apparently depends on very specific PPU behavior.

DelayStageClear is impossible with NewPPU of FCEUX,debbuger of Nintendulator but DelayStageClear is possible with OldPPU of FCEUX, Nestopia, Nintendulator. Mr.Inzult tested this by using PowerPak with a real NES. In a word, DelayStageClear in the BombManStage is possible also in real NES! (OldPPU of FCEUX is proven to be correct.)

http://tasvideos.org/2921S.html

[obscurumlux01]

I love this thread so much right now.
Disch you don't need to dig in further if you don't feel like it (unless you actually want to do so).
I greatly appreciate the references and thorough explanations of things.

To be honest, it will take me a few days to really parse things and read up enough on it to understand the coding aspect of it (this is why I'm not a programmer heh).  Those references will help ^_^
But I can understand the general logic of it even if the exact syntax stuff escapes me.

So since others have chimed in, how about a reasonable consensus?  What is the most accurate NES emulator out right now and how well does it perform?
If Nestopia isn't the best of the best anymore then I'm still waiting for an answer on 'best NES emulator'  ^_^

[Disch]

Initial impressions of puNES is that it is exemplary.

Checking the source for puNES would be difficult, as it's not in English  =(

[obscurumlux01]

I'll give puNES a look.
What parts of the source aren't in English?  I looked through briefly and couldn't find non-english other than the descriptions on the GitHub page (which aren't part of the source code itself but just descriptions).

[Disch]

Bah.  I guess the bulk of it is in English.  Maybe I just saw the github comments, as well as in-code comments and it messed with my brain.

Yeah the actual function and var names do seem to be in English.

[obscurumlux01]

Just a quick update.
Found some test results for various emulators here along with full technical details on what tests they pass/fail and why.

Would love to hear feedback and thoughts on these results and what they mean for NES/Famicom emulation going forward.

[chillyfeez]

Has anyone ever done this sort of analysis with SNES emulators?
I recently discovered a bit of glitchiness in my project (it has to do with dialogue boxes in FFIV), but the manifestation of the glitch varies depending on the emulator. I've figured out ways to mitigate the issue, but it would help my peace of mind to know that what I'm doing to "fix" (hide) it would translate to actual SNES hardware (as I don't currently have the means to test it myself on actual SNES hardware).

[Jorpho]

but the manifestation of the glitch varies depending on the emulator.

The general consensus seems to be that if you're seeing it in ZSNES you should probably forget about it.

[obscurumlux01]

Well maybe we should rename the thread to just be a general 'emu accuracy analysis' thread
While the TAS (tool assisted speedrun) community prefers a particular emulator for the tools it provides to assist in speedruns, that doesn't mean they always choose the most accurate emualtors.

For SNES, the primary information I could find seems to be Higan as high-accuracy and cycle-accurate, then SNES9X being medium accuracy and tolerable, then everything else either being equal to SNES9X or worse.  ZSNES is so bad that almost everyone has already migrated away from using it as a standard in their ROM hacks.  RetroArch using the bsnes accuracy/balanced/performance cores helps to mitigate usability problems and let people just enjoy the games without messing with importing and libraries and crap.

The sad thing is that a high-profile ROM hack like Crimson Echoes mentions 'this plays best on ZSNES' despite that no longer being true for over 2 years now.  CE has been fixed to no longer require the buggy sound core of ZSNES to play songs properly and yet it still has that message.

For the Sega Systems, the closed-source Kega Fusion still wins by leaps and bounds over others (plus native 32x support that is better than any other).  Others come close but Kega Fusion is the only one so far that has everything in one go.

The RetroArch information is the most comprehensive that I could find, but keep in mind that it is outdated by at least a year (if not longer) due to the progression of emulators and technology.  Maybe we need to crowdfund a comprehensive emulator analysis and comparison to the actual NES system (with physical non-flash NES test cartridges for comparison).

[chillyfeez]

Cool. Thanks!

[Dr. Floppy]

Maybe we need to crowdfund a comprehensive emulator analysis and comparison to the actual NES system (with physical non-flash NES test cartridges for comparison).

Is there a crowdfund for a FCEUX/xD SP-quality debugging utility for SNES?

(I'll volunteer as the crowd.)

[Rodimus Primal]

I've been told PUNes was the most accurate, but I wonder if accurate also means having bad input lag. Nestopia doesn't give me the same issues input wise.

Higan's GUI and filing system bothered me so bad I went SNES9x and never went back. That and ZSNES has been surpassed for years now.

I don't care for the fuzzy screen on Kega, but it works and works well.

[obscurumlux01]

I've been told PUNes was the most accurate, but I wonder if accurate also means having bad input lag. Nestopia doesn't give me the same issues input wise.
Higan's GUI and filing system bothered me so bad I went SNES9x and never went back. That and ZSNES has been surpassed for years now.
I don't care for the fuzzy screen on Kega, but it works and works well.

I've seen that puNES has one fatal flaw (in my eyes) which is the inability to PAUSE a currently emulated game.  There's no pause function!  Only power (hard reset) and reset (soft reset) are available.

On the audio front, I'm happy to report that we now have emulated STEREO SOUND for the NES.  It sounds AMAZING!  I use Ninja Gaiden as an example and holy cow.  It really is like how it feels when putting on surround sound headphones for the first time.  Just WOW.
I say that puNES seems to have the most accurate and pitch-perfect audio and audio sync of any emulator I've ever experienced!
I can say this with 99.9% confidence for Ninja Gaiden 1, Final Fantasy 1, Super Mario Bros 1/2/3, Contra, SuperC, and The Legend of Zelda.
Those are the games I played in my early childhood for a combined total of hundreds of hours (especially FF1 and SMB3 in the pre-internet days).
I clearly still remember the tunes; some say I have an ear for music. 

As for video, I've had no issues with it so far even though the lack of documentation lead to lots of experimenting with how things work.  The video options include a wide variety of filters, an interpolation toggle (doesn't say what kind of visual interpolation is used), and under 'Pixel Aspect Ratio' it has an 'HLSL soft stratch' toggle option.  From what I can tell, both the interpolation and HLSL soft stretch options just make things more blurred to mitigate jaggy pixels.  When sitting about one or two meters away from the screen, the effect is very nice and still affords decent readability of in-game text.

I've had zero input lag with puNES and this is with both Ninja Gaiden 1 and Megaman 2.  If you know those games you know that precise input is absolutely critical.  Zero issues (although I do play with a gamepad so that may be something).  You have to remember that you are meant to play emulated games with a gamepad rather than on the keyboard.  Most consumer-quality and non-mechanical keyboards are not designed for gaming and you run into the often-complained FAQ regarding Chrono Trigger and certain button combinations to open a door or passageway.

SNES9X is fantastic but still uses many game-specific hacks to get things to work; ZSNES is still piss-poor by comparison as far as emulation quality, updating, and accuracy.  The difference is SNES9X team has accepted accuracy-related fixes from byuu and other contributors while ZSNES has declined such improvements and their GitHub has been completely stagnant for a while.  They are 'working' on a major revamp update but have given absolutely no indication or public information about it outside of a post or two on their forums.
I also hated the Higan folder-based system and saw that RetroArch presumably has a Higan-based setup WITHOUT the folder crap.  You could possibly give that a try if you want to ^_^

KEGA Fusion has a fuzzy screen?  What!?  Have you looked through the options?

Go to your Video menu and do the following:
-No Scanlines
-Normal unfiltered output
-Fixed Aspect Ratio (Fit)
-Nearest Multiple (optional but recommended)
-Uncheck the 'Filtered' option (Filtered = Interpolation = Fuzzy/Blurry screen)
-Brighten (optional but recommended)
-Use NTSC Aspect (optional but highly recommended)

Hope that helps!

[Rodimus Primal]

I agree with you that the sound is better on puNES. The input lag I get is from Super Mario Bros 1 and 3. I play with a PS2 controller plugged into my PC via USB and Nestopia handles it well. But puNES I notice the jumps take a split second longer to register and I get killed even in 1-1!

For the longest time I was using bsnes and I even took the time to take screenshots of the titlescreens, make folders ending in .sfc and placing the ROMs, save files, states, etc in them. Then I ran into a major bug with Donkey Kong Country (1.0). I had an .smc file that snespurify converted to .sfc and it created a bug. If you used the BARRAL code and then get hit by an enemy, the screen went blank. I never liked the newer versions of bsnes, now Higan, and the straw that broke the camel's back was the fact that the new library system COPIES your Roms and creates its own folder system seperate from your existing ROMS. Granted that in today's day and age hard drive space is not really a concern but its still a waste of space. I took all of my ROM files out of that crappy folder system, made my own system and went to using SNES9x.

What I mean in Kega Fusion is the screen when you first load it up. It's like turning on an old TV to channel 3 with the scrambled white snow. It's cool the first time you see it, but its annoying the more you use the emulator.

[KingMike]

I couldn't reproduce the DKC error (using higan 094 accuracy).
Maybe it was just your ROM.
All snespurify is doing really is removing the header (if present) and changing the extension. Maybe deinterleaving but I suppose interleaving is technically a bad dump in the first place.