Super Pitfall NES (improvements)

[Vanya]

Actually I was thinking of a Pitfall 2 themed version of SMB1.
And maybe an even simpler one as a hack of Dr. Mario.
Dr. Harry: Beyond the Jungle if you will.

[John Enigma]

The new adventure patch will include all patches on it.
The graphics patch will include the fixes patch.
The game design patch will include the fixes patch.
And there's the standalone fixes patch.

I figure that some people may not want to change the game so much. I might merge the graphics and game design into a single patch but I'm not sure yet.

But my question is, when are you planning on releasing it.

[jonk]

2- Sometimes Harry will simply slip from ladders for no reason. This may sound like in super smash bros brawl with the tripping, but I'll consider it a bug and try to fix it.

I have a random guess to offer, given some of the code I've recently read (for example, code that calls a multiplication routine to multiply by 10, then calls a division routine to divide by 20, when a simple instruction would have done the same work -- what a waste.) It may have to do with polling (clocking/strobing) the serialized controller, with widely varying times between polling, and how that relates to the timers being used elsewhere. Just a wild guess, though. But "slip for no reason" may be about what you perceive vs what the software perceives.

Glad to hear you are digging into code writing. Misery loves company. 

[nesrocks]

Great stuff, you sure do know your way around. Are you using fceux's debugger or did you disassemble it somehow?

Here are tasvideos Alyosha's notes on the matter:

"So far what I found is that the glitch appears to be a bit of poor programming combined with a bit of lucky interrupt timing.

Here is the code where everything goes wrong:


$FEA4:85 D5 STA $00D5 = #$94 A:06 X:02 Y:05 S:EC P:nvUBdIZC
.
.
.
$C298:40 RTI A:01 X:00 Y:3D S:7A P:nvUBdIzC
.
.
.

$FD6D:A5 D5 LDA $00D5 = #$06 A:3D X:00 Y:3D S:7D P:nvUbdIzc
$FD6F:60 RTS (from $FDD3) --------------------------- A:06 X:00 Y:3D S:7D P:nvUbdIzc
$EAE9:C9 72 CMP #$72 A:06 X:00 Y:3D S:7F P:nvUbdIzc
$EAEB:B0 21 BCS $EB0E A:06 X:00 Y:3D S:7F P:NvUbdIzc


The problem is that normally, $00D5 is $#94 and the CMP #$72 fails. However, the interrupt call set $00D5 to #$06, so it passes. From there it goes on to increase the players x position. The code seems to reuse a lot of these addresses several times per frame, so its not really surprising something like this comes up. "

So what I did to fix it is use another address instead of D5. Temporarily it works but I need to find an address that is never used for sure throughout the whole game. Or maybe a better solution.
edit: I think that simply removing that STA to D5 on FEA4 fixes it. It seems to be a deliberate feature and they knew it would sometimes pass so he falls. The problem is that he also falls through floors sometimes and goes through walls because of this. So it's better without.

[jonk]

Are you using fceux's debugger or did you disassemble it somehow?

Were you referring to me? Or someone else? I can't tell.

[nesrocks]

Were you referring to me? Or someone else? I can't tell.

Sorry, yes, you. That was quick how you noticed those things. I wonder how much could be optimized in this game. It took me a long time but even a newbie to assembly like me could notice some stuff arent great in the code.

[jonk]

Sorry, yes, you. That was quick how you noticed those things. I wonder how much could be optimized in this game. It took me a long time but even a newbie to assembly like me could notice some stuff arent great in the code.

Hehe. I was just making a random guess based upon decades of experience and ZERO knowledge about hacking and/or writing games. It's just where I'd go look if I saw the behavior you mentioned.

Given what little I've learned from a few hours of reading code (that's the total sum of my experience on this subject), it just popped to mind. The code generation is horrible, from what tiny experience I've had reading code so far.

It is pretty clear that a C compiler was involved. (That's easy to spot, if you've ever written a C compiler yourself. It's easy to spot even if you haven't.) And it was clearly a very terrible quality C compiler. Truly and unbelievably bad. I didn't know anyone wrote C compilers that primitive, except perhaps as a class final in college. It did nothing at all more than simply do a direct conversion, without any optimizing analysis I can find. Zero peep-hole analysis, and not even the simplest of constant folding? I'm still kind of amazed at how little it did. Shocked, actually. Even the open-source and freely available SDCC for the 8051 does so much more, including odd-ball things like call-frame static overlaying to save on stack usage. The compiler clearly did support all the C features, though. I did see place where an array of function pointers was accessed, and because the 65c816/6502 doesn't have any instructions that can be used to call a function via a register value, they had to do some interesting stack manipulation together with an RTS in order to make the calls work out. So the C compiler wasn't entirely brain-dead. But just about.

What parly annoys me about that is that I'm also getting the sense (from responses here and elsewhere) that there is insufficient interest and/or pressure from compiler consumers. So I'm curious what the current spate of compilers actually handle, today. CC65 and snes-sdk have both been mentioned to me and I need to look them over. (Though someone said that CC65 generates 6502, not 65c816, code.) Back when game development was active, when professional teams were writing SNES game code, when they were paying a decent price for access and compiler tools, and when getting good code generation might really make a difference as to whether or not the resulting product was competitive enough to survive, .... well, I just can't explain what I'm seeing in this ROM! If all that wasn't enough to get a decent C compiler back then, what possible hope is there today? A hope is that good people used gcc's source and did a yeoman's job of shoveling in 65c816 knowledge into the code generation. But then, they'd need to add call branch analysis, which so far as I'm aware no one has added to gcc for any reason, to get really good code generation. (Call stack analysis is more useful where recursion isn't an issue.) There are a lot of specialized techniques, available in quality free and professional compilers for other microcontroller cores, that would need to be worked out in gcc. I wonder if anyone would go there. Another hope is that someone wrote one from scratch and applied themselves well to the task.

I'm just not yet sufficiently motivated to go look, just yet. I'm spending what time I have on assembly and helping my son (where he wants to accept it) with his current project. I will go look, someday, though. But for now, what I see in the game code is terrible and I just cannot explain it with any reasonable logic. No sane development team would ACCEPT this crap I see. Not unless they had NO OTHER CHOICE, at all. And even then? I might simply refuse to use my capital to develop a game, if Nintendo forced me to use C compilers this badly done. I would feel that Nintendo might be putting me and my team "at risk" in competing with other developers who may be using assembly-only for their development. The problems you've identified are an example of how my product might fail in the market and do so partly because Nintendo forced a crappy C compiler onto their developers wanting to use C. I don't like it when they don't care enough to deal with the basics, like that. It's not as though Nintendo didn't have enough time. The SNES didn't arrive in a vacuum. There was the NES before it. So they had plenty of time and reason to do up a decent C compiler.

Anyway, I've a lot yet to observe here. A few hours' effort in no way makes me any kind of expert. But the little bit I've seen looks scary. And I also know just how low the skill levels are for many embedded developers who'd be using the C compiler as a tool to generate code. I've met such folks, worked with them, been able to measure how few are truly good at their work and care about it deeply, and have seen how barely most of them manage to "scrape by" and how often they don't and, as a result, seen how it is that companies not infrequently lose huge investments in new product development because they hired people who had very thin levels of skills and background. (It's one of the ways I market myself, in fact.)

But it seemed pretty obvious from your description where I'd start looking. I gather from your post that others have thought similarly, too. I guess that suggests my instinct formed out of a few hours of looking around here might have been close to the mark.

[KingMike]

Is it really the compiler, or was it the developers?
(turns out a LOT of NES developed was contracted out to unnamed ghost developers like TOSE and Micronics. Not sure of the developer competency, but they probably also had so much on their plate to really care to do better. Didn't they probably alone count for like 10% of the NES/FC's massive library?)

[jonk]

Is it really the compiler, or was it the developers?
(turns out a LOT of NES developed was contracted out to unnamed ghost developers like TOSE and Micronics. Not sure of the developer competency, but they probably also had so much on their plate to really care to do better. Didn't they probably alone count for like 10% of the NES/FC's massive library?)

I have a lot of experience with C compilers, having written one and having been involved in working with other compiler writers on their commercial products, and I can recognize the difference between hand-coded assembly and C generated code. I'm sure I've also come across bad assembly coding, but I see that all the time anyway and so it didn't catch my eye at the time. What I've recently seen, and was discussing before, is clearly the result of a uniquely lousy C compiler. The rest you suggest I cannot address well, except to say that it doesn't change my earlier comments, so far as I can tell right now.

[skyrunner14]

Dang, this is all really nice! As bad as it is, I've always had a soft spot for Super Pitfall, so I'm definitely looking forward to seeing this get finished. Do you intend on compiling all the unused stuff you've found onto The Cutting Room Floor?

[John Enigma]

^I don't think The Cutting Room Floor even has a Super Pitfall entry.

Hopefully that'll be changed.

[nesrocks]

jonk: Just some clarification, it seems that your comments were regarding the super pitfalll rom, correct? That game was made for the NES. I got the feeling you had the impression it was for the SNES. Also, it is funny that you said the compiler looks like something made by college students because there's a rumor that micronics was in fact comprised of fresh graduates or something similar. Finally, I believe nintendo didn't have anything to do with this game other than authorizing its publication (but they didn't care a lot about the code and more about the content being on par with their "kids friendly" policies). Nintendo's own games had well written code, but they didn't care how other developers made their games, it seems.

I think the sound engine in super pitfall may be very, very badly optimized (or more probably not at all). I have kind of turned it off by changing a loop to 2 cycles instead of $2F and the game runs a lot faster with no side effects besides having no sound, so I feel like a great deal of perfomance boost would be possible to achieve by hacking this game. My skills are just not there yet.

skyrunner14: yes, sure. I'd like to compile a full map of the unused rooms that are obviously older versions of the used rooms in the main map but I lack the skills. It may be possible in the future if we move forward with the map editor (or painstakingly do it by hand, which I can't do right now, I'll focus on the hack for now). There are also older versions of the normal enemies' sprites that have less cartoony eyes, graphics of spiders moving vertically up walls, unused hud icons, and code for enemies and items that I couldn't exactly determine what they were, except for an object that goes up and down that is clearly a spider hanging from the ceiling like in the PC-88 version. I will certainly not cover everything because I'm not that proficient with assembly, but I'll be able to cover a good amount of content right off the bat.

[jonk]

jonk: Just some clarification, it seems that your comments were regarding the super pitfalll rom, correct?

No. Not correct. I'm looking at SNES ROMs and talking about those observations. The rest therefore doesn't follow, I think. (I intend to look at super pitfall, but haven't done so yet.)

[John Enigma]

^The only Pitfall game for the Super NES that exists, is the sequel Pitfall: The Mayan Adventures, and that game has less bugs compared to Super Pitfall.

[nesrocks]

No. Not correct. I'm looking at SNES ROMs and talking about those observations. The rest therefore doesn't follow, I think. (I intend to look at super pitfall, but haven't done so yet.)

My mistake then. Was the game Dragon Quest 3 for the SNES?

[jonk]

^The only Pitfall game for the Super NES that exists, is the sequel Pitfall: The Mayan Adventures, and that game has less bugs compared to Super Pitfall.

That's also not what I was talking about.

I understand the source of the confusion, since I've been discussing the possibility of helping write hopefully useful code with nesrocks in a different thread and, as a consequence of my asking for input there, nesrocks brought up an idea that involved super pitfall for the NES. To me, that is an entirely different discussion and has little or nothing (so far as I'm aware) to do with my comment here in this thread where I made a completely random guess, trying to offer a thought of help when nesrocks was also discussing some issues on his current project, NES super pitfall. I said it was random and a guess, because I wanted to be completely clear about the fact that I had NO facts at all to support it except for long experience doing embedded programming work (itself also completely independent of super pitfall and any NES code.) I guess I could have made my later, longer response to nesrocks still longer in order to add what I'm writing here.. but it just didn't seem necessary or appropriate at the time I wrote it. I either need to learn to write more, or less, I suppose.

I haven't looked at super pitfall OR the mayan adventures follow-on for the SNES. In fact, I didn't even know there was one. I based my comments upon reading other, as yet unnamed SNES games over this last week. I didn't intent to state what they were, either. It doesn't matter. I have had some experience reading SNES code over the last week or so. That experience has taught me that a really poor quality C compiler was in use by significant game companies. But also, that experience had nothing whatever to do with the NES or any discussion here. It was only that, given what I saw there for the SNES recently, and assuming that I could "generalize" (or perhaps better, "project") a little bit towards the NES, then I simply hoped I may have had a small thought to offer nesrocks. It was a long-shot and I had no idea if nesrocks would understand it, or care about it, or agree or disagree with it, or anything else really. It was just a thought and I offered it to see if it might jog something in his head. His response was interesting to me and I decided to offer another thought about my very modest experiences looking at SNES code generation.

I'm not even sure what my expectations should be, if I someday decide to look at some NES ROMs (such as super pitfall.) It's possible that a similarly terrible C compiler was used there, too. But part of me also thinks that fewer game companies would have used C, at all, preferring instead to stay with assembly coders writing assembly code at that time. I have no reason to believe that the NES game system development world, coming out of a different time and place, would look exactly like the SNES game system development world. Again, I'm really ignorant about all this, so I have to remain cautious about projecting a week of experience too far. Again, this is why I said I have a random guess and nothing more than that, trying to offer a thought to nesrocks here.

I hope the above clarifies where I'm coming from and what I meant earlier.
May 09, 2016, 01:41:42 PM - (Auto Merged - Double Posts are not allowed before 7 days.)

My mistake then. Was the game Dragon Quest 3 for the SNES?

Assuming you are talking about my comments regarding the code I've seen recently and the comments I made in this thread about C compilers for the SNES, then that would be "'Romance of the Three Kingdoms III,' to name a truly bad case I recall." Also SNES, from Koei Co, Ltd.

[AWJ]

Interrupt-related bugs like the described Super Pitfall one are pretty common in console games, even highly-regarded ones created by experienced developers. Mega Man 2 (NES) has a bug that can trigger when enough objects are onscreen to cause slowdown, that allows you to advance by falling into pits (instead of dying) or to warp out of a boss room into a different stage. Dragon Quest 6 (SNES) has a bug that destroys the state of the game's pseudorandom number generator if the NMI triggers at exactly the right time when the PRNG is in exactly the right state; the most obvious visible consequence of the bug is that all attacks by player characters become critical hits until you reset the SNES.

(If you're curious, compare the PRNG and NMI code in DQ6 and DQ3 SNES; DQ3 was based on DQ6's codebase but contains a fix for the PRNG-trashing bug)

[jonk]

Interrupt-related bugs like the described Super Pitfall one are pretty common in console games, even highly-regarded ones created by experienced developers. <snip of interesting notes about some games>

I can't speak to that, as I was (1) merely trying to suggest a random guess (I am largely ignorant about NES) to nesrocks; and, (2) expanding later upon a part of my reasoning.

Now, if you are excusing the creation (by 'I don't know who') of very poor C compilers for SNES development, or are excusing the use of such tools without proper evaluation by so-called 'experienced developers,' or are excusing experienced game developers when creating interrupt vs polling timing behavior issues in embedded game development, then that is a different story. I'd have additional comments to add. But since I can't tell if any of those excuses of outcomes happen to be your intent, I won't add them. I'll just assume we are talking at cross-purposes and leave it at that.

[AWJ]

I think we are talking at cross-purposes. My point was that non-interrupt-safe code isn't at all limited to games written in compiled languages (DQ6 and Mega Man are both pretty definitely hand-written assembly language) and it isn't just outsourced shovelware games that have bugs.

[jonk]

I think we are talking at cross-purposes. My point was that non-interrupt-safe code isn't at all limited to games written in compiled languages (DQ6 and Mega Man are both pretty definitely hand-written assembly language) and it isn't just outsourced shovelware games that have bugs.

On that point, I agree with you.