News: 11 March 2016 - Forum Rules

Author Topic: ROM archives of the world are being DESTROYED! (see page 2)  (Read 23051 times)

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #20 on: February 07, 2013, 06:41:21 am »
Except that "clean rom" doesn't say anything about headers. It means an unaltered rom, no previous patching and no random corruption.

FAST6191

  • Hero Member
  • *****
  • Posts: 3341
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #21 on: February 07, 2013, 09:06:03 am »
we have different entities (ROM providers, patchers, hackers, flash-cart producers, mapper makers, etc), and each is producing something which should be 100% compatible with another

As it see it SNES emulation is moving to bigger and better things with the likes of bsnes/higan and the header stuff was largely a leftover from the bad old/bodge it and scarper days of the SNES "scene"- from the GBA upwards there were proper standards of a sort and people did follow them but the SNES was nuts/a proverbial wild west which has seen a hangover last to this day. Such transitions are often painful but I would argue quite necessary. At least here nothing will be lost that can not be recovered with minimal effort so that is not such a problem (depending upon how you view a handful of tools).

Also "I assumed".... historically not a great start to an endeavour.

Re extensions. Something starts trying to associate all the system type ".bin" files and I say things.
Also because "ren *.bin *.md" is hard? There are a few other workarounds as well (*.* in the name section and many exe files as far as changing supported extensions are not hard to edit either).

Anyway the worst I usually have to deal with is an intro on a GBA game (there were technically a few with dodgy headers but if you can find one of those now you are doing well or you were there at the time- certainly they did not make it into the "clean" sets) or a header added to a save file (save file containers were not a thing on the GBA and DS and so they get added for good reason by several emulators, launching programs and commercial dumping tools).

KingMike

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 7253
  • *sigh* A changed avatar. Big deal.
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #22 on: February 07, 2013, 09:20:46 am »
For SNES patches, if a patch requires a header, there are tools to add/remove a header from the ROM. So you can add a header, apply the patch (and then it is suggested to remove the header afterwards).
Since PCE headers (seemingly rarely present) are the same size as SNES, you could use the same tools to add them (but remove them after patching, as the data is almost certainly not the same format, if emulators even use them).
"My watch says 30 chickens" Google, 2018

Zoinkity

  • Hero Member
  • *****
  • Posts: 565
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #23 on: February 07, 2013, 08:03:25 pm »
From the perspective of patch creators there's another really important point in removing headers during their work: pointers to data based on physical file position.  If you didn't remove the header you have to subtract the difference from every pointer you're working with, which is an added layer of needless work.

So, unless they use an editor or something high-level to completely avoid working on the hex itself, you're looking at them having to add a header back on before generating a patch.  Plus, even from the beginning not every ROM had a header, so making a patch for a headerless version still happened.

The one problem--the pivotal problem--is that many patch creators never tell you the expected format.  :banghead:  If they use a tool they may have never known what it was!

It's always been a mess.  I can understand forcing standardization for systems with pointless headers, at least in the long run.  What's irksome is they obviously didn't bother to find out which systems those are and removed them for everything.

ChronoMoogle

  • Sr. Member
  • ****
  • Posts: 358
  • Super Famicom all the way.
    • View Profile
    • SNESfreaks
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #24 on: February 07, 2013, 08:36:08 pm »
I know there is quite a rejection of the existence of snes headers, but please keep in mind that satellaview headers are often mistook to be snes headers (and I didn't even see one mention of them here after my post) but they are very much needed to let the roms be functional. Just wanted to point that out again. Kiddo wanted to create a isolated satellaview set, so I guess this is very needed if you see the general opinion about SNES roms. Esspeacialy if you keep in mind that most of those games havent been hacked or emulated so far. (some emulators and the sd2snes may be an exception) The newer no-intro sets scrapped most of the satellaview headers which made the roms non-functional.

RetroProf

  • Full Member
  • ***
  • Posts: 149
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #25 on: February 08, 2013, 04:24:44 am »
I've.... never even tried to use a Satellaview game/ROM before. But I will remember what you've said about them absolutely requiring headers for future conversations.

So basically, in the blind pursuit of removing useless headers, they also removed essential headers, thereby destroying certain ROMs?

Incredible. And also a little sad. :(

Didn't anybody knowledgeable in the scene try to point this out before it started happening? Or did it just happen and those who knew the Satellaview only found out afterwards? I mean, did you just log on one day and find: oh crap, they've broken the Satellaview ROMs!

I find this all very fascinating.

Zoinkity

  • Hero Member
  • *****
  • Posts: 565
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #26 on: February 08, 2013, 11:13:43 am »
I was originally going to write that this sounds like private initiative by someone who was grossly underinformed.  That said, I got a PM yesterday through emutalk asking if they need to delete the 0x1000(ish) bootstrap from N64 games too.  Apparently there's a misconception that the internal N64 headers which are a physical part of ROM and used by the bootstrap to set up PI access and timing are superfluous and have already started deleting them.  Obviously they never tried to boot a cart where everything is 64 bytes off...

So, WTF is going on?  Is this some initiative through No-Intro?
No-intro uses .md instead of .gen for generated filenames, so that would explain that bit.  They're also only throwing valid against headerless SNES roms now.  Plus, none of my selfdumped titles show valid anymore.  Great.


Poor Satellaview.  With all the difficulty involved in preserving the thing properly the last thing it needs is another kick in the pants.

KingMike

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 7253
  • *sigh* A changed avatar. Big deal.
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #27 on: February 08, 2013, 01:58:48 pm »
I was originally going to write that this sounds like private initiative by someone who was grossly underinformed.  That said, I got a PM yesterday through emutalk asking if they need to delete the 0x1000(ish) bootstrap from N64 games too.  Apparently there's a misconception that the internal N64 headers which are a physical part of ROM and used by the bootstrap to set up PI access and timing are superfluous and have already started deleting them.

Wouldn't they notice when the ROM is an unusual size?
(I know it was common to use multiple ROMs on NES and SNES. I'm guessing N64 games tended to use only one or two ROMs (as 12 MB seems to be the only common non-power-of-two size, though I believe there were a couple 40 MB as well).
But I don't know as I haven't myself opened any N64 carts, with the (seeming useless) metal plates inside the carts. Did those have a purpose besides making the carts unnecessarily heavy (and probably more expensive)? :P
"My watch says 30 chickens" Google, 2018

henke37

  • Hero Member
  • *****
  • Posts: 643
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #28 on: February 08, 2013, 08:06:28 pm »
Shielding? People always like to throw metal shields on electronics.

Zoinkity

  • Hero Member
  • *****
  • Posts: 565
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #29 on: February 08, 2013, 09:09:46 pm »
The metal plate is a floating ground.

The PCBs are surprisingly standardized and boring, and for larger games they simply replaced smaller chips with larger ones.  Never saw one with more than 2 ROM chips that wasn't a dev tool or an early HK knockoff, and the devtools almost all use 8 8MB eeproms.  About the only game-to-game variation you'd see would be between FLASHROM save types. 


Revenant

  • Full Member
  • ***
  • Posts: 206
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #30 on: February 08, 2013, 10:28:42 pm »
I know there is quite a rejection of the existence of snes headers, but please keep in mind that satellaview headers are often mistook to be snes headers (and I didn't even see one mention of them here after my post) but they are very much needed to let the roms be functional. Just wanted to point that out again. Kiddo wanted to create a isolated satellaview set, so I guess this is very needed if you see the general opinion about SNES roms. Esspeacialy if you keep in mind that most of those games havent been hacked or emulated so far. (some emulators and the sd2snes may be an exception) The newer no-intro sets scrapped most of the satellaview headers which made the roms non-functional.
I've.... never even tried to use a Satellaview game/ROM before. But I will remember what you've said about them absolutely requiring headers for future conversations.

So basically, in the blind pursuit of removing useless headers, they also removed essential headers, thereby destroying certain ROMs?

Incredible. And also a little sad. :(

Didn't anybody knowledgeable in the scene try to point this out before it started happening? Or did it just happen and those who knew the Satellaview only found out afterwards? I mean, did you just log on one day and find: oh crap, they've broken the Satellaview ROMs!

I find this all very fascinating.

The only sort of "Satellaview header" that exists is just an alternate form of the internal SNES ROM header, not anything similar to the (now-useless) copier headers and such being discussed in this thread.

From what I understand, the issue surrounding No-Intro's Satellaview ROMs involves copies of games in which certain information in the internal header (time-sensitive info, namely the games' download dates), for whatever reason, is missing, whether removed by hand or otherwise. No-Intro seems to incidentally have a high concentration of Satellaview dumps with this information missing, while GoodSNES is almost the opposite.

It's certainly an issue for Satellaview preservationists, but it's not at all related to the push for headerless SNES ROMs in the sense that you're thinking of.

Edit: I asked Kiddo about it on IRC, since I remembered him talking about this issue a bit in #tcrf, and he clarified the circumstances a bit:
Quote
(10:42:33 PM) Kiddo: All I know is that I tried tracing some ROMs to the earliest source possible
(10:42:41 PM) Kiddo: found ORIGINAL SCENE RELEASES of some BS marvelous ROMs
(10:42:46 PM) Kiddo: and the dates were STILL missing from the headers.
(10:42:58 PM) Kiddo: It baffled me.
(10:43:47 PM) Kiddo: For some reason GoodSNES kept dates in as many ROMs as it could while in the no-intro ROMs they seemed to mostly be gone besides what was dumped when I started Satellablog
(10:44:07 PM) Kiddo: I tried working to correct that but it appears there's some ROMs where we simply can't find ones with dates in.
(10:44:20 PM) Kiddo: The BS Marvelous example there being notable because I basicaly traced it as early as possible.
(10:44:32 PM) Kiddo: it was released that way.
(10:44:40 PM) Kiddo: And that's the only release we can work with.
(10:44:43 PM) Kiddo: :<
(10:45:04 PM) Kiddo: I don't exactly know why or when this started happening, though.
« Last Edit: February 08, 2013, 10:49:41 pm by Revenant »

RetroProf

  • Full Member
  • ***
  • Posts: 149
    • View Profile
Re: Why would Planetemu replace all their ROM sets with headerless versions?!
« Reply #31 on: February 09, 2013, 05:56:35 am »
I was originally going to write that this sounds like private initiative by someone who was grossly underinformed.  That said, I got a PM yesterday through emutalk asking if they need to delete the 0x1000(ish) bootstrap from N64 games too.  Apparently there's a misconception that the internal N64 headers which are a physical part of ROM and used by the bootstrap to set up PI access and timing are superfluous and have already started deleting them. 

What the bloody hell?!

It sounds to me like some uninformed idiot is riding roughshod over the work of others, screwing stuff up willy-nilly.

What the hell is going on in the ROM preservation community?

Surely those with more knowledge/sense should be putting a stop to this? Isn't there, you know, some kind of gate keeper or authority committee which oversees this and prevents incompetence like that taking place?

Does this mean entire archives of N64 ROMs are going to be useless?

To what end are they removing those N64 headers, which based on your description are integral to the data?

What you've described is so abhorrent, frankly I'm having trouble comprehending it. Are any gaming news websites covering this? I feel like I've stumbled into some kind of nefarious conspiracy!

Am I over-reacting? Shouldn't whoever is behind this destruction of data e stopped, and those going along with it be educated on the matter? It feels like something needs to be done about this, right now. Should we do something? Are we doing something? Are people who know about this doing what needs to be done? I'm just a writer, but am a keen emulation user for article writing. I want the entire sphere of emulation to be protected, preserved and made to be the best and most accessible that it can be.

FallenAngel2387

  • Hero Member
  • *****
  • Posts: 740
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #32 on: February 09, 2013, 09:09:31 am »
You just asked if there was some kind of authority on something that could be considered illegal.

Anyway, if we all know gamers, there will be plenty of bitching when they realize the game don't work, and this will be corrected before long. Unless they just decide to get roms from somewhere else.

RetroProf

  • Full Member
  • ***
  • Posts: 149
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #33 on: February 09, 2013, 10:01:51 am »
Drugs are illegal, but that doesn't stop the mafia and cartels from controlling production and distribution.

Anyway, I hope some kind of logical consensus is reached.

Zoinkity

  • Hero Member
  • *****
  • Posts: 565
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #34 on: February 09, 2013, 10:50:07 am »
Technically speaking this place's archive was altered because ROM verification tools (which in some cases automatically rename and/or patch) are being updated to unintentionally damage ROMs.

Presuming you're someplace where it's legal to make a backup copy of a ROM, that means these tools can damage an otherwise good dump.
It also means that anyone who makes a patch (which depending where you're at is also mostly legal) has to take into account damaged ROMs when dealing with the bug reports.
So, it isn't just throwing a dead kitten in the rum to kill the pirates. 

RetroProf

  • Full Member
  • ***
  • Posts: 149
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #35 on: February 10, 2013, 09:38:39 am »
Bloody hell, had a look on Emu paradise, and they TOO are running .md Mega Drive romsN.

Someone might suggest using tools to fix this problem, but that defeats the whole purpose of everything. There shouldn't be a need to work harder. >_<

FallenAngel2387

  • Hero Member
  • *****
  • Posts: 740
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #36 on: February 10, 2013, 02:42:42 pm »
Yeah, but they carry GoodRom complete sets, which counters the issue with the individual roms.

Myria

  • Jr. Member
  • **
  • Posts: 49
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #37 on: February 14, 2013, 02:17:55 am »
I'm going to make my Breath of Fire 1 patch tell you what you did wrong if you try to apply the patch to a ROM with an SMC header.  Similarly for if you patch the wrong version.

RadioTails

  • Sr. Member
  • ****
  • Posts: 391
  • Holding the Mega Drive controller
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #38 on: February 14, 2013, 08:11:03 am »
Why would anyone do this? ROMs without headers are useless. Utterly useless. I thought the gaming community had long ago agreed that ROMs need to be single files rather than broken up, that we'd all agreed on a specific naming convention, and that headers were to be left alone as standard?

Without headers:

* You can't translate patch them.

* You can't apply hacks to them.

* Worst of all - the FDS games will NOT WORK in the PowerPak cartridge, because they absolutely MUST be disk images with their original headers. I had to download the entire GOODNES set to get them working, which I only discovered after wasting my time with Planetemus broken collection of non-functioning disk images.

You do realise that Nintendo themselves use unheaded roms in the SNES Virtual Console games? Should we go and complain at them? :P
Avatar by LazyNinjartist

RetroProf

  • Full Member
  • ***
  • Posts: 149
    • View Profile
Re: ROM archives of the world are being DESTROYED! (see page 2)
« Reply #39 on: February 14, 2013, 09:03:20 am »
As I've come to realise, different ROMs for different system emulators, require different standards.

For some headers are useless, for others they are essential.

As we've ascertained, the problem here appears to be the arbitrary removal of headers for games which require them, by imbeciles who have zero idea of what they are doing.

Full disclosure:
I haven't patched a SNES ROM in a few years. All of my complaints in this thread are NES, FDS, PCE and Mega Drive related.

And SNES Satellaview, based on statements from others.