News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Hasher-js: Browser-based ROM validation  (Read 1258 times)

snarfblam

  • Submission Reviewer
  • Hero Member
  • *****
  • Posts: 587
  • CANT HACK METROID
    • View Profile
    • snarfblam
Hasher-js: Browser-based ROM validation
« on: September 26, 2018, 09:32:22 pm »
In the future, the most prevalent software platform will be the moral equivalent of a word processor document with a hacked-in scripting engine. That future is now. Fittingly, ROM Hasher has been ported to javascript. You can verify your ROM right in the same browser you just used to download the patch. And since Hasher-js will come in the form of a javascript library, in theory it can eventually be incorporated directly into RHDN's download pages (or any page).

A temporary deployment can be accessed at https://snarfblam.com/hash/



I'd kindly ask that people report back on their experience, especially if there is any kind of problem. If there is an issue, it's massively helpful if you can provide any output in the debug console (CTRL+SHIFT+I in Chrome and FF) as well as anything you can tell me about the file you're attempting to hash.

Suggestions and criticism are welcome. For example, Zynk suggested that the summary should include not only the SHA-1, but CRC32 and MD5 hashes. Our goal is to guide people toward using SHA-1 as the standard, and the additional hashes are available below, but if most people agree that these additional hashes should be more conspicuous I can look into that. Point out anything that can make the tool simpler, faster, or more useful.  :thumbsup:

oracylum

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Hasher-js: Browser-based ROM validation
« Reply #1 on: September 27, 2018, 07:42:08 am »
Awesome! Been waiting for someone to create something like this. I tried it with a few NES ROMs and had no issues.

Do you plan to add zip support?

Thanks for the hard work.

snarfblam

  • Submission Reviewer
  • Hero Member
  • *****
  • Posts: 587
  • CANT HACK METROID
    • View Profile
    • snarfblam
Re: Hasher-js: Browser-based ROM validation
« Reply #2 on: October 11, 2018, 09:55:37 pm »
Do you plan to add zip support?

No such plans. I guess it would be useful for people who have a library of zipped ROMs and use soft-patching? I couldn't justify spending the time on it unless there were some serious demand for it. And I don't think there was much demand for this project at all to begin with.

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5716
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Hasher-js: Browser-based ROM validation
« Reply #3 on: October 12, 2018, 03:45:09 pm »
I tried this with several SNES ROMs and all information was correct. No issues. Interesting progress bar there. ;D

A few minor suggestions:

1. Display hex in uppercase like ROM Hasher.
2. Display the 'Header offset' field in hex like the rest of the values on that tab.
3. Since you're still displaying internal checksum information, add back the 'Checksum Valid' field from ROM Hasher.


CRC32 might be worth adding to the main output. It does seem to still have much legacy momentum and use for people.

Do you think there is any usefulness to adding PSX SLUS/SCES/SLES numbers? Hashes are probably less useful for CD based systems like that and that would allow identification. Not my area of expertise though.
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

snarfblam

  • Submission Reviewer
  • Hero Member
  • *****
  • Posts: 587
  • CANT HACK METROID
    • View Profile
    • snarfblam
Re: Hasher-js: Browser-based ROM validation
« Reply #4 on: October 21, 2018, 11:15:00 am »
1. Display hex in uppercase like ROM Hasher.

I think I'm with you on that one. Javascript produces lowercase hex by default, and once I got started I stuck with lowercase to be consistent. I think I'd prefer uppercase. Don't know what the general preference is, or if there is one.


2. Display the 'Header offset' field in hex like the rest of the values on that tab.

Yes, I should have. I think I should also add a $ prefix to hex numbers for clarity.


3. Since you're still displaying internal checksum information, add back the 'Checksum Valid' field from ROM Hasher.

That's easy enough. While I was looking into this, I also noticed I'm calculating checksums for headered SNES ROMs incorrectly. Oops.


CRC32 might be worth adding to the main output. It does seem to still have much legacy momentum and use for people.

I suppose you're right. There was a logic to excluding it, but adding it in there will probably be a net benefit to users.


Do you think there is any usefulness to adding PSX SLUS/SCES/SLES numbers? Hashes are probably less useful for CD based systems like that and that would allow identification. Not my area of expertise though.

I know next to nothing about optical media and the relevant ROM containers. Is this information presently readily available to users by any means? Is it in the ROM in a programatically accessible form? Or would it need to be looked up in a database based on uniquely identifiable data in the ROM? I just don't know whether or not there is a need here, and if there is, I'd need to educate myself on the matter before adding any kind of support.

Mugi

  • Sr. Member
  • ****
  • Posts: 251
  • Personal text
    • View Profile
    • Blacklabel-translations
Re: Hasher-js: Browser-based ROM validation
« Reply #5 on: October 21, 2018, 12:10:26 pm »

I know next to nothing about optical media and the relevant ROM containers. Is this information presently readily available to users by any means? Is it in the ROM in a programatically accessible form? Or would it need to be looked up in a database based on uniquely identifiable data in the ROM? I just don't know whether or not there is a need here, and if there is, I'd need to educate myself on the matter before adding any kind of support.

ps1 stores the "slus number" in various places;
the name of the main executable in the format of XXXX_YYY.ZZ, and inside system.cfg as a boot parameter, both the SLUFILE (main executable) and the system.cfg are always located at the root of the disk filesystem.

ps2 is more or less identical to ps1 in this behavior, sans the fact that ps2 may have dvdrom instead of cdrom (i suppose reading them differs, not really my area of expertise.)

in PSP, the catalog or "SLUS" number is located inside UMDDATA.BIN at the root of the disk filesystem. the bin file is practically a normal txt so reading it should not be awfully complicated. Various other locations for obtaining the game id exist, most notably params.sfo inside PSP_GAME folder in the disk filesystem.
In PSP we trust.

mz

  • Sr. Member
  • ****
  • Posts: 404
  • Whore
    • View Profile
Re: Hasher-js: Browser-based ROM validation
« Reply #6 on: October 21, 2018, 01:49:00 pm »
ps1 stores the "slus number" in various places
Correction: they *may* store it.

Most early Japanese games don't even have a SYSTEM.CNF file, and just use a PSX.EXE file at the root of the disk (which is automatically run). In Gunners Heaven, for instance, you won't find the "SLUS number" anywhere on the disk.

Other games can name the main executable anything else (Medarot R, for instance, calls it GAME.EXE.)

Also, different versions of the same game may use the same serial number, so I find this kind of identification to be mostly useless...
There has to be a better life.

Mugi

  • Sr. Member
  • ****
  • Posts: 251
  • Personal text
    • View Profile
    • Blacklabel-translations
Re: Hasher-js: Browser-based ROM validation
« Reply #7 on: October 22, 2018, 05:10:35 am »
well, yeah, it *should* store it, but there's always the few special snowflakes that just dont :P

ps1 was a gigantic mess as far as standards go, so that be that. Point was moreso, that's how it should be and that's where it should be read from.
if it's not there then one could always dig into the executable to find it but yeah, in terms of general identification, the executable name and/or the boot line on system.cfg is the way to go.

as far as using the gameID for identifying a good/bad image for patching, as already stated, revisions of a game might and mostly do reuse the gameID, so identifying a specific rom from the gameID is pointless.

in the case of psp, it could technically work as the param.sfo does list the version number, as well as the required firmware version number, and a whole bunch of other shit that could technically be used to identify a specific build of a game. But then again, just using a checksum does the same thing, no ?
In PSP we trust.

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5716
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Hasher-js: Browser-based ROM validation
« Reply #8 on: October 22, 2018, 08:50:07 am »
in the case of psp, it could technically work as the param.sfo does list the version number, as well as the required firmware version number, and a whole bunch of other shit that could technically be used to identify a specific build of a game. But then again, just using a checksum does the same thing, no ?

I was under the impression that disc images on some platforms were not exact enough that there can be only one checksum that defines a valid target image. I think this applies to games with CD audio tracks at least (if not other cases). I thought that may have been a reason the catalog ids were being used to tell people what version of a game their patch was for. Even if the checksum and the catalog ID are flawed, I would think that would still be better than no identifying information whatsoever. If it really is completely useless though, maybe it's not.
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

Mugi

  • Sr. Member
  • ****
  • Posts: 251
  • Personal text
    • View Profile
    • Blacklabel-translations
Re: Hasher-js: Browser-based ROM validation
« Reply #9 on: October 22, 2018, 10:00:25 am »
well, with disk based games, i believe that the only point of checksum validating an iso is if the targeting patch actually requires a checksum match to function (xdelta throws a fit if you try to patch an xdelta file to an iso that has a different checksum than it was intended for) but as with optical image isos the checksums might and oftentimes do differ while the game data itself might still be just fine. Especially in the case of PSP where rippung the UPDATE folder out from the iso to save space is extremely common. But since people in general use patches such as ips, bps or xdelta for games, a checksum matching .iso is required.

like earlier was mentioned, especially ps1 and ps2 tend to reuse their game ID between versions of the game and oftentimes identifying what is what can get really tricky. For an example, there is 3 revisions of japanese valkyrie profile for ps1, and aside from the size of the data container (which is compressed and encrypted.) their contents are, to my knowledge identical.
Telling them apart is virtually impossible if you dont know what you're looking for, such as internal file build dates or specific filesizes.

it really boils down on how the game and it's data is intended to be used.
It is for this reason alone I'm personally shifting away from using "patch" files in general completely (I only work with systems using optical disk media), and the hacks I'm doing nowadays will eventually be released as pure assembly code and resource assets (graphics, video, audio) instead. Providing a copy or a download link to Kingcom's armips to assemble the code and a tool for rebuilding both the .iso file and the relevant data containers will do the job regardless of if the game's checksum matches anything, as long as the data itself is not damaged.


I haven't gotten to the point of releasing anything yet, but Im hoping that once my little project of translating zill o'll for psp is complete, i will along with the patch, provide a solid enough patching framework (commandline .iso tools, amongst other things.) that maybe perhaps in my ideal little world, people adapt to use, making this whole checksum-specific .iso files thing obsolete.

i do see the good thing in having such a validation for checking, for example the SLUS code of a playstation *** game, which can quickly identify if you have a PAL/US version of the disk for example, but beyond that there is little use for it what comes to identifying a suitable iso for a specific patch, because as already said, a lot of patches simply require a matching checksum, so any further validation is instantly rendered pointless.
In PSP we trust.

Nightcrawler

  • Hero Member
  • *****
  • Posts: 5716
    • View Profile
    • Nightcrawler's Translation Corporation
Re: Hasher-js: Browser-based ROM validation
« Reply #10 on: October 22, 2018, 02:52:28 pm »
OK. Custom build tools are certainly a nice solution.

Checksums can be optional with xdelta, both in patch creation and application. Delta Patcher supports this in both cases, for example. So, you can still create an xdelta patch with multiple possible targets if you desire.
TransCorp - Over 20 years of community dedication.
Dual Orb 2, Wozz, Emerald Dragon, Tenshi No Uta, Glory of Heracles IV SFC/SNES Translations

Mugi

  • Sr. Member
  • ****
  • Posts: 251
  • Personal text
    • View Profile
    • Blacklabel-translations
Re: Hasher-js: Browser-based ROM validation
« Reply #11 on: October 23, 2018, 05:54:34 am »
Im aware of this with xdelta, the problem is that lots of people are not, as by default it needs a checksum matching file, and most patches are just made like that. I guess we should just throw an awareness campaign on how to patch things :P
In PSP we trust.

phonymike

  • Jr. Member
  • **
  • Posts: 8
    • View Profile
Re: Hasher-js: Browser-based ROM validation
« Reply #12 on: October 25, 2018, 08:28:47 am »
I ran Tengai Makyou Zero through and seem to get the wrong checksum (internal header calculated). Ucon64 and SNES9x give the same checksum but your app gives a different one.

From fullsnes:

Quote
Observe that the SPC7110 ROM checksums at [FFDCh..FFDFh] are calculated unconventionally: 3MB/5MB aren't "rounded-up" to 4MB/8MB. Instead, 3MB is checksummed twice (rounded to 6MB). 2MB/5MB are checksummed as 2MB/5MB (without rounding).

There's also a list of a few edge case games to test under "ROM Size / Checksum Notes"



Also Tales of Phantasia seems to be wrong.
« Last Edit: October 25, 2018, 08:45:46 am by phonymike »