News: 11 March 2016 - Forum Rules
Current Moderators - DarkSol, KingMike, MathOnNapkins, Azkadellia, Danke

Author Topic: Lightweight and Fast Checksum Calculator for SNES  (Read 581 times)

Cthulhu88

  • Jr. Member
  • **
  • Posts: 61
    • View Profile
Lightweight and Fast Checksum Calculator for SNES
« on: September 10, 2019, 02:06:13 am »
One thing that bothered me while hacking Chrono Trigger was the lack of a decent and modern CLI checksum calculator.
The ones available either weren't fast enough for my taste, too bloated or didn't calculate the sum correctly for all variations (ExROM, size not being a power of 2, etc).
Initially I had it written for just 32Mbits+ (Ex)HiROMs in both PowerShell and C++, as this is all that I needed for Chrono Trigger. However, I've since improved on it and added support for all ROM types.

It supports HiROMs and LoROMs ranging from 2 Mbits to 64 Mbits (as long as their size is divisible by 32 KB). The 512 bytes SMC header is also supported and is neither added nor removed from the ROM after fixing its checksum.
The sum is performed by 8 threads, greatly reducing execution time.

This is an 64 Mbits ROM running on the debug build (no optimizations) on an i7-4770k @ 4.3 GHz and a 7200 RPM HDD.
Code: [Select]
[DEBUG] HEADER: NO
[DEBUG] ROM SIZE: 8388608 bytes
[DEBUG] MODE: HiROM
[DEBUG] CHECKSUM: 0x7941 0x86BE
[DEBUG] CHECKSUMEX: 0x2A79 0xD586
[DEBUG] TIME: 0.0383094 seconds

To use it, just run it with the ROM file as the first argument. Ex:
Code: [Select]
checksum "Chrono Trigger (USA).sfc"
In the compressed file below I've included the C++ source code, Visual Studio's project files and a 32-bits Windows compiled binary.
The source code will compile just fine under GCC on Linux (I haven't tested on Clang).

Since I've not tested it extensively, I'll leave it here for some testing before I submit it to the RHDN.

UPDATED 1.0.1 (10/09/2019):
  • Fixed header detection for some ROMs not having the region code as the last byte in Game ID. Example: Final Fantasy VI (NA).
  • Fixed LoROMs bigger than 4 MBs being corrupted by writing to a second non-existent header at 0x407FB0 (LoROM, regardless of size, only has one header at 0x007FB0). Example: Star Ocean.

Download
« Last Edit: September 10, 2019, 09:01:45 pm by Cthulhu88 »

PowerPanda

  • Jr. Member
  • **
  • Posts: 86
    • View Profile
Re: Lightweight and Fast Checksum Calculator for SNES
« Reply #1 on: September 10, 2019, 09:37:06 am »
Thank you! It was very easy to use, despite the lack of GUI.

Cthulhu88

  • Jr. Member
  • **
  • Posts: 61
    • View Profile
Re: Lightweight and Fast Checksum Calculator for SNES
« Reply #2 on: September 10, 2019, 02:34:19 pm »
Thank you! It was very easy to use, despite the lack of GUI.

I've written it mainly to be used by scripts, such as make scripts. This is why there is no GUI.

EDIT:

Updated with some fixes.
« Last Edit: September 10, 2019, 09:09:33 pm by Cthulhu88 »

KingMike

  • Forum Moderator
  • Hero Member
  • *****
  • Posts: 6906
  • *sigh* A changed avatar. Big deal.
    • View Profile
Re: Lightweight and Fast Checksum Calculator for SNES
« Reply #3 on: September 11, 2019, 01:03:10 am »
The sum is performed by 8 threads, greatly reducing execution time.
That sounds like extreme overkill. Calculating a SNES checksum should not take much time for practically anything capable of running SNES games.
"My watch says 30 chickens" Google, 2018

Cthulhu88

  • Jr. Member
  • **
  • Posts: 61
    • View Profile
Re: Lightweight and Fast Checksum Calculator for SNES
« Reply #4 on: September 11, 2019, 02:27:53 am »
That sounds like extreme overkill. Calculating a SNES checksum should not take much time for practically anything capable of running SNES games.

Well, the time it takes to calculate the checksum depends on the language. My PowerShell implementation even when multithreading was hitting 10-15 seconds.
And yes, on native code even without multiple threads it will be really fast, but there is still a difference between near instantaneous and instantaneous.

PCs today are all multi-core, so why not use it?
My code is about 700 lines long and about 70% of it is header detection, error handling and debugging code. It was no trouble at all to add multithreading, even with a low-level API such as std::thread.
A 32 Mbits ROM will have 4+ millions additions just to calculate the checksum, and assuming it's just unoptimized x86 mov and add instructions, there is no reason not to spread the load across multiple CPUs, if the implementation is simple enough.

As for it being 8 threads, the size of the ROM is divisible by 32 KB, so it's also divisible by 8, no need to perform any extra calculations to get the block size for each thread.
Also most modern CPUs are quad-cores and some run it on HT (8 logical cores).

Overall, as I've mentioned, I've made this mainly for including in scripts. The faster the implementation, the less clogging on the script execution, more so when I need to frequently execute it (I just attach this calculator to all my make scripts).