2016-09-10 09:31:17 -07:00
************************************************
* SabreTools - DAT management software *
* https://github.com/mnadareski/wizzardDesktop *
************************************************
Table of Contents
-----------------
1.0 Introduction and History
2.0 Included Programs
2016-09-12 16:49:18 -07:00
2.1 RombaSharp
2.2 SabreTools
2.3 SimpleSort
2016-09-21 20:16:57 -07:00
3.0 Removed Commands
4.0 Examples
5.0 Contributors
6.0 Licensing
2016-09-10 09:31:17 -07:00
** Section 1.0 - Introduction and History
Hello and welcome to the latest release of SabreTools! We are happy that you
chose to use our software for your DAT management needs. But what exactly is
SabreTools? How did it get started? Why is it named so weirdly? Why should we
care? Well, the first two can be answered, though the last one is up to you.
SabreTools has its roots in an internal tool developed by RomShepherd member
The Wizard of DATz (WoD). The aim of the PHP-based program was to collate data
frommultiple sources and create new DAT files for custom sources as well as
DAT files that have duplicate data removed. WoD maintained this software and
used it to create near-monthly releases for quite a while.
Around the middle-end of 2015, WoD announced that they would be taking a break
from the process of creating and releasing new files. A couple of months went
by with no information. Finally, WoD finally told the community that they would
no longer be able to maintain either the project or the software involved,
releasing the source code of the internal project in the process. During that time,
RomShepherd member darksabre76 picked it up to see how the program even worked.
After updating the code to the newest version of PHP and trying to figure out
what each page did, he announced that it was going to be easier to do a complete
rewrite while maintaining functionality. He was given a lot of support and got
started with the aim to keep the code completely open source so that anyone can
help. It took a lot of work and help from fellow user emuLOAD, but the code
finally started to shape up in the form of WoD Redux. This project is still
active and can be found at https://github.com/mnadareski/wizzardRedux.
After about a month of work on this, and continued support from the community,
a relatively new user to the site, now known as @tractivo, sent him a message
and asked if a desktop port of some of the core features was possible. It was
difficult for most people to run the PHP code locally, so it made sense. With
the aim for people to use the web version primarlily, the desktop version,
originally called DATabase, was written. Pretty soon, this got even more support
and soon became the main focus. For a while, code parity between the C# and PHP
versions was attempted, but it became apparent that the desktop version was going
to be the better option.
Development accelerated, soon leaving the PHP version in the dust. Once the basics
were implemented, more features were requested, including better merging and DAT
management. Spurred on by an increasing number of community members, DATabase
accrued a lot of features and helper programs. The accuracy of the program also
increased, soon matching or surpassing existing options in functionality. At some
point during this development, some of the smaller tools got wrapped into DATabase,
slowly getting further from just a desktop port of WoD Redux.
With these changes, some external prodding, and a little bit of ego, darksabre76
2016-09-12 16:31:52 -07:00
decided to rechristen the set of programs to SabreTools. This freed the program
2016-09-10 09:31:17 -07:00
from being just about DAT management and allowed it to be more of a Swiss Army
Knife of DAT tools. Development has been on and off since then, and is still
constantly evolving to this day (assuming that at this very moment development is
still active).
** Section 2.0 - Included Programs
Below are a list of the programs that are included in the current SabreTools
release. Each of them have a brief description of the tool along with in-depth
desciptions of all flags.
2016-09-12 16:49:18 -07:00
** Section 2.1 - RombaSharp
2016-09-10 09:31:17 -07:00
2016-09-12 14:08:32 -07:00
RombaSharp is an ongoing "spiritual port" of the Romba tool (https://github.com/uwedeportivo/romba).
The code is not based on the actual source, rather taking the features and using the code
already written for a lot of other features. The following descriptions are based on what WILL
be done with each flag. Not all features are currently available.
Usage:
RombaSharp.exe [options] [filename|dirname] ...
Options:
-?, -h, --help Show the built-in help text
Built-in to most of the programs is a basic help text
archive Adds ROM files from the specified directories to depot
-only-needed Only archive ROM files in database
build For each specified DAT file it creates TZip files
dbstats Prints db stats
diffdat Creates a DAT file for entries found in the new DAT
-new= DAT to compare to
dir2dat Creates a DAT file for the specified input directory
-out= Filename to save out to
fixdat For each specified DAT file it creates a fix DAT
lookup For each specified hash, look up available information
memstats Prints memory stats
miss For each specified DAT file, create miss and have file
progress Shows progress of currently running command [OBSOLETE]
purge-backup Moves DAT index entries for orphaned DATs
purge-delete Deletes DAT index entries for orphaned DATs
refresh-dats Refreshes the DAT index from the files in the DAT root
shutdown Gracefully shuts down server [OBSOLETE]
2016-09-10 09:31:17 -07:00
2016-09-12 16:49:18 -07:00
** Section 2.2 - SabreTools
2016-09-10 09:31:17 -07:00
2016-09-12 13:51:13 -07:00
SabreTools is the main application of the SabreTools suite. It is mostly just a frontend
to a lot of features that are available in the DLL and can be considered the reference
implementation. As such, it has a lot of possible flags and options that a user can select
from.
Included within this tool are a few former standalone executables:
- Convert/DATToMiss: Convert an arbitrary input DAT to a different format
- DATFromDir: Create a DAT file from a folder or file, sometimes called dir2dat
- DatSplit: Split a DAT based on 2 different file extensions
- Filter: Filter a DAT based on various user-defined criteria, optionally using wildcards
- HashSplit: Split a DAT based on the best available hash (No-dump, SHA-1, MD5, CRC)
- MergeDAT: Merge and optionally dedupe an arbitrary number of DAT files
- SingleGame: Trim game and rom names to fit NTFS length standards, optionally merging
all roms to a single game named "!" and forcing unpack
- UncompressedSize: Get statistics from one or more input DATs, including number of
roms, disks, files with available hash, and size
2016-09-20 17:39:01 -07:00
Formerly included within this tool is a former standalone executable:
- DATabase/DATabaseTwo: A managed DAT tool that allows for creating automatically merged
DATs based on one or more systems, sources, or a combination thereof
2016-09-21 20:16:57 -07:00
- OfflineMerge: Use merged DATs to create DATs used for managing offline arrays
2016-09-12 13:51:13 -07:00
Usage:
SabreTools.exe [options] [filename|dirname] ...
Options:
-?, -h, --help Show the built-in help text
Built-in to most of the programs is a basic help text
-d, --dfd Create a DAT from each input directory
Create a DAT file from an input directory or set of files. By default, this will
output a DAT named based on the input directory and the current date. It will also
treat all archives as possible games and add all three hashes for each file.
-nm, --noMD5 Don't include MD5 in output
This allows the user to skip calculating the MD5 for each of the files which will
speed up the creation of the DAT.
-ns, --noSHA1 Don't include SHA1 in output
This allows the user to skip calculating the SHA-1 for each of the files which will
speed up the creation of the DAT.
-b, --bare Don't include date in file name
Normally, the DAT will be created with the date in the file name. This flag removes
that but keeps the date tag intact.
2016-10-03 15:05:07 -07:00
-fp=, --forcepack= Set force packing
Set the forcepacking flag to one of the supported values:
None, Zip, Unzip
2016-09-12 13:51:13 -07:00
-f, --files Treat archives as files
Instead of trying to enumerate the files within archives, treat the archives as
files themselves. This is good for uncompressed sets that include archives that
should be read as-is.
2016-10-04 12:12:26 -07:00
-oa, --output-all Output in all available formats
Add outputting the created DAT in all available formats. See specific formats for
additional flags that may be used.
2016-09-12 13:51:13 -07:00
-oc, --output-cmp Output in CMP format
2016-09-28 10:28:10 -07:00
Add outputting the created DAT to clrmamepro format
2016-10-04 12:12:26 -07:00
-ocsv, --output-csv Output in Comma-Separated Value format
Add outputting the created DAT to standardized CSV format
2016-09-28 10:28:10 -07:00
2016-09-21 14:59:39 -07:00
-od, --output-dc Output in DOSCenter format
Add outputting the created DAT to DOSCenter format
2016-09-12 13:51:13 -07:00
-om, --output-miss Output in Missfile format
Add outputting the created DAT to GoodTools miss format
-omd5, --output-md5 Output in MD5 format
Add outputting the created DAT to MD5 format
2016-09-28 10:28:10 -07:00
-ool, --output-ol Output in OfflineList format
Add outputting the created DAT to OfflineList XML format
2016-09-12 13:51:13 -07:00
-or, --output-rc Output in RomCenter format
Add outputting the created DAT to RomCenter format
-os, --output-sd Output in SabreDAT format
Add outputting the created DAT to SabreDAT XML format
2016-09-28 10:28:10 -07:00
-osfv, --output-sfv Output in SFV format
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to SFV format
2016-09-28 10:28:10 -07:00
-osha1, -output-sha1 Output in SHA-1 format
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to SHA1 format
2016-09-28 10:28:10 -07:00
-osl, --output-sl Output in Software List format
Add outputting the created DAT to Software List XML format
2016-10-04 12:12:26 -07:00
-otsv, --output-tsv Output in Tab-Separated Value format
Add outputting the created DAT to standardized TSV format
2016-09-28 10:28:10 -07:00
-ox, -output-xml Output in Logiqx XML format (default)
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to Logiqx XML format
-gz, --gz-files Allow reading of GZIP files as archives
Since GZip files are not commonly used for file storage, this flag allows for
any GZip archives to have their contents hashed instead.
-ro, --romba Read files from a Romba input
Allow for reading of GZipped files as if they were from a Romba depot. This
implies that the files will be in the TorrentGZ format as well, including
naming convention.
-f=, --filename= Set the external name of the DAT
Set the base filename for the output DAT(s) [default is folder name plus date]
-n=, --name= Set the internal name of the DAT
Set the internal name for the output DAT(s) [default is folder name plus date]
-de=, --desc= Set the description of the DAT
Set the description for the output DAT(s) [default is the folder name]
-c=, --cat= Set the category of the DAT
Set the category for the output DAT(s) [default is blank]
-v=, --version= Set the version of the DAT
Set the version for the output DAT(s) [default is blank]
-au=, --author= Set the author of the DAT
Set the author for the output DAT(s) [default is blank]
-sd, --superdat Enable SuperDAT creation
Set the type flag to "SuperDAT" for the output DAT as well as preserving the
directory structure of the inputted folder, if applicable
2016-10-04 10:26:19 -07:00
-xof, --exclude-of Exclude romof, cloneof, sampleof tags
If this flag is enabled, then the romof, cloneof, and sampleof tags will be
omitted from the outputted DAT or DATs.
2016-09-14 10:25:01 -07:00
-ab, --add-blank Output blank files for folders
If this flag is set, then blank entries will be created for each of the empty
directories in the source. This is useful for tools that require all folders
be accounted for in the output DAT.
-ad, --add-date Output dates for each file parsed
If this flag is set, then the Date will be appended to each file information
in the output DAT. The output format is standardized as "yyyy/MM/dd HH:mm:ss".
2016-09-21 21:16:53 -07:00
-cf, --copy-files Copy files to the temp directory before parsing
If this flag is set, then all files that are going to be parsed are moved to the temporary directory before being hashed. This can be helpful in cases where the
temp folder is located on an SSD and the user wants to take advantage of this.
2016-10-03 22:08:10 -07:00
-h=, --header= Remove headers from hash calculations
If this is set, then all files that have copier headers that are detected will
2016-10-03 15:05:07 -07:00
have them removed from the hash calculation. This will allow for a headered collection
2016-10-03 22:08:10 -07:00
to be hashed without possibly variant information. If a particular header skipper is
defined, and that skipper exists, then it will be used instead of trying to find one
that matches.
2016-10-03 15:05:07 -07:00
2016-09-12 13:51:13 -07:00
-t=, --temp= Set the name of the temporary directory
Optionally, a temp folder can be supplied in the case the default temp directory
(inside the running folder) is not preferred. This is used for any operations
that require an archive to be extracted.
-mt={4} Amount of threads to use
Optionally, set the number of threads to use for the multithreaded operations.
2016-09-13 12:40:09 -07:00
The default is 4 threads; -1 means unlimited threads created. If the user specifies
that only 1 thread is to be used, it defaults to the original, serial implementation
of the DFD code.
2016-09-12 13:51:13 -07:00
-es, --ext-split Split a DAT by two file extensions
For a DAT, or set of DATs, allow for splitting based on a list of input extensions.
This can allow for combined DAT files, such as those combining two separate systems,
to be split. Files with any extensions not listed in the input lists will be included
in both outputted DAT files.
-exta= First set of extensions (comma-separated)
Set the extensions to be used to populate the first DAT. If more than one
extension is defined, they must be separated by commas.
-extb= Second set of extensions (comma-separated)
Set the extensions to be used to populate the second DAT. If more than one
extension is defined, they must be separated by commas.
-out= Set the name of the output directory
This sets an output folder to be used when the files are created. If a path
is not defined, the application directory is used instead.
2016-09-12 16:03:05 -07:00
-hd, --headerer Backup or restore copier headers from a variety of file types
2016-09-12 16:50:51 -07:00
Headerer is meant as an intermediary between header skipper files (which, a bit
apart from their name, do not just show how to skip copier headers) and rom managers
that do not use them.
2016-09-12 16:03:05 -07:00
By default, this will detect, store, and remove copier headers from a file or folder
of files. The headers are backed up and collated by the hash of the unheadered file.
Files are then output without the detected copier header alongside the originals with
the suffix .new. No input files are altered in the process.
The following systems have headers that this program can work with:
- Atari 7800
- Atari Lynx
- Commodore PSID Music
- NEC PC-Engine / TurboGrafx 16
- Nintendo Famicom / Nintendo Entertainment System
- Nintendo Famicom Disk System
- Nintendo Super Famicom / Super Nintendo Entertainment System
- Nintendo Super Famicom / Super Nintendo Entertainment System SPC Music
-re, --restore Restore headers to file(s)
Instead of the default extraction, this flag enables use of stored copier headers
to reapply them to files if they match the included hash. More than one header can
be applied to a file, so they will be output to new files, suffixed with .newX,
where X is a number. No input files are altered in the process.
2016-09-12 17:47:16 -07:00
-out= Set the name of the output directory
This sets an output folder to be used when the files are created. If a path
is not defined, the application directory is used instead.
2016-09-12 16:03:05 -07:00
2016-09-12 13:51:13 -07:00
-hs, --hash-split Split a DAT or folder by best-available hashes
For a DAT, or set of DATs, allow for splitting based on the best available hash for
each file within. The order of preference for the outputted DATs is as follows:
- Nodump
- SHA-1 available
- MD5 available
- CRC or worse available
-out= Set the name of the output directory
This sets an output folder to be used when the files are created. If a path
is not defined, the application directory is used instead.
-input= Set an input string
This should only be used if one of the inputs starts with a flag or another already
defined input.
-st, --stats Get statistics on all input DATs
This will output by default the combined statistics for all input DAT files. The stats
that are outputted are as follows:
- Total uncompressed size
- Number of games found
- Number of roms found
- Number of disks found
- Roms that include a CRC
- Roms that include a MD5
- Roms that include a SHA-1
- Roms with Nodump status
2016-09-26 16:42:06 -07:00
-bc, --baddump-col Add statistics for baddumps to output
Add a new column or field for counting the number of baddumps in the DAT
2016-09-26 14:38:05 -07:00
-csv, --csv Write all statistics to CSV
Output all rom information in standardized CSV format
2016-09-26 14:45:55 -07:00
-f=, --filename= Set the filename for the output
Set the filename (without extension) for the outputted report
2016-09-26 14:38:05 -07:00
-html, --html Write all statistics to HTML
This will output by default the combined statistics for all input DAT files.
2016-09-26 16:42:06 -07:00
-nc, --nodump-col Add statistics for nodumps to output
Add a new column or field for counting the number of nodumps in the DAT
2016-09-12 13:51:13 -07:00
-si, --single Show individual statistics
Optionally, the statistics for each of the individual input DATs can be output
as well. This can be useful to show where the size or amount of files found
in the combined totals can be broken down from.
2016-09-26 14:38:05 -07:00
-tsv, --tsv Output in Tab-Separated Value format
Output all rom information in standardized TSV format
2016-09-12 13:51:13 -07:00
-ts, --type-split Split a DAT or folder by file types (rom/disk)
For a DAT, or set of DATs, allow for splitting based on the types of the files,
specifically if the type is a rom or a disk.
-out= Set the name of the output directory
This sets an output folder to be used when the files are created. If a path
is not defined, the application directory is used instead.
-ud, --update Update a DAT file
This is the multitool part of the program, allowing for almost every manipulation
to a DAT, or set of DATs. This is also a combination of many different programs
that performed DAT manipulation that work better together.
2016-09-28 10:28:10 -07:00
2016-10-04 12:12:26 -07:00
-oa, --output-all Output in all available formats
Add outputting the created DAT in all available formats. See specific formats for
additional flags that may be used.
2016-09-12 13:51:13 -07:00
-oc, --output-cmp Output in CMP format
Add outputting the created DAT to clrmamepro format
2016-10-04 12:12:26 -07:00
-ocsv, --output-csv Output in Comma-Separated Value format
Add outputting the created DAT to standardized CSV format
-pre=, --prefix= Set prefix for all lines
Set a generic prefix to be prepended to all outputted lines
-post=, --postfix= Set postfix for all lines
Set a generic postfix to be appended to all outputted lines
Both prefix and postfix can use one of the following special strings:
- %game% - Replaced with the Game/Machine name
- %name% - Replaced with the Rom name
- %crc% - Replaced with the CRC
- %md5% - Replaced with the MD5
- %sha1% - Replaced with the SHA-1
- %size% - Replaced with the size
-q, --quotes Put double-quotes around each item
This quotes only the item and not the prefix and postfix
2016-09-28 10:28:10 -07:00
2016-09-21 14:59:39 -07:00
-od, --output-dc Output in DOSCenter format
Add outputting the created DAT to DOSCenter format
2016-09-12 13:51:13 -07:00
-om, --output-miss Output in Missfile format
Add outputting the created DAT to GoodTools miss format
-r, --roms Output roms to miss instead of sets
By default, the outputted file will include the name of the game, so this
flag allows for the name of the rom to be output instead.
-gp, --game-prefix Add game name as a prefix
Mainly used with the previous flag, this allows for the name of the game
to be used as a prefix to each file.
-pre=, --prefix= Set prefix for all lines
Set a generic prefix to be prepended to all outputted lines
-post=, --postfix= Set postfix for all lines
Set a generic postfix to be appended to all outputted lines
2016-09-12 14:52:46 -07:00
Both prefix and postfix can use one of the following special strings:
- %game% - Replaced with the Game/Machine name
- %name% - Replaced with the Rom name
- %crc% - Replaced with the CRC
- %md5% - Replaced with the MD5
- %sha1% - Replaced with the SHA-1
- %size% - Replaced with the size
2016-09-12 13:51:13 -07:00
-q, --quotes Put double-quotes around each item
This quotes only the item and not the prefix and postfix
-ae=, --add-ext= Add an extension to each item
To each item, a postfixed extension is added
-re=, --rep-ext= Replace all extensions with specified
When an extension exists, replace it with the provided instead
2016-09-12 14:52:46 -07:00
-rme, --rem-ext Remove all extensions from all items
For each item, the extension is removed
2016-09-12 13:51:13 -07:00
-ro, --romba Output in Romba format (requires SHA-1)
Instead of outputting the game or rom name, output the SHA-1 of the files
instead. This requires the source DAT to have SHA-1 hashes.
-omd5, --output-md5 Output in MD5 format
Add outputting the created DAT to MD5 format
2016-09-28 11:49:25 -07:00
-gp, --game-prefix Add game name as a prefix
This allows for the name of the game to be used as a prefix to each file
2016-09-28 10:28:10 -07:00
-ool, --output-ol Output in OfflineList format
Add outputting the created DAT to OfflineList XML format
2016-09-12 13:51:13 -07:00
-or, --output-rc Output in RomCenter format
Add outputting the created DAT to RomCenter format
-os, --output-sd Output in SabreDAT format
Add outputting the created DAT to SabreDAT XML format
2016-09-28 10:28:10 -07:00
-osfv, --output-sfv Output in SFV format
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to SFV format
2016-09-28 11:49:25 -07:00
-gp, --game-prefix Add game name as a prefix
This allows for the name of the game to be used as a prefix to each file
2016-09-28 10:28:10 -07:00
-osha1, -output-sha1 Output in SHA-1 format
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to SHA1 format
2016-09-28 11:49:25 -07:00
-gp, --game-prefix Add game name as a prefix
This allows for the name of the game to be used as a prefix to each file
2016-09-28 10:28:10 -07:00
-osl, --output-sl Output in Software List format
Add outputting the created DAT to Software List XML format
2016-10-04 12:12:26 -07:00
-otsv, --output-tsv Output in Tab-Separated Value format
Add outputting the created DAT to standardized TSV format
-pre=, --prefix= Set prefix for all lines
Set a generic prefix to be prepended to all outputted lines
-post=, --postfix= Set postfix for all lines
Set a generic postfix to be appended to all outputted lines
Both prefix and postfix can use one of the following special strings:
- %game% - Replaced with the Game/Machine name
- %name% - Replaced with the Rom name
- %crc% - Replaced with the CRC
- %md5% - Replaced with the MD5
- %sha1% - Replaced with the SHA-1
- %size% - Replaced with the size
-q, --quotes Put double-quotes around each item
This quotes only the item and not the prefix and postfix
2016-09-28 10:28:10 -07:00
-ox, -output-xml Output in Logiqx XML format (default)
2016-09-12 13:51:13 -07:00
Add outputting the created DAT to Logiqx XML format
-f=, --filename= Set the external name of the DAT
Set the base filename for the output DAT(s)
-n=, --name= Set the internal name of the DAT
Set the internal name for the output DAT(s)
-de=, --desc= Set the description of the DAT
Set the description for the output DAT(s)
-r=, --root= Set a new rootdir
Set the rootdir (as used by SuperDAT mode) for the output DAT(s)
-c=, --cat= Set the category of the DAT
Set the category for the output DAT(s)
-v=, --version= Set the version of the DAT
Set the version for the output DAT(s)
-da=, --date= Set a new date
Set the date for the output DAT(s)
-au=, --author= Set the author of the DAT
Set the author for the output DAT(s)
-em=, --email= Set a new email
Set the email for the output DAT(s)
-hp=, --homepage= Set a new homepage
Set the homepage for the output DAT(s)
-u=, --url= Set a new URL
Set the URL for the output DAT(s)
-co=, --comment= Set a new comment
Set the comment for the output DAT(s)
-h=, --header= Set a new header skipper
Set the header skipper for the output DAT(s)
-sd, --superdat Enable SuperDAT creation
Set the type flag to "SuperDAT" for the output DAT
-fm=, --forcemerge= Set force merging
Set the forcemerge tag to one of the supported values:
None, Split, Full
-fn=, --forcend= Set force nodump
Set the forcenodump tag to one of the supported values:
None, Obsolete, Required, Ignore
-fp=, --forcepack= Set force packing
Set the forcepacking flag to one of the supported values:
None, Zip, Unzip
2016-10-04 10:26:19 -07:00
-xof, --exclude-of Exclude romof, cloneof, sampleof tags
If this flag is enabled, then the romof, cloneof, and sampleof tags will be
omitted from the outputted DAT or DATs.
2016-09-12 13:51:13 -07:00
-clean Clean game names according to WoD standards
Game names will be santitized to remove what the original WoD standards
deemed as unneeded information, such as parenthized or bracketed strings
-sl, --softlist Use Software List name instead of description
By default, software list DATs are treated as "incorrect", using the game
descriptions as the name instead, since they tend to be more descriptive.
Enabling this flag allows for the original name to be preserved and keeping
the description as just a description.
-trim Trim file names to fit NTFS length
In the cases where files will have too long a name, this allows for trimming
the name of the files to the NTFS maximum length at most
-rd=, --root-dir= Set the root directory for calculation
In the case that the files will not be stored from the root directory,
a new root can be set for path length calculations
-si, --single All game names replaced by '!'
This is useful for keeping all roms in a DAT in the same archive or folder
-dd, --dedup Enable deduping in the created DAT
For all outputted DATs, allow for hash deduping. This makes sure that there
are effectively no duplicates in the output files.
-m, --merge Merge the input DATs
By default, all DATs are processed individually with the user-specified flags.
With this flag enabled, all of the input DATs are merged into a single output.
This is best used with the dedupe flag.
-b, --bare Don't include date in file name
Normally, the DAT will be created with the date in the file name. This flag
removes that instead of the default.
-di, --diff Create diffdats from inputs (all outputs)
By default, all DATs are processed individually with the user-specified flags.
With this flag enabled, input DATs are diffed against each other in one of the
following ways (flags below are used for specific diff files to be output instead
of all types at once):
-did, --diff-du Create diffdat containing just duplicates
All files that have duplicates outside of the original DAT are included
-dii, --diff-in Create diffdats for individual DATs
All files that have no duplicates outside of the original DATs are put
into DATs that are named after the source DAT
-din, --diff-nd Create diffdat containing no duplicates
All files that have no duplicates outside of the original DATs are included
-b, --bare Don't include date in file name
Normally, the DAT will be created with the date in the file name. This flag
removes that instead of the default.
-c, --cascade Enable cascaded diffing
-rc, --rev-cascade Enable reverse cascaded diffing
Each of the above flags allow for a special type of diffing in which the first
(or last) DAT is considered a base, and for each additional input DAT, it only
leaves the files that are not in one of the previous DATs. This can allow for
the creation of rollback sets or even just reduce the amount of duplicates
across multiple sets
-ip, --inplace Enable inplace, cascaded diff
This will overwrite the source files instead of writing them out to the
program folder by default (or the output folder if overridden)
-sf, --skip Skip output of first DAT
In times where the first DAT does not need to be written out a second time,
this will skip writing it. This can often speed up the output process.
-gn=, --game-name= Filter by game name
-rn=, --rom-name= Filter by rom name
-crc=, --crc= Filter by CRC hash
-md5=, --md5= Filter by MD5 hash
-sha1=, --sha1= Filter by SHA-1 hash
For each of the flags above, the user can specify either an exact match or can use a
wildcard as defined below (case-insensitive):
*00 means ends with '00'
00* means starts with '00'
*00* means contains '00'
00 means exactly equals '00'
-rt=, --rom-type= Filter by rom type
This allows users to only include roms or disks to their liking
-sgt=, --greater= Filter by size >=
-slt=, --less= Filter by size <=
-seq=, --equal= Filter by size ==
For each of the flags above, the user can specify a standard integer or one of the
various standard postfixes for size:
e.g. 8kb => 8000 or 8kib => 8192
2016-09-21 16:19:58 -07:00
-is=, status= Include only items with a given status
Include items with one of the supported values:
2016-09-21 16:46:01 -07:00
None, Good, BadDump, Nodump, Verified, NotNodump
2016-09-12 13:51:13 -07:00
-out= Set the name of the output directory
This sets an output folder to be used when the files are created. If a path
is not defined, the application directory is used instead.
2016-09-10 22:05:03 -07:00
2016-09-12 16:49:18 -07:00
** Section 2.3 - SimpleSort
2016-09-10 09:31:17 -07:00
SimpleSort is a WIP program that is meant as a command-line tool to quickly rebuild and
verify files based on a supplied DAT file. The eventual aim for this program is to have
a full rom management tool without a GUI, though this may not happen for a while.
Usage:
SimpleSort.exe [options] [filename|dirname] ...
Options:
-?, -h, --help Show the built-in help text
Built-in to most of the programs is a basic help text
-dat= Name of the DAT to be used for the various options
This DAT file is required for everything that SimpleSort currently does, so not
supplying one will result in an error. Depending on the additional flags that are
supplied, the DAT will be used differently, as described below. By default, the
DAT is used to check which files need to be rebuilt.
-out= Set the name of the output directory
This sets an output folder to be used by various parts of the program. As with the
DAT file, this input is used differently based on the flags that are supplied. By
default, the output folder is used as the target to rebuild files to.
-t=, --temp= Set the name of the temporary directory
Optionally, a temp folder can be supplied in the case the default temp directory
(inside the running folder) is not preferred. This is used for any operations that
require an archive to be extracted.
2016-09-12 22:00:25 -07:00
-d, --delete Enable deletion of the input files
Optionally, the input files, once processed, can be deleted. This can be useful
when the original file structure is no longer needed or if there is limited space
on the source drive.
2016-09-10 09:31:17 -07:00
-qs, --quick Enable quick scanning of archives
For all archives, if this flag is enabled, it will only use the header information
to get the archive entries' file information. The upside to this is that it is much
quicker than extracting all files to the temp folder. On the downside, it can only
get the CRC and size from most archive formats, leading to possible issues.
-v, --verify Enable verification of output directory
This overrides the default rebuilding and only requires the DAT and the output folder.
Here, the DAT is used to verify the output directory directly and then output a
simple FixDAT. This can be misleading, currently, because it only checks for exact
matches.
2016-09-12 22:00:25 -07:00
-c, --convert Enable conversion of input files to TGZ
This allows conversion of a folder or set of folders to TorrentGZ format without
requiring a DAT to rebuild from. It is only useful in a small amount of situations at
2016-10-05 16:54:52 -07:00
the present, but it is mostly meant for Romba compatibility at the present. If a DAT
is supplied, then files NOT matching the DAT will be written out only.
2016-09-12 22:00:25 -07:00
2016-09-16 11:58:44 -07:00
-tzip Enable TorrentZip output
Instead of outputting the files to standard ZIP archives, files will be rebuilt to
TorrentZip (TZ) files. This format is based on the ZIP archive format, but with
custom header information. This is primarily used by external tool RomVault
(http://www.romvault.com/) and is already widely used.
2016-09-10 09:31:17 -07:00
-tgz Enable Torrent GZ output
Instead of outputting the files to ZIP archives, files will be rebuilt to TorrentGZ
(TGZ) files. This format is based on the GZip archive format, but with custom header
information and a file name replaced by the SHA-1 of the file inside. This is
primarily used by external tool Romba (https://github.com/uwedeportivo/romba), but
may be used more widely in the future.
-r, --romba Enable Romba depot directory output
As an extension of the parent flag, this outputs the TGZ files into directories
based on the structure used by Romba. This uses nested folders using the first
4 bytes of the SHA-1, 1 byte for each layer of the directory name. It also
includes two auxilary files, .romba_size and .romba_size.backup, that have the
compressed size of the folder inside for use with Romba.
-do, --directory Enable outputting files uncompressed
Instead of outputting the files to ZIP archives, files will be rebuilt to named
subdirectories within the output folder. This is useful for when the DAT does not
already have the flag specified.
-7z={0} Set scanning level for 7z archives
-gz={2} Set scanning level for GZip archives
-rar={2} Set scanning level for RAR archives
-zip={0} Set scanning level for ZIP archives
For each of the major archive types recognized by the libraries used by this
program, scan the archive in one of the following ways:
0 Hash both archive and its contents
1 Only hash contents of the archive
2 Only hash archive itself (treat like a regular file)
-ud, --update-dat Output updated DAT (rebuild only)
Once the files that were able to rebuilt are taken care of, a DAT of the files
that could not be matched will be output to the program directory.
2016-09-21 20:16:57 -07:00
** Section 3.0 - Removed Commands
In this section, there is a list of commands that have been retired and will no longer work.
One thing to note is that not all commands will have a full description and are often copied
from the original help text from the program. Also worth noting is that some of them were used
in conjuction with other flags that may still exist and are thus not listed below.
Below are originally from DATabaseTwo (Standalone) -
-h, -?, --help Show this help dialog
-g, --generate Start tool in generate mode
-ga, --generate-all Start tool in generate all mode
-i, --ignore Don't prompt for new sources
-lsy, --list-systems List all systems (id <= name)
-nr, --no-rename Don't auto-rename games by source/system
-o, --old Output DAT in CMP format instead of XML
-sys=, --system= System ID to generate from
Below are originally from DATFromDir (Standalone) -
-h, -?, --help Show this help dialog
-m, --noMD5 Don't include MD5 in output
-nm, --noMD5 Don't include MD5 in output
-ns, --noSHA1 Don't include SHA1 in output
-b, --bare Don't include date in file name
-u, --unzip Force unzipping in created DAT
-f, --files Treat archives as files
-o, --old Output DAT in CMP format instead of XML
-gz, --gz-files Allow reading of GZIP files as archives
-ro, --romba Read files from a Romba input
-n=, --name= Set the internal name of the DAT
-d=, --desc= Set the filename and description of the DAT
-c=, --cat= Set the category of the DAT
-v=, --version= Set the version of the DAT
-au=, --author= Set the author of the DAT
-sd, --superdat Enable SuperDAT creation
-t=, --temp= Set the temporary directory to use
Below are originally from DatToMiss (Standalone) -
-h, -?, --help Show this help dialog
-l, --log Enable log to file
-r, --roms Output roms to miss instead of sets
-pre=, --prefix= Set prefix to be printed in front of all lines
-post=, --postfix= Set postfix to be printed behind all lines
Below are originally from Filter (Standalone) -
-h, -?, --help Show this help dialog
-out=, --out= Output directory
-gn=, --game-name= Game name to be filtered on
-rn=, --rom-name= Rom name to be filtered on
-rt=, --rom-type= Rom type to be filtered on
-sgt=, --greater= Size greater than or equal to
-slt=, --less= Size less than or equal to
-seq=, --equal= Size equal to
-crc=, --crc= CRC to be filtered on
-md5=, --md5= MD5 to be filtered on
-sha1=, --sha1= SHA-1 to be filtered on
-nd, --nodump Only match nodump roms
-nnd, --not-nodump Exclude all nodump roms
Game name, Rom name, CRC, MD5, SHA-1 can do partial matches using asterisks as
follows (case insensitive):
*00 means ends with '00'
00* means starts with '00'
*00* means contains '00'
00 means exactly equals '00'
Below are originally from HashSplit (Standalone) -
-h, -?, --help Show this help dialog
-out= Output directory
Below are originally from Headerer (Standalone) -
-?, -h, -help Show the built-in help text
Built-in to most of the programs is a basic help text
-e, -extract Enable detect and remove mode
This mode allows the user to detect, store, and remove copier headers from a file
or folder of files. The headers are backed up and collated by the hash of the un-
headered file. Files are then output without the detected copier header alongside
the originals with the suffix .new. No input files are altered in the process.
-r, -restore Restore headers to file(s)
This mode uses stored copier headers and reapplies them to files if they match the
included hash. More than one header can be applied to a file, so they will be out-
put to new files, suffixed with .newX, where X is a number. No input files are
altered in the process.
Below are originally from MergeDAT / DiffDat (Standalone) -
-h, -?, --help Show this help dialog
-l, --log Enable log to file
-d, --diff Switch to diffdat mode
-di, --diff Switch to diffdat mode
-m, --merge Enable deduping in the created DAT
-dd, --dedup Enable deduping in the created DAT
-b, --bare Don't include date in file name
-u, --unzip Force unzipping in created DAT
-o, --old Output DAT in CMP format instead of XML
-n=, --name= Set the name of the DAT
-d=, --desc= Set the description of the DAT
-c=, --cat= Set the category of the DAT
-v=, --version= Set the version of the DAT
-a=, --author= Set the author of the DAT");
Below are originally from OfflineMerge (Standalone) -
-h, -?, --help Show this help dialog
-f, --fake Replace all hashes and sizes by the default
-Inputs:
-com= Complete current DAT
-fix= Complete current Missing
-new= New Complete DAT
com= Complete current DAT
fix= Complete current Missing
new= New Complete DAT
This program will output the following DATs:
(a) Net New - (NewComplete)-(Complete)
(b) Unneeded - (Complete)-(NewComplete)
(c) New Missing - (Net New)+(Missing-(Unneeded))
(d) Have - (NewComplete)-(New Missing)
OR (Complete or NewComplete)-(Missing) if one is missing");
Below are originally from SabreTools / DATabase -
-a, --add Add a new system or source to the database
Add a new system or source to the DAT database, including additional information.
manu= Manufacturer name (system only)
system= System name (system only)
source= Source name (source only)
url= URL (source only)
-manu= Manufacturer name
Used only when adding a system to the database
-system= System name
Used only when adding a system to the database
-source= Source name
Used only when adding a source to the database
-url= Source URL
Used only when adding a source to the database
-ad, --all-diff Enable output of all diff variants
-c=, --cat= Set the category of the DAT
-cc, --convert-cmp Convert any DAT to CMP
-clean Clean game names according to WoD standards
-out= Output directory
out= Output directory
-cm, --convert-miss Convert from DAT to miss
-r, --roms Output roms to miss instead of sets
-gp, --game-prefix Add game name as a prefix to each item
-pre=, --prefix= Set prefix to be printed in front of all lines
-post=, --postfix= Set postfix to be printed behind all lines
-q, --quotes Put double-quotes around each item
-ae=, --add-ext= Add an extension to each item
-re=, --rep-ext= Replace all extensions with specified
-ro, --romba Output roms in Romba format (requires SHA-1)
-tsv, --tsv Output roms in Tab-Separated Value format
-cs, --convert-sd Convert any DAT to SabreDAT
-clean Clean game names according to WoD standards
-out= Output directory
-cr, --convert-rc Convert any DAT to RomCenter
-clean Clean game names according to WoD standards
-out= Output directory
-cr, --convert-rv Convert an XML DAT to RV
out= Output directory
-cx, --convert-xml Convert a RV DAT to XML
out= Output directory
-cx, --convert-xml Convert any DAT to XML
-clean Clean game names according to WoD standards
-out= Output directory
out= Output directory
-d=, --desc= Set the description of the DAT
Set the description for the output DAT(s) [default is the folder name]
-dp, -dfdp Create a DAT from each input directory using parallel code
Create a DAT file from an input directory or set of files. By default, this will
output a DAT named based on the input directory and the current date. It will also
treat all archives as possible games and add all three hashes for each file.
-nm, -noMD5 Don't include MD5 in output
This allows the user to skip calculating the MD5 for each of the files which will
speed up the creation of the DAT.
-ns, -noSHA1 Don't include SHA1 in output
This allows the user to skip calculating the SHA-1 for each of the files which will
speed up the creation of the DAT.
-b, -bare Don't include date in file name
Normally, the DAT will be created with the date in the file name. This flag removes
that but keeps the date tag intact.
-u, -unzip Force unzipping in created DAT
This sets the 'forcepacking="unzip"' flag in the outputted DAT. When used with a
file manager that allows for it, this will force the outputted files to be in
subdirectories instead of archives.
-f, -files Treat archives as files
Instead of trying to enumerate the files within archives, treat the archives as
files themselves. This is good for uncompressed sets that include archives that
should be read as-is.
2016-09-28 10:28:10 -07:00
-oc, --output-cmp Output in CMP format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to clrmamepro format
2016-09-28 10:28:10 -07:00
-om, --output-miss Output in Missfile format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to GoodTools miss format
2016-09-28 10:28:10 -07:00
-omd5, --output-md5 Output in MD5 format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to MD5 format
2016-09-28 10:28:10 -07:00
-or, --output-rc Output in RomCenter format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to RomCenter format
2016-09-28 10:28:10 -07:00
-os, --output-sd Output in SabreDAT format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to SabreDAT XML format
2016-09-28 10:28:10 -07:00
-osfv, --output-sfv Output in SFV format
2016-09-21 20:16:57 -07:00
Add outputting the created DAT to SFV format
-osha1, -output-sha1 Output in SHA-1 format
Add outputting the created DAT to SHA1 format
-ox, -output-xml Output in Logiqx XML format (default)
Add outputting the created DAT to Logiqx XML format
-gz, -gz-files Allow reading of GZIP files as archives
Since GZip files are not commonly used for file storage, this flag allows for
any GZip archives to have their contents hashed instead.
-ro, -romba Read files from a Romba input
Allow for reading of GZipped files as if they were from a Romba depot. This
implies that the files will be in the TorrentGZ format as well, including
naming convention.
-f=, -filename= Set the external name of the DAT
Set the base filename for the output DAT(s) [default is folder name plus date]
-n=, -name= Set the internal name of the DAT
Set the internal name for the output DAT(s) [default is folder name plus date]
-de=, -desc= Set the description of the DAT
Set the description for the output DAT(s) [default is the folder name]
-c=, -cat= Set the category of the DAT
Set the category for the output DAT(s) [default is blank]
-v=, -version= Set the version of the DAT
Set the version for the output DAT(s) [default is blank]
-au=, -author= Set the author of the DAT
Set the author for the output DAT(s) [default is blank]
-sd, -superdat Enable SuperDAT creation
Set the type flag to "SuperDAT" for the output DAT as well as preserving the
directory structure of the inputted folder, if applicable
-t=, -temp= Set the name of the temporary directory
Optionally, a temp folder can be supplied in the case the default temp directory
(inside the running folder) is not preferred. This is used for any operations
that require an archive to be extracted.
-mt={4} Amount of threads to use
Optionally, set the number of threads to use for the multithreaded operations.
The default is 4 threads; -1 means unlimited threads created.
exta= First extension to split by
extb= Second extension to split by
-f, --filter Filter DATs by inputted criteria
-out=, --out= Output directory
-gn=, --game-name= Game name to be filtered on
-rn=, --rom-name= Rom name to be filtered on
-rt=, --rom-type= Rom type to be filtered on
-sgt=, --greater= Size greater than or equal to
-slt=, --less= Size less than or equal to
-seq=, --equal= Size equal to
-crc=, --crc= CRC to be filtered on
-md5=, --md5= MD5 to be filtered on
-sha1=, --sha1= SHA-1 to be filtered on
-nd, --nodump Only match nodump roms
-nnd, --not-nodump Exclude all nodump roms
-g, -generate Start tool in generate mode
This starts the tool in DATabase generate mode. This will allow for creation of
managed DATs based on the inputted systems and sources as defined by other flags.
-system= Comma-separated list of system IDs
-source= Comma-separated list of source IDs
-out= Output directory
-system= System ID to generate from
Set the system ID to be used to create an output DAT
-nr, -no-rename Don't auto-rename games
By default, games are automatically renamed with the source (for system-derived
DATs), system (for source-derived DATs), or both (for the complete merged DAT).
This flag disables the automatic renaming and uses the game names as they are.
-o, -old Output DAT in CMP format instead of XML
As a holdover from only two output formats, this tool defaults to Logiqx XML
DAT outputs. If this flag is enabled, a clrmamepro DAT will be created instead.
-ga, -generate-all Start tool in generate all mode
This starts the tool in DATabase generate all mode. This will allow for creation of
managed DATs based on the entire DAT folder.
-nr, -no-rename Don't auto-rename games
By default, games are automatically renamed with the source (for system-derived
DATs), system (for source-derived DATs), or both (for the complete merged DAT).
This flag disables the automatic renaming and uses the game names as they are.
-o, -old Output DAT in CMP format instead of XML
As a holdover from only two output formats, this tool defaults to Logiqx XML
DAT outputs. If this flag is enabled, a clrmamepro DAT will be created instead.
-old, --romvault Produce a DAT in RV format
-i, -import Start tool in import mode
This starts the tool in DATabase import mode. This will allow for hashing of new
DAT files in the dats folder. If a source for the DAT cannot be automatically
determined, the user will be promted to select a source or enter a new one.
-ig, -ignore Don't prompt for new sources
If a source cannot be determined, then use the "Default" source instead of
asking the user.
-l, --log Enable logging of program output
-lso, -list-sources List all sources (id <= name)
List all sources in the database, ordered by the internal ID and mapped to the name
-lsy, -list-systems List all systems (id <= name)
List all systems in the database, ordered by the internal ID and mapped to the name
-nd, --nodump Include only match nodump roms
-nnd, --not-nodump Exclude all nodump roms
These flags allow for filtering based on the nodump status in the source DAT(s)
-ol, --offmerge Update DATS for offline arrays
This is a power user tool for dealing with offline arrays, specifically creating
have and miss lists without having to reconnect drives.
This option will output the following DATs based on the inputs as defined below:
(a) Net New = (New Complete)-(Current Complete)
(b) Unneeded (Current Complete)-(New Complete)
(c) New Missing (Net New)+(Current Missing-(Unneeded))
(d) Have (New Complete)-(New Missing)
OR (Complete or NewComplete) (Missing) if one is missing
-com= Complete current DAT
This is a merged and deduped DAT that includes all DATs that should be checked
-fix= Complete current Missing
This is a merged and deduped DAT that includes all fixdats representing missing
files
-new= New Complete DAT
This is a merged and deduped DAT that includes all current and updated DATs that
should be checked
-fk, -fake Replace all hashes and sizes by the default
For further offline management, this option will replace all hashes in the output
DATs to be their 0-byte equivalents. This allows for file managers to be used
to still deal with the DATs without reconnecting the drives
out= Output directory
-rm, -remove Remove a system or source from the database
Remove a system or source to the DAT database so it can no longer be used
-system= System ID
Internal ID of the system to be removed
-source= Source ID
Internal ID of the source to be removed
system= Comma-separated list of system IDs
source= Comma-separated list of source IDs
-tm, --trim-merge Consolidate DAT into a single game and trim entries
In the cases where files will have too long a name, this allows for trimming
the name of the files to the NTFS maximum length at most
-rd=, --root-dir= Set the root directory for trimming calculation
In the case that the files will not be stored from the root directory,
a new root can be set for path length calculations
-nr, --no-rename Keep game names instead of using '!'
By default, this will rename games to a single "!" character. This flag enables
the original game names to be kept
-df, --disable-force Disable forceunzipping
By default, this will set the forcepacking="unzip" flag on the output DAT. This
flag disables this output
2016-10-03 15:05:07 -07:00
-u, --unzip Force unzipping in created DAT
This sets the 'forcepacking="unzip"' flag in the outputted DAT. When used with a
file manager that allows for it, this will force the outputted files to be in
subdirectories instead of archives.
2016-09-21 20:16:57 -07:00
Below are originally from SingleGame (Standalone) -
-r=rootdir Set the directory name for path size
-rd=, --root-dir= Set the directory name for path size
-l, --log Enable logging to file
-nr, --no-rename Disable single-game mode
-df, --disable-force Disable forceunzipping
Below are originally from TGZConvert / TGZTest (Standalone) -
-?, -h, -help Show the built-in help text
Built-in to most of the programs is a basic help text
-out= Set the name of the output directory
This sets an output folder to be used by when files are rebuilt. It also serves
as the base folder if Romba mode is enabled. See -romba for more details
-t=, -temp= Set the name of the temporary directory
Optionally, a temp folder can be supplied in the case the default temp directory
(inside the running folder) is not preferred. This is used for any operations that
require an archive to be extracted.
-d, -delete Enable deletion of the input files
Optionally, the input files, once processed, can be deleted. This can be useful
when the original file structure is no longer needed or if there is limited space
on the source drive.
-r, -romba Enable Romba depot directory output
Optionally, this outputs the TGZ files into directories based on the structure
used by Romba. This uses nested folders using the first 4 bytes of the SHA-1,
1 byte for each layer of the directory name. It also includes two auxilary
files, .romba_size and .romba_size.backup, that have the compressed size of the
folder inside for use with Romba.
-7z={0} Set scanning level for 7z archives
-gz={2} Set scanning level for GZip archives
-rar={2} Set scanning level for RAR archives
-zip={0} Set scanning level for ZIP archives
For each of the major archive types recognized by the libraries used by this
program, scan the archive in one of the following ways:
0 Hash both archive and its contents
1 Only hash contents of the archive
2 Only hash archive itself (treat like a regular file)
** Section 4.0 - Examples
2016-09-12 14:40:16 -07:00
Here, any user-requested examples will go
2016-09-21 20:16:57 -07:00
** Section 5.0 - Contributors
2016-09-10 09:31:17 -07:00
Programmer / Lead: Matt Nadareski (darksabre76)
Additional code: emuLOAD, @tractivo
Testing: emuLOAD, @tractivo, Kludge, Obiwantje, edc
Suggestions: edc, AcidX, Amiga12, EliUmniCk
Based on work by: The Wizard of DATz
We welcome any contributors for coding, suggestions, optimizations, critisism,
heckling, abject anger, praise, and/or apathy.
2016-09-21 20:16:57 -07:00
** Section 6.0 - Licensing
2016-09-10 09:31:17 -07:00
This program uses, in part or in whole, code, libraries, and/or applications
from the 7-zip project (www.7-zip.org). 7-zip is licenced under the GNU LGPL.
2016-09-15 12:16:33 -07:00
The preceeding programs use, in part or in whole, code, libraries, and/or applications from DotNetSharp (https://dotnetzip.codeplex.com). DotNetSharp is licensed under the Microsoft Public License of October 2006.
2016-09-10 09:31:17 -07:00
All other external code is marked as such within the source and correctly
attributed to the site and/or person(s) that originally wrote the code. All
code written by project members is licensed under GPL v3. See LICENSE for
more details.
