mirror of
https://github.com/SabreTools/SabreTools.Serialization.git
synced 2026-07-19 15:25:18 +00:00
49aa6895b6
* Add GCZ, WIA/RVZ, and NintendoDisc (GameCube/Wii) format support
Port of DolphinIsoLib into SabreTools.Serialization architecture:
- Data.Models: GCZ/, WIA/, NintendoDisc/ model subdirectories (15 files)
- Serialization.Readers: GCZ, WIA, NintendoDisc readers
- Serialization.Writers: GCZ, WIA writers (structural metadata; full round-trip TODO)
- Wrappers: NintendoDisc, GCZ, WIA wrappers with Encryption partial class
- Wrappers: WiaRvzCompressionHelper (BZip2/LZMA/LZMA2/Zstd, net462+ guarded)
- WrapperType + WrapperFactory: GCZ, WIA, NintendoDisc entries added
GetInnerWrapper() decompression and NintendoDisc.Extraction FST extraction
are stubbed with TODO comments pending full implementation.
* Implement GetInnerWrapper for GCZ and WIA, full NintendoDisc extraction
* Add GCZ/WIA/RVZ write pipeline and Nintendo disc compression helpers
* Add WIA/RVZ table decompression, NintendoDisc/GCZ printing, NintendoDisc detection for .iso files
* Fix NintendoDisc header layout, GC magic, and add embedded disc header to WIA/GCZ printing
- Fix GCMagicWord: 0xC23D3C1F -> 0xC2339F3D (confirmed from Dolphin DiscUtils.h)
- Fix disc header field layout to match Dolphin's confirmed offsets:
MakerCode is bytes 4-5 of the 6-char GameId (no separate field at 0x006),
DiscNumber at 0x006, DiscVersion at 0x007, unused region is 14 bytes (0x00A-0x017)
- Update NintendoDisc reader: derive MakerCode from GameId[4..5], fix skip count
- Add ParseDiscHeaderOnly() to reader for partial (short) stream parsing
- Guard DisableHash/DisableEnc reads at the 0x080 boundary for 128-byte embedded headers
- Guard DOL/FST skip for streams shorter than full 0x440 boot block
- Fix WrapperFactory: NintendoDisc magic detection now precedes .iso -> ISO9660 fallback
- Add GameId-prefix heuristic in WrapperFactory for GC discs lacking magic word
- Add GameId-prefix platform fallback in reader for GC discs without GCMagicWord
- Add DiscHeader property to WIA wrapper (parsed from Header2.DiscHeader bytes)
- Add DiscHeader property to GCZ wrapper (decompresses first block only)
- Add ReadDiscHeader() helper to GCZ for lightweight first-block decompression
- Print embedded disc header (Game ID, Maker, Disc/Rev, Title) in WIA.Printing.cs and GCZ.Printing.cs
* Fix Wii partition extraction: correct IV, FST size shift, partition naming
- Block decryption: IV is at raw block offset 0x3D0 (still-encrypted),
matching Dolphin/DolphinIsoLib WiiPartitionDecryptor.DecryptBlock exactly.
- FST size field at boot.bin 0x428 is also stored >>2 on Wii; apply <<2
to get true byte size.
- Partition folder naming now matches DolphinIsoLib WiiDiscExtractor exactly:
type 0->GM+n, 1->UP+n, 2->CH+n, printable ASCII unknown->raw 4-char string,
non-printable->P{index}. SSBB VC channels extract as HA8E, HA9E, etc.
- ExtractionTool peek buffer increased from 16 to 32 bytes.
Verified: SSBB GM0 extracts 5524 files, boot.bin/fst.bin byte-identical
to Dolphin reference extraction.
* Fix FST extraction: create zero-byte files instead of skipping them
Files with fileSize=0 in the FST were silently skipped. Now they are
created as empty files, matching Dolphin/DolphinIsoLib behavior.
Verified: SSBB now extracts 5958 files with 0 missing, 0 extra,
0 size mismatches, and 0 hash mismatches vs DolphinIsoLib reference.
* Add GCZ/WIA/RVZ virtual stream extraction via NintendoDisc wrapper
* Address PR #85 review comments (Copilot + mnadareski)
* Address PR #85 review comments
* Replace custom endian helpers and SHA1 with SabreTools.IO equivalents
* Update GCZ.Printing.cs
* Update NintendoDisc.Printing.cs
* Update WIA.Printing.cs
* Add WIA/RVZ Wii partition crypto round-trip support
- Add AesCbc internal helper (BouncyCastle AES-CBC encrypt/decrypt)
- Add NintendoDisc.CommonKeyProvider hook for injectable test keys
- Fix sha1.Terminate() missing in all three ComputeSha1 helpers in WIA.cs
- Fix Wii partition dataOff alignment to 0x8000 boundary
- Add WIA.EncryptWiiGroup (internal) for re-encrypting plaintext groups
- Add WIA.DumpIso to WIA.Writing.cs (WIA/RVZ -> flat ISO conversion)
- Add WiaVirtualStream on-demand group decompression
- Add _preDecryptedReader bypass on NintendoDisc for WIA extraction path
- Add WIATests.cs with Wii crypto round-trip test using synthetic data
- Move DumpIso from WIA.Extraction.cs to WIA.Writing.cs
- Bump DumpIso read buffer from 1 MiB to 2 MiB (aligns to WIA chunk size)
- Add InternalsVisibleTo SabreTools.Wrappers.Test in csproj
* Remove hardcoded Wii common keys from NintendoDisc.Encryption
- Delete the embedded WiiCommonKeyRetail and WiiCommonKeyKorean byte
arrays from NintendoDisc.Encryption.cs.
- Make CommonKeyProvider public so any caller (not just tests) can
inject keys; DecryptTitleKey now returns null when no key is
available for the requested index rather than falling back to
hardcoded values.
- Add NintendoDiscEncryptionTests.cs:
- Argument guard and no-provider tests for DecryptTitleKey.
- Fake-key round-trip test (encrypt then decrypt with injected key).
- Integration test that reads TestData/NintendoDisc/keys.json,
verifies each key against hardcoded SHA256 constants, and skips
silently if the file is absent or the keys do not match.
- LoadKeyProvider helper (named JSON format, index-keyed).
- Add [Collection(NintendoDisc)] to both NintendoDiscEncryptionTests
and WIATests to prevent parallel access to the static
CommonKeyProvider from racing between test classes.
- Add TestData/NintendoDisc/keys.json.example documenting the
expected key file format.
- Add Newtonsoft.Json reference to SabreTools.Wrappers.Test.csproj.
* Didn't actually commit the changes. My bad. Fixed.
* Edited a comment
* Added in XUnit outputs that show up in Test Viewer in VS
---------
Co-authored-by: Matt Nadareski <mnadareski@outlook.com>
SabreTools.Serialization
This meta-library comprises of serializers that both read and write from files and streams to the dedicated models as well as convert to and from the common internal models. This library is partially used by the current parsing and writing code but none of the internal model serialization is used.
Find the link to the Nuget package here.
Releases
For the most recent stable build, download the latest release here: Releases Page
For the latest WIP build here: Rolling Release
InfoPrint
InfoPrint is a reference implementation for the deserialization and printing features of the library, packaged as a standalone executable for all supported platforms. It will attempt to detect and display information about many supported file types, optionally both hashing the file and outputting the information to a JSON file.
InfoPrint <options> file|directory ...
Options:
-?, -h, --help Display this help text
-d, --debug Enable debug mode
-c, --hash Output file hashes
-f, --file Print to file only
-j, --json Print info as JSON
ExtractionTool
ExtractionTool is a reference implementation for the extraction features of the library, packaged as a standalone executable for all supported platforms. It will attempt to detect and extract many supported file types. See the table below for supported extraction functionality.
ExtractionTool <options> file|directory ...
Options:
-?, -h, --help Display this help text
-d, --debug Enable debug mode
-o, --outdir [PATH] Set output path for extraction (required)
| Format Name | Notes |
|---|---|
| 7-zip archive | .NET Framework 4.6.2 and greater |
| Atari 7800 (A78) Cart Image | Header and ROM data |
| Atari Lynx (LNX) Cart Image | Header and ROM data |
| BFPK custom archive format | |
| bzip2 archive | .NET Framework 4.6.2 and greater |
| Compound File Binary (CFB) | Only CFB common pieces extractable |
| fwNES FDS file | Header and disk data |
| gzip archive | |
| Half-Life Game Cache File (GCF) | |
| Half-Life Level (BSP) | |
| Half-Life Package File (PAK) | |
| Half-Life Texture Package File (WAD3) | |
| Half-Life 2 Level (VBSP) | |
| InstallShield Archive V3 (Z) | |
| InstallShield CAB | |
| ISO9660 | ISO and BIN files |
| Microsoft cabinet file | Does not support LZX or Quantum compression |
| Microsoft LZ-compressed files | KWAJ, QBasic, and SZDD variants |
| MoPaQ game data archive (MPQ) | Windows only |
| New Exectuable | Embedded archives and executables in the overlay and Wise installer |
| Nintendo Entertainment System (NES) Cart Image | Header, Trainer data, PRG-ROM, CHR-ROM, Unheadered ROM, and PlayChoice-10 INST-ROM |
| NovaLogic Game Archive Format (PFF) | |
| PKZIP and derived files (ZIP, etc.) | .NET Framework 4.6.2 and greater |
| Portable Executable | Embedded archives and executables in the resources and overlay, CExe-packed data, SFX archives (7-zip, PKZIP, and RAR), and Wise installer |
| Quantum archive (Q) | Currently not working |
| RAR archive (RAR) | .NET Framework 4.6.2 and greater |
| SGA game archive | |
| Tape archive (TAR) | |
| Valve Package File (VPK) | |
| XBox Package File (XZP) | |
| Xbox DVD Filesystem (XISO) | |
| xz archive (XZ) | .NET Framework 4.6.2 and greater |
| ZArchive |
Namespaces
Below are a list of the included namespaces with links to their README files:
- SabreTools.Data.Extensions
- SabreTools.Data.Models
- SabreTools.Metadata.DatFiles
- SabreTools.Metadata.DatItems
- SabreTools.Metadata.Filter
- SabreTools.Metadata.Reports
- SabreTools.ObjectIdentifier
- SabreTools.Serialization.CrossModel
- SabreTools.Serialization.Readers
- SabreTools.Serialization.Writers
- SabreTools.Wrappers