Files

99 lines
3.3 KiB
C#
Raw Permalink Normal View History

using System.IO;
2026-08-27 13:57:49 -04:00
using Nanook.GrindCore;
using Nanook.GrindCore.DeflateZLib;
using SabreTools.IO.Extensions;
using static SabreTools.Data.Models.GCZ.Constants;
Dolphin lib (#85) * Add GCZ, WIA/RVZ, and NintendoDisc (GameCube/Wii) format support Port of DolphinIsoLib into SabreTools.Serialization architecture: - Data.Models: GCZ/, WIA/, NintendoDisc/ model subdirectories (15 files) - Serialization.Readers: GCZ, WIA, NintendoDisc readers - Serialization.Writers: GCZ, WIA writers (structural metadata; full round-trip TODO) - Wrappers: NintendoDisc, GCZ, WIA wrappers with Encryption partial class - Wrappers: WiaRvzCompressionHelper (BZip2/LZMA/LZMA2/Zstd, net462+ guarded) - WrapperType + WrapperFactory: GCZ, WIA, NintendoDisc entries added GetInnerWrapper() decompression and NintendoDisc.Extraction FST extraction are stubbed with TODO comments pending full implementation. * Implement GetInnerWrapper for GCZ and WIA, full NintendoDisc extraction * Add GCZ/WIA/RVZ write pipeline and Nintendo disc compression helpers * Add WIA/RVZ table decompression, NintendoDisc/GCZ printing, NintendoDisc detection for .iso files * Fix NintendoDisc header layout, GC magic, and add embedded disc header to WIA/GCZ printing - Fix GCMagicWord: 0xC23D3C1F -> 0xC2339F3D (confirmed from Dolphin DiscUtils.h) - Fix disc header field layout to match Dolphin's confirmed offsets: MakerCode is bytes 4-5 of the 6-char GameId (no separate field at 0x006), DiscNumber at 0x006, DiscVersion at 0x007, unused region is 14 bytes (0x00A-0x017) - Update NintendoDisc reader: derive MakerCode from GameId[4..5], fix skip count - Add ParseDiscHeaderOnly() to reader for partial (short) stream parsing - Guard DisableHash/DisableEnc reads at the 0x080 boundary for 128-byte embedded headers - Guard DOL/FST skip for streams shorter than full 0x440 boot block - Fix WrapperFactory: NintendoDisc magic detection now precedes .iso -> ISO9660 fallback - Add GameId-prefix heuristic in WrapperFactory for GC discs lacking magic word - Add GameId-prefix platform fallback in reader for GC discs without GCMagicWord - Add DiscHeader property to WIA wrapper (parsed from Header2.DiscHeader bytes) - Add DiscHeader property to GCZ wrapper (decompresses first block only) - Add ReadDiscHeader() helper to GCZ for lightweight first-block decompression - Print embedded disc header (Game ID, Maker, Disc/Rev, Title) in WIA.Printing.cs and GCZ.Printing.cs * Fix Wii partition extraction: correct IV, FST size shift, partition naming - Block decryption: IV is at raw block offset 0x3D0 (still-encrypted), matching Dolphin/DolphinIsoLib WiiPartitionDecryptor.DecryptBlock exactly. - FST size field at boot.bin 0x428 is also stored >>2 on Wii; apply <<2 to get true byte size. - Partition folder naming now matches DolphinIsoLib WiiDiscExtractor exactly: type 0->GM+n, 1->UP+n, 2->CH+n, printable ASCII unknown->raw 4-char string, non-printable->P{index}. SSBB VC channels extract as HA8E, HA9E, etc. - ExtractionTool peek buffer increased from 16 to 32 bytes. Verified: SSBB GM0 extracts 5524 files, boot.bin/fst.bin byte-identical to Dolphin reference extraction. * Fix FST extraction: create zero-byte files instead of skipping them Files with fileSize=0 in the FST were silently skipped. Now they are created as empty files, matching Dolphin/DolphinIsoLib behavior. Verified: SSBB now extracts 5958 files with 0 missing, 0 extra, 0 size mismatches, and 0 hash mismatches vs DolphinIsoLib reference. * Add GCZ/WIA/RVZ virtual stream extraction via NintendoDisc wrapper * Address PR #85 review comments (Copilot + mnadareski) * Address PR #85 review comments * Replace custom endian helpers and SHA1 with SabreTools.IO equivalents * Update GCZ.Printing.cs * Update NintendoDisc.Printing.cs * Update WIA.Printing.cs * Add WIA/RVZ Wii partition crypto round-trip support - Add AesCbc internal helper (BouncyCastle AES-CBC encrypt/decrypt) - Add NintendoDisc.CommonKeyProvider hook for injectable test keys - Fix sha1.Terminate() missing in all three ComputeSha1 helpers in WIA.cs - Fix Wii partition dataOff alignment to 0x8000 boundary - Add WIA.EncryptWiiGroup (internal) for re-encrypting plaintext groups - Add WIA.DumpIso to WIA.Writing.cs (WIA/RVZ -> flat ISO conversion) - Add WiaVirtualStream on-demand group decompression - Add _preDecryptedReader bypass on NintendoDisc for WIA extraction path - Add WIATests.cs with Wii crypto round-trip test using synthetic data - Move DumpIso from WIA.Extraction.cs to WIA.Writing.cs - Bump DumpIso read buffer from 1 MiB to 2 MiB (aligns to WIA chunk size) - Add InternalsVisibleTo SabreTools.Wrappers.Test in csproj * Remove hardcoded Wii common keys from NintendoDisc.Encryption - Delete the embedded WiiCommonKeyRetail and WiiCommonKeyKorean byte arrays from NintendoDisc.Encryption.cs. - Make CommonKeyProvider public so any caller (not just tests) can inject keys; DecryptTitleKey now returns null when no key is available for the requested index rather than falling back to hardcoded values. - Add NintendoDiscEncryptionTests.cs: - Argument guard and no-provider tests for DecryptTitleKey. - Fake-key round-trip test (encrypt then decrypt with injected key). - Integration test that reads TestData/NintendoDisc/keys.json, verifies each key against hardcoded SHA256 constants, and skips silently if the file is absent or the keys do not match. - LoadKeyProvider helper (named JSON format, index-keyed). - Add [Collection(NintendoDisc)] to both NintendoDiscEncryptionTests and WIATests to prevent parallel access to the static CommonKeyProvider from racing between test classes. - Add TestData/NintendoDisc/keys.json.example documenting the expected key file format. - Add Newtonsoft.Json reference to SabreTools.Wrappers.Test.csproj. * Didn't actually commit the changes. My bad. Fixed. * Edited a comment * Added in XUnit outputs that show up in Test Viewer in VS --------- Co-authored-by: Matt Nadareski <mnadareski@outlook.com>
2026-05-12 09:41:32 -05:00
namespace SabreTools.Wrappers
{
public partial class GCZ : IExtractable
{
/// <inheritdoc/>
public bool Extract(string outputDirectory, bool includeDebug)
{
var inner = GetInnerWrapper();
if (inner is null)
return false;
return inner.Extract(outputDirectory, includeDebug);
Dolphin lib (#85) * Add GCZ, WIA/RVZ, and NintendoDisc (GameCube/Wii) format support Port of DolphinIsoLib into SabreTools.Serialization architecture: - Data.Models: GCZ/, WIA/, NintendoDisc/ model subdirectories (15 files) - Serialization.Readers: GCZ, WIA, NintendoDisc readers - Serialization.Writers: GCZ, WIA writers (structural metadata; full round-trip TODO) - Wrappers: NintendoDisc, GCZ, WIA wrappers with Encryption partial class - Wrappers: WiaRvzCompressionHelper (BZip2/LZMA/LZMA2/Zstd, net462+ guarded) - WrapperType + WrapperFactory: GCZ, WIA, NintendoDisc entries added GetInnerWrapper() decompression and NintendoDisc.Extraction FST extraction are stubbed with TODO comments pending full implementation. * Implement GetInnerWrapper for GCZ and WIA, full NintendoDisc extraction * Add GCZ/WIA/RVZ write pipeline and Nintendo disc compression helpers * Add WIA/RVZ table decompression, NintendoDisc/GCZ printing, NintendoDisc detection for .iso files * Fix NintendoDisc header layout, GC magic, and add embedded disc header to WIA/GCZ printing - Fix GCMagicWord: 0xC23D3C1F -> 0xC2339F3D (confirmed from Dolphin DiscUtils.h) - Fix disc header field layout to match Dolphin's confirmed offsets: MakerCode is bytes 4-5 of the 6-char GameId (no separate field at 0x006), DiscNumber at 0x006, DiscVersion at 0x007, unused region is 14 bytes (0x00A-0x017) - Update NintendoDisc reader: derive MakerCode from GameId[4..5], fix skip count - Add ParseDiscHeaderOnly() to reader for partial (short) stream parsing - Guard DisableHash/DisableEnc reads at the 0x080 boundary for 128-byte embedded headers - Guard DOL/FST skip for streams shorter than full 0x440 boot block - Fix WrapperFactory: NintendoDisc magic detection now precedes .iso -> ISO9660 fallback - Add GameId-prefix heuristic in WrapperFactory for GC discs lacking magic word - Add GameId-prefix platform fallback in reader for GC discs without GCMagicWord - Add DiscHeader property to WIA wrapper (parsed from Header2.DiscHeader bytes) - Add DiscHeader property to GCZ wrapper (decompresses first block only) - Add ReadDiscHeader() helper to GCZ for lightweight first-block decompression - Print embedded disc header (Game ID, Maker, Disc/Rev, Title) in WIA.Printing.cs and GCZ.Printing.cs * Fix Wii partition extraction: correct IV, FST size shift, partition naming - Block decryption: IV is at raw block offset 0x3D0 (still-encrypted), matching Dolphin/DolphinIsoLib WiiPartitionDecryptor.DecryptBlock exactly. - FST size field at boot.bin 0x428 is also stored >>2 on Wii; apply <<2 to get true byte size. - Partition folder naming now matches DolphinIsoLib WiiDiscExtractor exactly: type 0->GM+n, 1->UP+n, 2->CH+n, printable ASCII unknown->raw 4-char string, non-printable->P{index}. SSBB VC channels extract as HA8E, HA9E, etc. - ExtractionTool peek buffer increased from 16 to 32 bytes. Verified: SSBB GM0 extracts 5524 files, boot.bin/fst.bin byte-identical to Dolphin reference extraction. * Fix FST extraction: create zero-byte files instead of skipping them Files with fileSize=0 in the FST were silently skipped. Now they are created as empty files, matching Dolphin/DolphinIsoLib behavior. Verified: SSBB now extracts 5958 files with 0 missing, 0 extra, 0 size mismatches, and 0 hash mismatches vs DolphinIsoLib reference. * Add GCZ/WIA/RVZ virtual stream extraction via NintendoDisc wrapper * Address PR #85 review comments (Copilot + mnadareski) * Address PR #85 review comments * Replace custom endian helpers and SHA1 with SabreTools.IO equivalents * Update GCZ.Printing.cs * Update NintendoDisc.Printing.cs * Update WIA.Printing.cs * Add WIA/RVZ Wii partition crypto round-trip support - Add AesCbc internal helper (BouncyCastle AES-CBC encrypt/decrypt) - Add NintendoDisc.CommonKeyProvider hook for injectable test keys - Fix sha1.Terminate() missing in all three ComputeSha1 helpers in WIA.cs - Fix Wii partition dataOff alignment to 0x8000 boundary - Add WIA.EncryptWiiGroup (internal) for re-encrypting plaintext groups - Add WIA.DumpIso to WIA.Writing.cs (WIA/RVZ -> flat ISO conversion) - Add WiaVirtualStream on-demand group decompression - Add _preDecryptedReader bypass on NintendoDisc for WIA extraction path - Add WIATests.cs with Wii crypto round-trip test using synthetic data - Move DumpIso from WIA.Extraction.cs to WIA.Writing.cs - Bump DumpIso read buffer from 1 MiB to 2 MiB (aligns to WIA chunk size) - Add InternalsVisibleTo SabreTools.Wrappers.Test in csproj * Remove hardcoded Wii common keys from NintendoDisc.Encryption - Delete the embedded WiiCommonKeyRetail and WiiCommonKeyKorean byte arrays from NintendoDisc.Encryption.cs. - Make CommonKeyProvider public so any caller (not just tests) can inject keys; DecryptTitleKey now returns null when no key is available for the requested index rather than falling back to hardcoded values. - Add NintendoDiscEncryptionTests.cs: - Argument guard and no-provider tests for DecryptTitleKey. - Fake-key round-trip test (encrypt then decrypt with injected key). - Integration test that reads TestData/NintendoDisc/keys.json, verifies each key against hardcoded SHA256 constants, and skips silently if the file is absent or the keys do not match. - LoadKeyProvider helper (named JSON format, index-keyed). - Add [Collection(NintendoDisc)] to both NintendoDiscEncryptionTests and WIATests to prevent parallel access to the static CommonKeyProvider from racing between test classes. - Add TestData/NintendoDisc/keys.json.example documenting the expected key file format. - Add Newtonsoft.Json reference to SabreTools.Wrappers.Test.csproj. * Didn't actually commit the changes. My bad. Fixed. * Edited a comment * Added in XUnit outputs that show up in Test Viewer in VS --------- Co-authored-by: Matt Nadareski <mnadareski@outlook.com>
2026-05-12 09:41:32 -05:00
}
#region Inner Wrapper
/// <summary>
/// Returns a NintendoDisc wrapper backed by a virtual stream that decompresses
/// GCZ blocks on demand, avoiding loading the entire ISO into memory.
/// </summary>
public NintendoDisc? GetInnerWrapper()
{
if (BlockPointers.Length == 0)
return null;
if (Header.DataSize == 0)
return null;
var stream = new GczVirtualStream(this);
return NintendoDisc.Create(stream);
}
/// <summary>
/// Decompresses a single GCZ block by index and returns its raw bytes.
/// Returns null on failure; returns a zero-filled block if the compressed size is zero.
/// </summary>
internal byte[]? DecompressBlock(int blockIndex)
{
if (blockIndex < 0 || blockIndex >= BlockPointers.Length)
return null;
ulong ptr = BlockPointers[blockIndex];
long blockFileOffset = DataOffset + (long)(ptr & ~UncompressedFlag);
ulong nextRaw = (blockIndex + 1 < BlockPointers.Length)
? BlockPointers[blockIndex + 1] & ~UncompressedFlag
: CompressedDataSize;
int compSize = (int)(nextRaw - (ptr & ~UncompressedFlag));
if (compSize <= 0)
return new byte[BlockSize];
byte[] raw = ReadRangeFromSource(blockFileOffset, compSize);
if (raw.Length != compSize)
return null;
// Verify Adler-32 checksum on the compressed (raw) data before decompressing
if (BlockHashes is not null && blockIndex < BlockHashes.Length)
{
2026-08-27 13:57:49 -04:00
// TODO: Replace with call to Hashing library
uint actual = IO.Compression.Deflate.Adler.Adler32(1, raw, 0, raw.Length);
if (actual != BlockHashes[blockIndex])
return null;
}
// If the data is raw, just return
if ((ptr & UncompressedFlag) != 0)
return raw;
// GCZ blocks are zlib-framed: 2-byte header + deflate data + 4-byte Adler-32 trailer.
// Strip the frame and feed raw deflate data to DeflateStream.
if (raw.Length < 6)
return null;
try
{
using var cs = new MemoryStream(raw, 2, raw.Length - 6);
2026-08-27 13:57:49 -04:00
using var ds = new DeflateStream(cs, new CompressionOptions { Type = CompressionType.Decompress });
using var os = new MemoryStream();
ds.BlockCopy(os, blockSize: 4096);
return os.ToArray();
}
catch
{
return null;
}
}
#endregion
Dolphin lib (#85) * Add GCZ, WIA/RVZ, and NintendoDisc (GameCube/Wii) format support Port of DolphinIsoLib into SabreTools.Serialization architecture: - Data.Models: GCZ/, WIA/, NintendoDisc/ model subdirectories (15 files) - Serialization.Readers: GCZ, WIA, NintendoDisc readers - Serialization.Writers: GCZ, WIA writers (structural metadata; full round-trip TODO) - Wrappers: NintendoDisc, GCZ, WIA wrappers with Encryption partial class - Wrappers: WiaRvzCompressionHelper (BZip2/LZMA/LZMA2/Zstd, net462+ guarded) - WrapperType + WrapperFactory: GCZ, WIA, NintendoDisc entries added GetInnerWrapper() decompression and NintendoDisc.Extraction FST extraction are stubbed with TODO comments pending full implementation. * Implement GetInnerWrapper for GCZ and WIA, full NintendoDisc extraction * Add GCZ/WIA/RVZ write pipeline and Nintendo disc compression helpers * Add WIA/RVZ table decompression, NintendoDisc/GCZ printing, NintendoDisc detection for .iso files * Fix NintendoDisc header layout, GC magic, and add embedded disc header to WIA/GCZ printing - Fix GCMagicWord: 0xC23D3C1F -> 0xC2339F3D (confirmed from Dolphin DiscUtils.h) - Fix disc header field layout to match Dolphin's confirmed offsets: MakerCode is bytes 4-5 of the 6-char GameId (no separate field at 0x006), DiscNumber at 0x006, DiscVersion at 0x007, unused region is 14 bytes (0x00A-0x017) - Update NintendoDisc reader: derive MakerCode from GameId[4..5], fix skip count - Add ParseDiscHeaderOnly() to reader for partial (short) stream parsing - Guard DisableHash/DisableEnc reads at the 0x080 boundary for 128-byte embedded headers - Guard DOL/FST skip for streams shorter than full 0x440 boot block - Fix WrapperFactory: NintendoDisc magic detection now precedes .iso -> ISO9660 fallback - Add GameId-prefix heuristic in WrapperFactory for GC discs lacking magic word - Add GameId-prefix platform fallback in reader for GC discs without GCMagicWord - Add DiscHeader property to WIA wrapper (parsed from Header2.DiscHeader bytes) - Add DiscHeader property to GCZ wrapper (decompresses first block only) - Add ReadDiscHeader() helper to GCZ for lightweight first-block decompression - Print embedded disc header (Game ID, Maker, Disc/Rev, Title) in WIA.Printing.cs and GCZ.Printing.cs * Fix Wii partition extraction: correct IV, FST size shift, partition naming - Block decryption: IV is at raw block offset 0x3D0 (still-encrypted), matching Dolphin/DolphinIsoLib WiiPartitionDecryptor.DecryptBlock exactly. - FST size field at boot.bin 0x428 is also stored >>2 on Wii; apply <<2 to get true byte size. - Partition folder naming now matches DolphinIsoLib WiiDiscExtractor exactly: type 0->GM+n, 1->UP+n, 2->CH+n, printable ASCII unknown->raw 4-char string, non-printable->P{index}. SSBB VC channels extract as HA8E, HA9E, etc. - ExtractionTool peek buffer increased from 16 to 32 bytes. Verified: SSBB GM0 extracts 5524 files, boot.bin/fst.bin byte-identical to Dolphin reference extraction. * Fix FST extraction: create zero-byte files instead of skipping them Files with fileSize=0 in the FST were silently skipped. Now they are created as empty files, matching Dolphin/DolphinIsoLib behavior. Verified: SSBB now extracts 5958 files with 0 missing, 0 extra, 0 size mismatches, and 0 hash mismatches vs DolphinIsoLib reference. * Add GCZ/WIA/RVZ virtual stream extraction via NintendoDisc wrapper * Address PR #85 review comments (Copilot + mnadareski) * Address PR #85 review comments * Replace custom endian helpers and SHA1 with SabreTools.IO equivalents * Update GCZ.Printing.cs * Update NintendoDisc.Printing.cs * Update WIA.Printing.cs * Add WIA/RVZ Wii partition crypto round-trip support - Add AesCbc internal helper (BouncyCastle AES-CBC encrypt/decrypt) - Add NintendoDisc.CommonKeyProvider hook for injectable test keys - Fix sha1.Terminate() missing in all three ComputeSha1 helpers in WIA.cs - Fix Wii partition dataOff alignment to 0x8000 boundary - Add WIA.EncryptWiiGroup (internal) for re-encrypting plaintext groups - Add WIA.DumpIso to WIA.Writing.cs (WIA/RVZ -> flat ISO conversion) - Add WiaVirtualStream on-demand group decompression - Add _preDecryptedReader bypass on NintendoDisc for WIA extraction path - Add WIATests.cs with Wii crypto round-trip test using synthetic data - Move DumpIso from WIA.Extraction.cs to WIA.Writing.cs - Bump DumpIso read buffer from 1 MiB to 2 MiB (aligns to WIA chunk size) - Add InternalsVisibleTo SabreTools.Wrappers.Test in csproj * Remove hardcoded Wii common keys from NintendoDisc.Encryption - Delete the embedded WiiCommonKeyRetail and WiiCommonKeyKorean byte arrays from NintendoDisc.Encryption.cs. - Make CommonKeyProvider public so any caller (not just tests) can inject keys; DecryptTitleKey now returns null when no key is available for the requested index rather than falling back to hardcoded values. - Add NintendoDiscEncryptionTests.cs: - Argument guard and no-provider tests for DecryptTitleKey. - Fake-key round-trip test (encrypt then decrypt with injected key). - Integration test that reads TestData/NintendoDisc/keys.json, verifies each key against hardcoded SHA256 constants, and skips silently if the file is absent or the keys do not match. - LoadKeyProvider helper (named JSON format, index-keyed). - Add [Collection(NintendoDisc)] to both NintendoDiscEncryptionTests and WIATests to prevent parallel access to the static CommonKeyProvider from racing between test classes. - Add TestData/NintendoDisc/keys.json.example documenting the expected key file format. - Add Newtonsoft.Json reference to SabreTools.Wrappers.Test.csproj. * Didn't actually commit the changes. My bad. Fixed. * Edited a comment * Added in XUnit outputs that show up in Test Viewer in VS --------- Co-authored-by: Matt Nadareski <mnadareski@outlook.com>
2026-05-12 09:41:32 -05:00
}
}