Files
SabreTools.Serialization/SabreTools.Wrappers/GCZ.Writing.cs
Dimensional 49aa6895b6 Dolphin lib (#85)
* Add GCZ, WIA/RVZ, and NintendoDisc (GameCube/Wii) format support

Port of DolphinIsoLib into SabreTools.Serialization architecture:
- Data.Models: GCZ/, WIA/, NintendoDisc/ model subdirectories (15 files)
- Serialization.Readers: GCZ, WIA, NintendoDisc readers
- Serialization.Writers: GCZ, WIA writers (structural metadata; full round-trip TODO)
- Wrappers: NintendoDisc, GCZ, WIA wrappers with Encryption partial class
- Wrappers: WiaRvzCompressionHelper (BZip2/LZMA/LZMA2/Zstd, net462+ guarded)
- WrapperType + WrapperFactory: GCZ, WIA, NintendoDisc entries added

GetInnerWrapper() decompression and NintendoDisc.Extraction FST extraction
are stubbed with TODO comments pending full implementation.

* Implement GetInnerWrapper for GCZ and WIA, full NintendoDisc extraction

* Add GCZ/WIA/RVZ write pipeline and Nintendo disc compression helpers

* Add WIA/RVZ table decompression, NintendoDisc/GCZ printing, NintendoDisc detection for .iso files

* Fix NintendoDisc header layout, GC magic, and add embedded disc header to WIA/GCZ printing

- Fix GCMagicWord: 0xC23D3C1F -> 0xC2339F3D (confirmed from Dolphin DiscUtils.h)
- Fix disc header field layout to match Dolphin's confirmed offsets:
  MakerCode is bytes 4-5 of the 6-char GameId (no separate field at 0x006),
  DiscNumber at 0x006, DiscVersion at 0x007, unused region is 14 bytes (0x00A-0x017)
- Update NintendoDisc reader: derive MakerCode from GameId[4..5], fix skip count
- Add ParseDiscHeaderOnly() to reader for partial (short) stream parsing
- Guard DisableHash/DisableEnc reads at the 0x080 boundary for 128-byte embedded headers
- Guard DOL/FST skip for streams shorter than full 0x440 boot block
- Fix WrapperFactory: NintendoDisc magic detection now precedes .iso -> ISO9660 fallback
- Add GameId-prefix heuristic in WrapperFactory for GC discs lacking magic word
- Add GameId-prefix platform fallback in reader for GC discs without GCMagicWord
- Add DiscHeader property to WIA wrapper (parsed from Header2.DiscHeader bytes)
- Add DiscHeader property to GCZ wrapper (decompresses first block only)
- Add ReadDiscHeader() helper to GCZ for lightweight first-block decompression
- Print embedded disc header (Game ID, Maker, Disc/Rev, Title) in WIA.Printing.cs and GCZ.Printing.cs

* Fix Wii partition extraction: correct IV, FST size shift, partition naming

- Block decryption: IV is at raw block offset 0x3D0 (still-encrypted),
  matching Dolphin/DolphinIsoLib WiiPartitionDecryptor.DecryptBlock exactly.
- FST size field at boot.bin 0x428 is also stored >>2 on Wii; apply <<2
  to get true byte size.
- Partition folder naming now matches DolphinIsoLib WiiDiscExtractor exactly:
  type 0->GM+n, 1->UP+n, 2->CH+n, printable ASCII unknown->raw 4-char string,
  non-printable->P{index}. SSBB VC channels extract as HA8E, HA9E, etc.
- ExtractionTool peek buffer increased from 16 to 32 bytes.

Verified: SSBB GM0 extracts 5524 files, boot.bin/fst.bin byte-identical
to Dolphin reference extraction.

* Fix FST extraction: create zero-byte files instead of skipping them

Files with fileSize=0 in the FST were silently skipped. Now they are
created as empty files, matching Dolphin/DolphinIsoLib behavior.

Verified: SSBB now extracts 5958 files with 0 missing, 0 extra,
0 size mismatches, and 0 hash mismatches vs DolphinIsoLib reference.

* Add GCZ/WIA/RVZ virtual stream extraction via NintendoDisc wrapper

* Address PR #85 review comments (Copilot + mnadareski)

* Address PR #85 review comments

* Replace custom endian helpers and SHA1 with SabreTools.IO equivalents

* Update GCZ.Printing.cs

* Update NintendoDisc.Printing.cs

* Update WIA.Printing.cs

* Add WIA/RVZ Wii partition crypto round-trip support

- Add AesCbc internal helper (BouncyCastle AES-CBC encrypt/decrypt)
- Add NintendoDisc.CommonKeyProvider hook for injectable test keys
- Fix sha1.Terminate() missing in all three ComputeSha1 helpers in WIA.cs
- Fix Wii partition dataOff alignment to 0x8000 boundary
- Add WIA.EncryptWiiGroup (internal) for re-encrypting plaintext groups
- Add WIA.DumpIso to WIA.Writing.cs (WIA/RVZ -> flat ISO conversion)
- Add WiaVirtualStream on-demand group decompression
- Add _preDecryptedReader bypass on NintendoDisc for WIA extraction path
- Add WIATests.cs with Wii crypto round-trip test using synthetic data
- Move DumpIso from WIA.Extraction.cs to WIA.Writing.cs
- Bump DumpIso read buffer from 1 MiB to 2 MiB (aligns to WIA chunk size)
- Add InternalsVisibleTo SabreTools.Wrappers.Test in csproj

* Remove hardcoded Wii common keys from NintendoDisc.Encryption

- Delete the embedded WiiCommonKeyRetail and WiiCommonKeyKorean byte
  arrays from NintendoDisc.Encryption.cs.
- Make CommonKeyProvider public so any caller (not just tests) can
  inject keys; DecryptTitleKey now returns null when no key is
  available for the requested index rather than falling back to
  hardcoded values.
- Add NintendoDiscEncryptionTests.cs:
    - Argument guard and no-provider tests for DecryptTitleKey.
    - Fake-key round-trip test (encrypt then decrypt with injected key).
    - Integration test that reads TestData/NintendoDisc/keys.json,
      verifies each key against hardcoded SHA256 constants, and skips
      silently if the file is absent or the keys do not match.
    - LoadKeyProvider helper (named JSON format, index-keyed).
- Add [Collection(NintendoDisc)] to both NintendoDiscEncryptionTests
  and WIATests to prevent parallel access to the static
  CommonKeyProvider from racing between test classes.
- Add TestData/NintendoDisc/keys.json.example documenting the
  expected key file format.
- Add Newtonsoft.Json reference to SabreTools.Wrappers.Test.csproj.

* Didn't actually commit the changes. My bad. Fixed.

* Edited a comment

* Added in XUnit outputs that show up in Test Viewer in VS

---------

Co-authored-by: Matt Nadareski <mnadareski@outlook.com>
2026-05-12 10:41:32 -04:00

229 lines
8.7 KiB
C#

using System;
using SabreTools.IO.Compression.Deflate;
using System.IO;
using SabreTools.Data.Models.GCZ;
namespace SabreTools.Wrappers
{
public partial class GCZ : IWritable
{
/// <summary>
/// Compress a NintendoDisc wrapper to a GCZ file at the given path.
/// </summary>
/// <param name="source">Decompressed disc image to compress.</param>
/// <param name="outputPath">Destination file path.</param>
/// <param name="blockSize">
/// GCZ block size: 32 KiB, 64 KiB, or 128 KiB.
/// Defaults to <see cref="Constants.DefaultBlockSize"/> (32 KiB).
/// </param>
/// <returns>True on success, false on failure.</returns>
public static bool ConvertFromDisc(NintendoDisc source, string outputPath,
uint blockSize = Constants.DefaultBlockSize)
{
if (source is null)
return false;
if (string.IsNullOrEmpty(outputPath))
return false;
if (blockSize != Constants.BlockSize32K &&
blockSize != Constants.BlockSize64K &&
blockSize != Constants.BlockSize128K)
return false;
try
{
using var fs = File.Open(outputPath, FileMode.Create, FileAccess.ReadWrite, FileShare.None);
return WriteGcz(source, fs, blockSize);
}
catch
{
return false;
}
}
/// <inheritdoc/>
public bool Write(string outputPath, bool includeDebug)
{
// Re-serialise the structural metadata (header + tables) only.
// Full round-trip compression from an already-GCZ source requires ConvertFromDisc.
if (string.IsNullOrEmpty(outputPath))
{
string outputFilename = Filename is null
? (Guid.NewGuid().ToString() + ".gcz")
: (Filename + ".new");
outputPath = Path.GetFullPath(outputFilename);
}
if (Model?.Header is null)
{
if (includeDebug) Console.WriteLine("Model was invalid, cannot write!");
return false;
}
var writer = new Serialization.Writers.GCZ { Debug = includeDebug };
return writer.SerializeFile(Model, outputPath);
}
// -----------------------------------------------------------------------
// Core GCZ compression pipeline (ISO → GCZ)
// -----------------------------------------------------------------------
/// <summary>
/// Write a GCZ image to <paramref name="destination"/> from a decompressed disc source.
/// Matches Dolphin's CompressFileToBlob() in CompressedBlob.cpp.
/// </summary>
private static bool WriteGcz(NintendoDisc source, Stream destination, uint blockSize)
{
long sourceSize = source.DataLength;
if (sourceSize <= 0)
return false;
uint numBlocks = (uint)((sourceSize + blockSize - 1) / blockSize);
// ---- Step 1: Write placeholder header (will be patched at end) ----
long headerPos = destination.Position;
var header = new GczHeader
{
MagicCookie = Constants.MagicCookie,
SubType = 0,
CompressedDataSize = 0,
DataSize = (ulong)sourceSize,
BlockSize = blockSize,
NumBlocks = numBlocks,
};
WriteHeader(destination, header);
// ---- Step 2: Reserve block-pointer table (8 bytes each) ----
long blockTablePos = destination.Position;
var blockPointers = new ulong[numBlocks];
destination.Position += (long)numBlocks * 8;
// ---- Step 3: Reserve block-hash table (4 bytes each) ----
var blockHashes = new uint[numBlocks];
destination.Position += (long)numBlocks * 4;
// ---- Step 4: Data section starts here ----
long dataStartPos = destination.Position;
var readBuf = new byte[blockSize];
var compressBuf = new byte[(int)blockSize * 2];
for (uint bi = 0; bi < numBlocks; bi++)
{
long blockOffset = (long)bi * blockSize;
int blockDataSize = (int)Math.Min(blockSize, sourceSize - blockOffset);
byte[]? raw = source.ReadData(blockOffset, blockDataSize);
if (raw is null || raw.Length != blockDataSize)
return false;
if (blockDataSize < readBuf.Length)
Array.Copy(raw, readBuf, blockDataSize);
else
readBuf = raw;
// Record pointer as offset relative to data section start
ulong blockPointer = (ulong)(destination.Position - dataStartPos);
int compressedSize;
bool useCompression = TryCompressBlock(readBuf, blockDataSize, compressBuf, out compressedSize);
if (useCompression)
{
blockPointers[bi] = blockPointer;
destination.Write(compressBuf, 0, compressedSize);
blockHashes[bi] = Adler.Adler32(1, compressBuf, 0, compressedSize);
}
else
{
blockPointers[bi] = blockPointer | Constants.UncompressedFlag;
destination.Write(readBuf, 0, blockDataSize);
blockHashes[bi] = Adler.Adler32(1, readBuf, 0, blockDataSize);
}
}
// ---- Step 5: Patch header with final compressed-data size ----
long finalEnd = destination.Position;
header.CompressedDataSize = (ulong)(finalEnd - dataStartPos);
// ---- Step 6: Write block-pointer table ----
destination.Position = blockTablePos;
foreach (ulong ptr in blockPointers)
WriteUInt64LE(destination, ptr);
// ---- Step 7: Write block-hash table ----
foreach (uint h in blockHashes)
WriteUInt32LE(destination, h);
// ---- Step 8: Patch header ----
destination.Position = headerPos;
WriteHeader(destination, header);
destination.Position = finalEnd;
destination.Flush();
return true;
}
// -----------------------------------------------------------------------
// Compression helpers
// -----------------------------------------------------------------------
/// <summary>
/// Attempts to zlib-compress <paramref name="inputSize"/> bytes of <paramref name="input"/>
/// into <paramref name="output"/>. Returns true and sets <paramref name="compressedSize"/>
/// when the result is smaller than 97 % of the original (Dolphin's threshold).
/// GCZ uses the zlib framing: 2-byte header (0x78 0x9C) + deflate stream + 4-byte Adler-32 tail.
/// </summary>
private static bool TryCompressBlock(byte[] input, int inputSize, byte[] output, out int compressedSize)
{
using (var ms = new MemoryStream(output))
{
ms.WriteByte(0x78);
ms.WriteByte(0x9C);
using (var ds = new DeflateStream(ms, CompressionMode.Compress, leaveOpen: true))
{
ds.Write(input, 0, inputSize);
}
uint adler = Adler.Adler32(1, input, 0, inputSize);
ms.WriteByte((byte)(adler >> 24));
ms.WriteByte((byte)(adler >> 16));
ms.WriteByte((byte)(adler >> 8));
ms.WriteByte((byte)adler);
compressedSize = (int)ms.Position;
}
int threshold = inputSize * 97 / 100;
return compressedSize < threshold;
}
// -----------------------------------------------------------------------
// Little-endian binary write helpers
// -----------------------------------------------------------------------
private static void WriteHeader(Stream s, GczHeader h)
{
WriteUInt32LE(s, h.MagicCookie);
WriteUInt32LE(s, h.SubType);
WriteUInt64LE(s, h.CompressedDataSize);
WriteUInt64LE(s, h.DataSize);
WriteUInt32LE(s, h.BlockSize);
WriteUInt32LE(s, h.NumBlocks);
}
private static void WriteUInt32LE(Stream s, uint v)
{
s.WriteByte((byte)v);
s.WriteByte((byte)(v >> 8));
s.WriteByte((byte)(v >> 16));
s.WriteByte((byte)(v >> 24));
}
private static void WriteUInt64LE(Stream s, ulong v)
{
WriteUInt32LE(s, (uint)v);
WriteUInt32LE(s, (uint)(v >> 32));
}
}
}