Compare commits

..

38 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
b3ce90ae94 Remove foo.zip and add Zip.sozip.zip test archive with tests
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-06 13:36:21 +00:00
Adam Hathcock
130e169862 Merge remote-tracking branch 'origin/master' into copilot/add-so-optimized-zip-support
# Conflicts:
#	Directory.Packages.props
#	FORMATS.md
#	build/packages.lock.json
2026-01-06 13:22:45 +00:00
copilot-swe-agent[bot]
0dc63223ab Merge master branch and resolve FORMATS.md conflict
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-05 17:21:38 +00:00
Adam Hathcock
b8e5ee45eb Merge pull request #1109 from adamhathcock/dependabot/nuget/build/SimpleExec-13.0.0
Bump SimpleExec from 12.1.0 to 13.0.0
2026-01-05 17:17:55 +00:00
Adam Hathcock
9f20a9e7d2 Merge pull request #1110 from TwanVanDongen/master
Formats.md updated to reflect additions of Ace, Arc and Arj
2026-01-05 17:14:26 +00:00
Twan
201521d814 Merge branch 'adamhathcock:master' into master 2026-01-05 18:09:55 +01:00
Twan van Dongen
18bb3cba11 Added descriptions for archives Ace, Arc and Arj 2026-01-05 18:08:53 +01:00
Adam Hathcock
af951d6f6a Merge pull request #1102 from TwanVanDongen/master
Add support for ACE archives
2026-01-05 16:37:35 +00:00
Adam Hathcock
e5fe92bf90 Merge pull request #1108 from adamhathcock/dependabot/nuget/dot-config/csharpier-1.2.5
Bump csharpier from 1.2.4 to 1.2.5
2026-01-05 16:15:12 +00:00
dependabot[bot]
b1aca7c305 Bump SimpleExec from 12.1.0 to 13.0.0
---
updated-dependencies:
- dependency-name: SimpleExec
  dependency-version: 13.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-05 09:32:51 +00:00
dependabot[bot]
c0a0cc4a44 Bump csharpier from 1.2.4 to 1.2.5
---
updated-dependencies:
- dependency-name: csharpier
  dependency-version: 1.2.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-05 09:32:17 +00:00
Twan van Dongen
7a49eb9e93 Archives containing encrypted content throws exception. 2026-01-04 19:10:57 +01:00
Adam Hathcock
5aa0610882 Merge pull request #1104 from adamhathcock/copilot/fix-invalidoperationexception-rar
Fix InvalidOperationException when RAR uncompressed size exceeds header value
2026-01-04 12:18:38 +00:00
copilot-swe-agent[bot]
41ed4c8186 Add negative test case for premature stream termination
Added Rar_StreamValidation_ThrowsOnTruncatedStream test that verifies InvalidOperationException IS thrown when a RAR stream ends prematurely (position < expected length). Created TruncatedStream mock to simulate corrupted/truncated RAR files. This test validates the exception condition (_position < Length) works correctly.

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-04 11:47:57 +00:00
copilot-swe-agent[bot]
90a33ce6b0 Remove duplicate test that doesn't verify exception case
Removed Rar_StreamValidation_CorrectExceptionBehavior test as it duplicated the validation in Rar_StreamValidation_OnlyThrowsOnPrematureEnd without actually testing the exception case. The remaining test adequately validates that the fix works correctly across multiple RAR formats.

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-04 11:36:25 +00:00
copilot-swe-agent[bot]
12574798e1 Update test to document exception behavior for RAR stream validation
Renamed and enhanced the test to better document the fix. Added second test (Rar_StreamValidation_CorrectExceptionBehavior) that explicitly validates the difference between old and new behavior. Tests verify that InvalidOperationException is only thrown when position < expected length (premature termination), not when position >= expected length (which can be valid for some RAR files).

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-04 09:55:18 +00:00
copilot-swe-agent[bot]
83b11254db Add test for InvalidOperationException fix in RAR extraction
Added Rar_ExtractionCompletesWithoutInvalidOperationException test that verifies RAR extraction completes successfully without throwing InvalidOperationException when reading streams to EOF. The test validates the fix works across RAR, RAR5, RAR4, and RAR2 formats by reading all entries completely and ensuring no exceptions are thrown.

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-04 09:33:31 +00:00
copilot-swe-agent[bot]
b25493fd29 Fix InvalidOperationException when RAR unpacks more data than header specifies
Changed validation condition from `_position != Length` to `_position < Length` in RarStream.Read() and RarStream.ReadImplAsync() to only throw when unpacking ends prematurely, not when more data is unpacked than the header specifies. This allows successful extraction of RAR files where the actual uncompressed size exceeds the header size.

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-04 09:23:47 +00:00
copilot-swe-agent[bot]
bb66100486 Initial plan 2026-01-04 09:10:50 +00:00
Twan van Dongen
3ebf97dd49 MultiVolume not supported, tests provided by split archive. 2026-01-03 19:09:43 +01:00
Twan van Dongen
bfcdeb3784 Ace largefile test added 2026-01-03 18:48:54 +01:00
Twan van Dongen
feece3d788 Missed some CSharpier edits 2026-01-03 18:08:01 +01:00
Twan van Dongen
94adb77e9e Merge branch 'master' of https://github.com/TwanVanDongen/sharpcompress 2026-01-03 18:01:13 +01:00
Twan van Dongen
909d36c237 more subtle check of magic bytes for ARJ archives 2026-01-03 17:59:17 +01:00
Twan van Dongen
e1c8aa226d Add ACE archive support (read-only, stored entries) 2026-01-03 17:18:00 +01:00
Adam Hathcock
2327679f23 Merge pull request #1098 from adamhathcock/adam/remove-old-release
remove old release
2026-01-03 14:27:13 +00:00
Adam Hathcock
574d9f970c Merge pull request #1099 from adamhathcock/copilot/sub-pr-1098
Configure nuget-release workflow to validate PRs without publishing
2026-01-03 14:22:13 +00:00
copilot-swe-agent[bot]
235096a2eb Configure nuget-release.yml to run on PRs without publishing
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-03 14:15:50 +00:00
copilot-swe-agent[bot]
a739fdc544 Initial plan 2026-01-03 14:13:34 +00:00
Adam Hathcock
6196e26044 also remove from sln 2026-01-03 13:57:42 +00:00
Adam Hathcock
46a4064989 remove old bulid 2026-01-03 13:54:51 +00:00
Adam Hathcock
9058645fea sozip writing and validation 2025-11-27 10:49:19 +00:00
copilot-swe-agent[bot]
7339567880 Fix SOZip tests to work correctly with ZipReader and ZipArchive
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2025-11-27 08:18:27 +00:00
Adam Hathcock
8c6d914004 reader tests don't pass or make sense 2025-11-26 15:11:03 +00:00
copilot-swe-agent[bot]
d9c9612b8f Update documentation for SOZip support
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2025-11-26 08:31:49 +00:00
copilot-swe-agent[bot]
a35089900f Add SOZip detection in ZipEntry and additional tests
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2025-11-26 08:27:16 +00:00
copilot-swe-agent[bot]
ac4bcd0fe3 Add SOZip index data structure and basic tests
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2025-11-26 08:23:29 +00:00
copilot-swe-agent[bot]
0ac6b46379 Initial plan 2025-11-26 08:12:38 +00:00
55 changed files with 2653 additions and 83 deletions

View File

@@ -3,11 +3,11 @@
"isRoot": true,
"tools": {
"csharpier": {
"version": "1.2.4",
"version": "1.2.5",
"commands": [
"csharpier"
],
"rollForward": false
}
}
}
}

View File

@@ -1,25 +0,0 @@
name: SharpCompress
on:
push:
branches:
- 'master'
pull_request:
types: [ opened, synchronize, reopened, ready_for_review ]
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest, ubuntu-latest]
steps:
- uses: actions/checkout@v6
- uses: actions/setup-dotnet@v5
with:
dotnet-version: 10.0.x
- run: dotnet run --project build/build.csproj
- uses: actions/upload-artifact@v6
with:
name: ${{ matrix.os }}-sharpcompress.nupkg
path: artifacts/*

View File

@@ -7,6 +7,10 @@ on:
- 'release'
tags:
- '[0-9]+.[0-9]+.[0-9]+'
pull_request:
branches:
- 'master'
- 'release'
permissions:
contents: read
@@ -49,9 +53,9 @@ jobs:
name: ${{ matrix.os }}-nuget-package
path: artifacts/*.nupkg
# Push to NuGet.org using C# build target (Windows only)
# Push to NuGet.org using C# build target (Windows only, not on PRs)
- name: Push to NuGet
if: success() && matrix.os == 'windows-latest'
if: success() && matrix.os == 'windows-latest' && github.event_name != 'pull_request'
run: dotnet run --project build/build.csproj -- push-to-nuget
env:
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}

View File

@@ -7,7 +7,7 @@
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageVersion Include="Mono.Posix.NETStandard" Version="1.0.0" />
<PackageVersion Include="SimpleExec" Version="12.1.0" />
<PackageVersion Include="SimpleExec" Version="13.0.0" />
<PackageVersion Include="System.Text.Encoding.CodePages" Version="10.0.0" />
<PackageVersion Include="System.Buffers" Version="4.6.1" />
<PackageVersion Include="System.Memory" Version="4.6.3" />

View File

@@ -10,7 +10,10 @@
| Archive Format | Compression Format(s) | Compress/Decompress | Archive API | Reader API | Writer API |
| ---------------------- | ------------------------------------------------- | ------------------- | --------------- | ---------- | ------------- |
| Rar | Rar | Decompress (1) | RarArchive | RarReader | N/A |
| Ace | None | Decompress | N/A | AceReader | N/A |
| Arc | None, Packed, Squeezed, Crunched | Decompress | N/A | ArcReader | N/A |
| Arj | None | Decompress | N/A | ArjReader | N/A |
| Rar | Rar | Decompress | RarArchive | RarReader | N/A |
| Zip (2) | None, Shrink, Reduce, Implode, DEFLATE, Deflate64, BZip2, LZMA/LZMA2, PPMd | Both | ZipArchive | ZipReader | ZipWriter |
| Tar | None | Both | TarArchive | TarReader | TarWriter (3) |
| Tar.GZip | DEFLATE | Both | TarArchive | TarReader | TarWriter (3) |
@@ -22,7 +25,7 @@
| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Decompress | SevenZipArchive | N/A | N/A |
1. SOLID Rars are only supported in the RarReader API.
2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. Deflate64 is only supported for reading. See [Zip Format Notes](#zip-format-notes) for details on multi-volume archives and streaming behavior.
2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. Deflate64 is only supported for reading. SOZip (Seek-Optimized ZIP) detection is supported for reading. See [Zip Format Notes](#zip-format-notes) for details on multi-volume archives and streaming behavior.
3. The Tar format requires a file size in the header. If no size is specified to the TarWriter and the stream is not seekable, then an exception will be thrown.
4. The 7Zip format doesn't allow for reading as a forward-only stream so 7Zip is only supported through the Archive API. See [7Zip Format Notes](#7zip-format-notes) for details on async extraction behavior.
5. LZip has no support for extra data like the file name or timestamp. There is a default filename used when looking at the entry Key on the archive.

View File

@@ -20,7 +20,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Config", "Config", "{CDB425
.editorconfig = .editorconfig
Directory.Packages.props = Directory.Packages.props
NuGet.config = NuGet.config
.github\workflows\dotnetcore.yml = .github\workflows\dotnetcore.yml
.github\workflows\nuget-release.yml = .github\workflows\nuget-release.yml
USAGE.md = USAGE.md
README.md = README.md
FORMATS.md = FORMATS.md

View File

@@ -16,9 +16,9 @@
},
"SimpleExec": {
"type": "Direct",
"requested": "[12.1.0, )",
"resolved": "12.1.0",
"contentHash": "PcCSAlMcKr5yTd571MgEMoGmoSr+omwziq2crB47lKP740lrmjuBocAUXHj+Q6LR6aUDFyhszot2wbtFJTClkA=="
"requested": "[13.0.0, )",
"resolved": "13.0.0",
"contentHash": "zcCR1pupa1wI1VqBULRiQKeHKKZOuJhi/K+4V5oO+rHJZlaOD53ViFo1c3PavDoMAfSn/FAXGAWpPoF57rwhYg=="
}
}
}

View File

@@ -0,0 +1,61 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace SharpCompress.Common.Ace
{
public class AceCrc
{
// CRC-32 lookup table (standard polynomial 0xEDB88320, reflected)
private static readonly uint[] Crc32Table = GenerateTable();
private static uint[] GenerateTable()
{
var table = new uint[256];
for (int i = 0; i < 256; i++)
{
uint crc = (uint)i;
for (int j = 0; j < 8; j++)
{
if ((crc & 1) != 0)
crc = (crc >> 1) ^ 0xEDB88320u;
else
crc >>= 1;
}
table[i] = crc;
}
return table;
}
/// <summary>
/// Calculate ACE CRC-32 checksum.
/// ACE CRC-32 uses standard CRC-32 polynomial (0xEDB88320, reflected)
/// with init=0xFFFFFFFF but NO final XOR.
/// </summary>
public static uint AceCrc32(ReadOnlySpan<byte> data)
{
uint crc = 0xFFFFFFFFu;
foreach (byte b in data)
{
crc = (crc >> 8) ^ Crc32Table[(crc ^ b) & 0xFF];
}
return crc; // No final XOR for ACE
}
/// <summary>
/// ACE CRC-16 is the lower 16 bits of the ACE CRC-32.
/// </summary>
public static ushort AceCrc16(ReadOnlySpan<byte> data)
{
return (ushort)(AceCrc32(data) & 0xFFFF);
}
}
}

View File

@@ -0,0 +1,68 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common.Ace.Headers;
namespace SharpCompress.Common.Ace
{
public class AceEntry : Entry
{
private readonly AceFilePart _filePart;
internal AceEntry(AceFilePart filePart)
{
_filePart = filePart;
}
public override long Crc
{
get
{
if (_filePart == null)
{
return 0;
}
return _filePart.Header.Crc32;
}
}
public override string? Key => _filePart?.Header.Filename;
public override string? LinkTarget => null;
public override long CompressedSize => _filePart?.Header.PackedSize ?? 0;
public override CompressionType CompressionType
{
get
{
if (_filePart.Header.CompressionType == Headers.CompressionType.Stored)
{
return CompressionType.None;
}
return CompressionType.AceLZ77;
}
}
public override long Size => _filePart?.Header.OriginalSize ?? 0;
public override DateTime? LastModifiedTime => _filePart.Header.DateTime;
public override DateTime? CreatedTime => null;
public override DateTime? LastAccessedTime => null;
public override DateTime? ArchivedTime => null;
public override bool IsEncrypted => _filePart.Header.IsFileEncrypted;
public override bool IsDirectory => _filePart.Header.IsDirectory;
public override bool IsSplitAfter => false;
internal override IEnumerable<FilePart> Parts => _filePart.Empty();
}
}

View File

@@ -0,0 +1,52 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common.Ace.Headers;
using SharpCompress.IO;
namespace SharpCompress.Common.Ace
{
public class AceFilePart : FilePart
{
private readonly Stream _stream;
internal AceFileHeader Header { get; set; }
internal AceFilePart(AceFileHeader localAceHeader, Stream seekableStream)
: base(localAceHeader.ArchiveEncoding)
{
_stream = seekableStream;
Header = localAceHeader;
}
internal override string? FilePartName => Header.Filename;
internal override Stream GetCompressedStream()
{
if (_stream != null)
{
Stream compressedStream;
switch (Header.CompressionType)
{
case Headers.CompressionType.Stored:
compressedStream = new ReadOnlySubStream(
_stream,
Header.DataStartPosition,
Header.PackedSize
);
break;
default:
throw new NotSupportedException(
"CompressionMethod: " + Header.CompressionQuality
);
}
return compressedStream;
}
return _stream.NotNull();
}
internal override Stream? GetRawStream() => _stream;
}
}

View File

@@ -0,0 +1,35 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common.Arj;
using SharpCompress.Readers;
namespace SharpCompress.Common.Ace
{
public class AceVolume : Volume
{
public AceVolume(Stream stream, ReaderOptions readerOptions, int index = 0)
: base(stream, readerOptions, index) { }
public override bool IsFirstVolume
{
get { return true; }
}
/// <summary>
/// ArjArchive is part of a multi-part archive.
/// </summary>
public override bool IsMultiVolume
{
get { return false; }
}
internal IEnumerable<AceFilePart> GetVolumeFileParts()
{
return new List<AceFilePart>();
}
}
}

View File

@@ -0,0 +1,171 @@
using System;
using System.Buffers.Binary;
using System.Collections.Generic;
using System.IO;
using System.Xml.Linq;
using SharpCompress.Common.Arc;
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// ACE file entry header
/// </summary>
public sealed class AceFileHeader : AceHeader
{
public long DataStartPosition { get; private set; }
public long PackedSize { get; set; }
public long OriginalSize { get; set; }
public DateTime DateTime { get; set; }
public int Attributes { get; set; }
public uint Crc32 { get; set; }
public CompressionType CompressionType { get; set; }
public CompressionQuality CompressionQuality { get; set; }
public ushort Parameters { get; set; }
public string Filename { get; set; } = string.Empty;
public List<byte> Comment { get; set; } = new();
/// <summary>
/// File data offset in the archive
/// </summary>
public ulong DataOffset { get; set; }
public bool IsDirectory => (Attributes & 0x10) != 0;
public bool IsContinuedFromPrev =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.CONTINUED_PREV) != 0;
public bool IsContinuedToNext =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.CONTINUED_NEXT) != 0;
public int DictionarySize
{
get
{
int bits = Parameters & 0x0F;
return bits < 10 ? 1024 : 1 << bits;
}
}
public AceFileHeader(ArchiveEncoding archiveEncoding)
: base(archiveEncoding, AceHeaderType.FILE) { }
/// <summary>
/// Reads the next file entry header from the stream.
/// Returns null if no more entries or end of archive.
/// Supports both ACE 1.0 and ACE 2.0 formats.
/// </summary>
public override AceHeader? Read(Stream stream)
{
var headerData = ReadHeader(stream);
if (headerData.Length == 0)
{
return null;
}
int offset = 0;
// Header type (1 byte)
HeaderType = headerData[offset++];
// Skip recovery record headers (ACE 2.0 feature)
if (HeaderType == (byte)SharpCompress.Common.Ace.Headers.AceHeaderType.RECOVERY32)
{
// Skip to next header
return null;
}
if (HeaderType != (byte)SharpCompress.Common.Ace.Headers.AceHeaderType.FILE)
{
// Unknown header type - skip
return null;
}
// Header flags (2 bytes)
HeaderFlags = BitConverter.ToUInt16(headerData, offset);
offset += 2;
// Packed size (4 bytes)
PackedSize = BitConverter.ToUInt32(headerData, offset);
offset += 4;
// Original size (4 bytes)
OriginalSize = BitConverter.ToUInt32(headerData, offset);
offset += 4;
// File date/time in DOS format (4 bytes)
var dosDateTime = BitConverter.ToUInt32(headerData, offset);
DateTime = ConvertDosDateTime(dosDateTime);
offset += 4;
// File attributes (4 bytes)
Attributes = (int)BitConverter.ToUInt32(headerData, offset);
offset += 4;
// CRC32 (4 bytes)
Crc32 = BitConverter.ToUInt32(headerData, offset);
offset += 4;
// Compression type (1 byte)
byte compressionType = headerData[offset++];
CompressionType = GetCompressionType(compressionType);
// Compression quality/parameter (1 byte)
byte compressionQuality = headerData[offset++];
CompressionQuality = GetCompressionQuality(compressionQuality);
// Parameters (2 bytes)
Parameters = BitConverter.ToUInt16(headerData, offset);
offset += 2;
// Reserved (2 bytes) - skip
offset += 2;
// Filename length (2 bytes)
var filenameLength = BitConverter.ToUInt16(headerData, offset);
offset += 2;
// Filename
if (offset + filenameLength <= headerData.Length)
{
Filename = ArchiveEncoding.Decode(headerData, offset, filenameLength);
offset += filenameLength;
}
// Handle comment if present
if ((HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.COMMENT) != 0)
{
// Comment length (2 bytes)
if (offset + 2 <= headerData.Length)
{
ushort commentLength = BitConverter.ToUInt16(headerData, offset);
offset += 2 + commentLength; // Skip comment
}
}
// Store the data start position
DataStartPosition = stream.Position;
return this;
}
public CompressionType GetCompressionType(byte value) =>
value switch
{
0 => CompressionType.Stored,
1 => CompressionType.Lz77,
2 => CompressionType.Blocked,
_ => CompressionType.Unknown,
};
public CompressionQuality GetCompressionQuality(byte value) =>
value switch
{
0 => CompressionQuality.None,
1 => CompressionQuality.Fastest,
2 => CompressionQuality.Fast,
3 => CompressionQuality.Normal,
4 => CompressionQuality.Good,
5 => CompressionQuality.Best,
_ => CompressionQuality.Unknown,
};
}
}

View File

@@ -0,0 +1,153 @@
using System;
using System.IO;
using SharpCompress.Common.Arj.Headers;
using SharpCompress.Crypto;
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// Header type constants
/// </summary>
public enum AceHeaderType
{
MAIN = 0,
FILE = 1,
RECOVERY32 = 2,
RECOVERY64A = 3,
RECOVERY64B = 4,
}
public abstract class AceHeader
{
// ACE signature: bytes at offset 7 should be "**ACE**"
private static readonly byte[] AceSignature =
[
(byte)'*',
(byte)'*',
(byte)'A',
(byte)'C',
(byte)'E',
(byte)'*',
(byte)'*',
];
public AceHeader(ArchiveEncoding archiveEncoding, AceHeaderType type)
{
AceHeaderType = type;
ArchiveEncoding = archiveEncoding;
}
public ArchiveEncoding ArchiveEncoding { get; }
public AceHeaderType AceHeaderType { get; }
public ushort HeaderFlags { get; set; }
public ushort HeaderCrc { get; set; }
public ushort HeaderSize { get; set; }
public byte HeaderType { get; set; }
public bool IsFileEncrypted =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.FILE_ENCRYPTED) != 0;
public bool Is64Bit =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.MEMORY_64BIT) != 0;
public bool IsSolid =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.SOLID_MAIN) != 0;
public bool IsMultiVolume =>
(HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.MULTIVOLUME) != 0;
public abstract AceHeader? Read(Stream reader);
public byte[] ReadHeader(Stream stream)
{
// Read header CRC (2 bytes) and header size (2 bytes)
var headerBytes = new byte[4];
if (stream.Read(headerBytes, 0, 4) != 4)
{
return Array.Empty<byte>();
}
HeaderCrc = BitConverter.ToUInt16(headerBytes, 0); // CRC for validation
HeaderSize = BitConverter.ToUInt16(headerBytes, 2);
if (HeaderSize == 0)
{
return Array.Empty<byte>();
}
// Read the header data
var body = new byte[HeaderSize];
if (stream.Read(body, 0, HeaderSize) != HeaderSize)
{
return Array.Empty<byte>();
}
// Verify crc
var checksum = AceCrc.AceCrc16(body);
if (checksum != HeaderCrc)
{
throw new InvalidDataException("Header checksum is invalid");
}
return body;
}
public static bool IsArchive(Stream stream)
{
// ACE files have a specific signature
// First two bytes are typically 0x60 0xEA (signature bytes)
// At offset 7, there should be "**ACE**" (7 bytes)
var bytes = new byte[14];
if (stream.Read(bytes, 0, 14) != 14)
{
return false;
}
// Check for "**ACE**" at offset 7
return CheckMagicBytes(bytes, 7);
}
protected static bool CheckMagicBytes(byte[] headerBytes, int offset)
{
// Check for "**ACE**" at specified offset
for (int i = 0; i < AceSignature.Length; i++)
{
if (headerBytes[offset + i] != AceSignature[i])
{
return false;
}
}
return true;
}
protected DateTime ConvertDosDateTime(uint dosDateTime)
{
try
{
int second = (int)(dosDateTime & 0x1F) * 2;
int minute = (int)((dosDateTime >> 5) & 0x3F);
int hour = (int)((dosDateTime >> 11) & 0x1F);
int day = (int)((dosDateTime >> 16) & 0x1F);
int month = (int)((dosDateTime >> 21) & 0x0F);
int year = (int)((dosDateTime >> 25) & 0x7F) + 1980;
if (
day < 1
|| day > 31
|| month < 1
|| month > 12
|| hour > 23
|| minute > 59
|| second > 59
)
{
return DateTime.MinValue;
}
return new DateTime(year, month, day, hour, minute, second);
}
catch
{
return DateTime.MinValue;
}
}
}
}

View File

@@ -0,0 +1,97 @@
using System;
using System.Buffers.Binary;
using System.Collections.Generic;
using System.IO;
using SharpCompress.Common.Ace.Headers;
using SharpCompress.Common.Zip.Headers;
using SharpCompress.Crypto;
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// ACE main archive header
/// </summary>
public sealed class AceMainHeader : AceHeader
{
public byte ExtractVersion { get; set; }
public byte CreatorVersion { get; set; }
public HostOS HostOS { get; set; }
public byte VolumeNumber { get; set; }
public DateTime DateTime { get; set; }
public string Advert { get; set; } = string.Empty;
public List<byte> Comment { get; set; } = new();
public byte AceVersion { get; private set; }
public AceMainHeader(ArchiveEncoding archiveEncoding)
: base(archiveEncoding, AceHeaderType.MAIN) { }
/// <summary>
/// Reads the main archive header from the stream.
/// Returns header if this is a valid ACE archive.
/// Supports both ACE 1.0 and ACE 2.0 formats.
/// </summary>
public override AceHeader? Read(Stream stream)
{
var headerData = ReadHeader(stream);
if (headerData.Length == 0)
{
return null;
}
int offset = 0;
// Header type should be 0 for main header
if (headerData[offset++] != HeaderType)
{
return null;
}
// Header flags (2 bytes)
HeaderFlags = BitConverter.ToUInt16(headerData, offset);
offset += 2;
// Skip signature "**ACE**" (7 bytes)
if (!CheckMagicBytes(headerData, offset))
{
throw new InvalidDataException("Invalid ACE archive signature.");
}
offset += 7;
// ACE version (1 byte) - 10 for ACE 1.0, 20 for ACE 2.0
AceVersion = headerData[offset++];
ExtractVersion = headerData[offset++];
// Host OS (1 byte)
if (offset < headerData.Length)
{
var hostOsByte = headerData[offset++];
HostOS = hostOsByte <= 11 ? (HostOS)hostOsByte : HostOS.Unknown;
}
// Volume number (1 byte)
VolumeNumber = headerData[offset++];
// Creation date/time (4 bytes)
var dosDateTime = BitConverter.ToUInt32(headerData, offset);
DateTime = ConvertDosDateTime(dosDateTime);
offset += 4;
// Reserved fields (8 bytes)
if (offset + 8 <= headerData.Length)
{
offset += 8;
}
// Skip additional fields based on flags
// Handle comment if present
if ((HeaderFlags & SharpCompress.Common.Ace.Headers.HeaderFlags.COMMENT) != 0)
{
if (offset + 2 <= headerData.Length)
{
ushort commentLength = BitConverter.ToUInt16(headerData, offset);
offset += 2 + commentLength;
}
}
return this;
}
}
}

View File

@@ -0,0 +1,16 @@
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// Compression quality
/// </summary>
public enum CompressionQuality
{
None,
Fastest,
Fast,
Normal,
Good,
Best,
Unknown,
}
}

View File

@@ -0,0 +1,13 @@
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// Compression types
/// </summary>
public enum CompressionType
{
Stored,
Lz77,
Blocked,
Unknown,
}
}

View File

@@ -0,0 +1,33 @@
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// Header flags (main + file, overlapping meanings)
/// </summary>
public static class HeaderFlags
{
// Shared / low bits
public const ushort ADDSIZE = 0x0001; // extra size field present
public const ushort COMMENT = 0x0002; // comment present
public const ushort MEMORY_64BIT = 0x0004;
public const ushort AV_STRING = 0x0008; // AV string present
public const ushort SOLID = 0x0010; // solid file
public const ushort LOCKED = 0x0020;
public const ushort PROTECTED = 0x0040;
// Main header specific
public const ushort V20FORMAT = 0x0100;
public const ushort SFX = 0x0200;
public const ushort LIMITSFXJR = 0x0400;
public const ushort MULTIVOLUME = 0x0800;
public const ushort ADVERT = 0x1000;
public const ushort RECOVERY = 0x2000;
public const ushort LOCKED_MAIN = 0x4000;
public const ushort SOLID_MAIN = 0x8000;
// File header specific (same bits, different meaning)
public const ushort NTSECURITY = 0x0400;
public const ushort CONTINUED_PREV = 0x1000;
public const ushort CONTINUED_NEXT = 0x2000;
public const ushort FILE_ENCRYPTED = 0x4000; // file encrypted (file header)
}
}

View File

@@ -0,0 +1,22 @@
namespace SharpCompress.Common.Ace.Headers
{
/// <summary>
/// Host OS type
/// </summary>
public enum HostOS
{
MsDos = 0,
Os2,
Windows,
Unix,
MacOs,
WinNt,
Primos,
AppleGs,
Atari,
Vax,
Amiga,
Next,
Unknown,
}
}

View File

@@ -9,4 +9,5 @@ public enum ArchiveType
GZip,
Arc,
Arj,
Ace,
}

View File

@@ -34,14 +34,13 @@ namespace SharpCompress.Common.Arj.Headers
public byte[] ReadHeader(Stream stream)
{
// check for magic bytes
Span<byte> magic = stackalloc byte[2];
var magic = new byte[2];
if (stream.Read(magic) != 2)
{
return Array.Empty<byte>();
}
var magicValue = (ushort)(magic[0] | magic[1] << 8);
if (magicValue != ARJ_MAGIC)
if (!CheckMagicBytes(magic))
{
throw new InvalidDataException("Not an ARJ file (wrong magic bytes)");
}
@@ -138,5 +137,22 @@ namespace SharpCompress.Common.Arj.Headers
? (FileType)value
: Headers.FileType.Unknown;
}
public static bool IsArchive(Stream stream)
{
var bytes = new byte[2];
if (stream.Read(bytes, 0, 2) != 2)
{
return false;
}
return CheckMagicBytes(bytes);
}
protected static bool CheckMagicBytes(byte[] headerBytes)
{
var magicValue = (ushort)(headerBytes[0] | headerBytes[1] << 8);
return magicValue == ARJ_MAGIC;
}
}
}

View File

@@ -30,4 +30,5 @@ public enum CompressionType
Distilled,
ZStandard,
ArjLZ77,
AceLZ77,
}

View File

@@ -15,6 +15,10 @@ internal enum ExtraDataType : ushort
UnicodePathExtraField = 0x7075,
Zip64ExtendedInformationExtraField = 0x0001,
UnixTimeExtraField = 0x5455,
// SOZip (Seek-Optimized ZIP) extra field
// Used to link a main file to its SOZip index file
SOZip = 0x564B,
}
internal class ExtraData
@@ -233,6 +237,44 @@ internal sealed class UnixTimeExtraField : ExtraData
}
}
/// <summary>
/// SOZip (Seek-Optimized ZIP) extra field that links a main file to its index file.
/// The extra field contains the offset within the ZIP file where the index entry's
/// local header is located.
/// </summary>
internal sealed class SOZipExtraField : ExtraData
{
public SOZipExtraField(ExtraDataType type, ushort length, byte[] dataBytes)
: base(type, length, dataBytes) { }
/// <summary>
/// Gets the offset to the SOZip index file's local entry header within the ZIP archive.
/// </summary>
internal ulong IndexOffset
{
get
{
if (DataBytes is null || DataBytes.Length < 8)
{
return 0;
}
return BinaryPrimitives.ReadUInt64LittleEndian(DataBytes);
}
}
/// <summary>
/// Creates a SOZip extra field with the specified index offset
/// </summary>
/// <param name="indexOffset">The offset to the index file's local entry header</param>
/// <returns>A new SOZipExtraField instance</returns>
public static SOZipExtraField Create(ulong indexOffset)
{
var data = new byte[8];
BinaryPrimitives.WriteUInt64LittleEndian(data, indexOffset);
return new SOZipExtraField(ExtraDataType.SOZip, 8, data);
}
}
internal static class LocalEntryHeaderExtraFactory
{
internal static ExtraData Create(ExtraDataType type, ushort length, byte[] extraData) =>
@@ -246,6 +288,7 @@ internal static class LocalEntryHeaderExtraFactory
ExtraDataType.Zip64ExtendedInformationExtraField =>
new Zip64ExtendedInformationExtraField(type, length, extraData),
ExtraDataType.UnixTimeExtraField => new UnixTimeExtraField(type, length, extraData),
ExtraDataType.SOZip => new SOZipExtraField(type, length, extraData),
_ => new ExtraData(type, length, extraData),
};
}

View File

@@ -0,0 +1,150 @@
using System;
using System.Collections.Generic;
using System.IO;
using SharpCompress.Compressors;
using SharpCompress.Compressors.Deflate;
namespace SharpCompress.Common.Zip.SOZip;
/// <summary>
/// A Deflate stream that inserts sync flush points at regular intervals
/// to enable random access (SOZip optimization).
/// </summary>
internal sealed class SOZipDeflateStream : Stream
{
private readonly DeflateStream _deflateStream;
private readonly Stream _baseStream;
private readonly uint _chunkSize;
private readonly List<ulong> _compressedOffsets = new();
private readonly long _baseOffset;
private long _uncompressedBytesWritten;
private long _nextSyncPoint;
private bool _disposed;
/// <summary>
/// Creates a new SOZip Deflate stream
/// </summary>
/// <param name="baseStream">The underlying stream to write to</param>
/// <param name="compressionLevel">The compression level</param>
/// <param name="chunkSize">The chunk size for sync flush points</param>
public SOZipDeflateStream(Stream baseStream, CompressionLevel compressionLevel, int chunkSize)
{
_baseStream = baseStream;
_chunkSize = (uint)chunkSize;
_baseOffset = baseStream.Position;
_nextSyncPoint = chunkSize;
// Record the first offset (start of compressed data)
_compressedOffsets.Add(0);
_deflateStream = new DeflateStream(baseStream, CompressionMode.Compress, compressionLevel);
}
/// <summary>
/// Gets the array of compressed offsets recorded during writing
/// </summary>
public ulong[] CompressedOffsets => _compressedOffsets.ToArray();
/// <summary>
/// Gets the total number of uncompressed bytes written
/// </summary>
public ulong UncompressedBytesWritten => (ulong)_uncompressedBytesWritten;
/// <summary>
/// Gets the total number of compressed bytes written
/// </summary>
public ulong CompressedBytesWritten => (ulong)(_baseStream.Position - _baseOffset);
/// <summary>
/// Gets the chunk size being used
/// </summary>
public uint ChunkSize => _chunkSize;
public override bool CanRead => false;
public override bool CanSeek => false;
public override bool CanWrite => !_disposed && _deflateStream.CanWrite;
public override long Length => throw new NotSupportedException();
public override long Position
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
public override void Flush() => _deflateStream.Flush();
public override int Read(byte[] buffer, int offset, int count) =>
throw new NotSupportedException();
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count)
{
if (_disposed)
{
throw new ObjectDisposedException(nameof(SOZipDeflateStream));
}
var remaining = count;
var currentOffset = offset;
while (remaining > 0)
{
// Calculate how many bytes until the next sync point
var bytesUntilSync = (int)(_nextSyncPoint - _uncompressedBytesWritten);
if (bytesUntilSync <= 0)
{
// We've reached a sync point - perform sync flush
PerformSyncFlush();
continue;
}
// Write up to the next sync point
var bytesToWrite = Math.Min(remaining, bytesUntilSync);
_deflateStream.Write(buffer, currentOffset, bytesToWrite);
_uncompressedBytesWritten += bytesToWrite;
currentOffset += bytesToWrite;
remaining -= bytesToWrite;
}
}
private void PerformSyncFlush()
{
// Flush with Z_SYNC_FLUSH to create an independent block
var originalFlushMode = _deflateStream.FlushMode;
_deflateStream.FlushMode = FlushType.Sync;
_deflateStream.Flush();
_deflateStream.FlushMode = originalFlushMode;
// Record the compressed offset for this sync point
var compressedOffset = (ulong)(_baseStream.Position - _baseOffset);
_compressedOffsets.Add(compressedOffset);
// Set the next sync point
_nextSyncPoint += _chunkSize;
}
protected override void Dispose(bool disposing)
{
if (_disposed)
{
return;
}
_disposed = true;
if (disposing)
{
_deflateStream.Dispose();
}
base.Dispose(disposing);
}
}

View File

@@ -0,0 +1,367 @@
using System;
using System.Buffers.Binary;
using System.IO;
namespace SharpCompress.Common.Zip.SOZip;
/// <summary>
/// Represents a SOZip (Seek-Optimized ZIP) index that enables random access
/// within DEFLATE-compressed files by storing offsets to sync flush points.
/// </summary>
/// <remarks>
/// SOZip index files (.sozip.idx) contain a header followed by offset entries
/// that point to the beginning of independently decompressable DEFLATE blocks.
/// </remarks>
[CLSCompliant(false)]
public sealed class SOZipIndex
{
/// <summary>
/// SOZip index file magic number: "SOZo" (0x534F5A6F)
/// </summary>
public const uint SOZIP_MAGIC = 0x6F5A4F53; // "SOZo" little-endian
/// <summary>
/// Current SOZip specification version
/// </summary>
public const byte SOZIP_VERSION = 1;
/// <summary>
/// Index file extension suffix
/// </summary>
public const string INDEX_EXTENSION = ".sozip.idx";
/// <summary>
/// Default chunk size in bytes (32KB)
/// </summary>
public const uint DEFAULT_CHUNK_SIZE = 32768;
/// <summary>
/// The version of the SOZip index format
/// </summary>
public byte Version { get; private set; }
/// <summary>
/// Size of each uncompressed chunk in bytes
/// </summary>
public uint ChunkSize { get; private set; }
/// <summary>
/// Total uncompressed size of the file
/// </summary>
public ulong UncompressedSize { get; private set; }
/// <summary>
/// Total compressed size of the file
/// </summary>
public ulong CompressedSize { get; private set; }
/// <summary>
/// Number of offset entries in the index
/// </summary>
public uint OffsetCount { get; private set; }
/// <summary>
/// Array of compressed offsets for each chunk
/// </summary>
public ulong[] CompressedOffsets { get; private set; } = Array.Empty<ulong>();
/// <summary>
/// Creates a new empty SOZip index
/// </summary>
public SOZipIndex() { }
/// <summary>
/// Creates a new SOZip index with specified parameters
/// </summary>
/// <param name="chunkSize">Size of each uncompressed chunk</param>
/// <param name="uncompressedSize">Total uncompressed size</param>
/// <param name="compressedSize">Total compressed size</param>
/// <param name="compressedOffsets">Array of compressed offsets</param>
public SOZipIndex(
uint chunkSize,
ulong uncompressedSize,
ulong compressedSize,
ulong[] compressedOffsets
)
{
Version = SOZIP_VERSION;
ChunkSize = chunkSize;
UncompressedSize = uncompressedSize;
CompressedSize = compressedSize;
OffsetCount = (uint)compressedOffsets.Length;
CompressedOffsets = compressedOffsets;
}
/// <summary>
/// Reads a SOZip index from a stream
/// </summary>
/// <param name="stream">The stream containing the index data</param>
/// <returns>A parsed SOZipIndex instance</returns>
/// <exception cref="InvalidDataException">If the stream doesn't contain valid SOZip index data</exception>
public static SOZipIndex Read(Stream stream)
{
var index = new SOZipIndex();
Span<byte> header = stackalloc byte[4];
// Read magic number
if (stream.Read(header) != 4)
{
throw new InvalidDataException("Invalid SOZip index: unable to read magic number");
}
var magic = BinaryPrimitives.ReadUInt32LittleEndian(header);
if (magic != SOZIP_MAGIC)
{
throw new InvalidDataException(
$"Invalid SOZip index: magic number mismatch (expected 0x{SOZIP_MAGIC:X8}, got 0x{magic:X8})"
);
}
// Read version
var versionByte = stream.ReadByte();
if (versionByte < 0)
{
throw new InvalidDataException("Invalid SOZip index: unable to read version");
}
index.Version = (byte)versionByte;
if (index.Version != SOZIP_VERSION)
{
throw new InvalidDataException(
$"Unsupported SOZip index version: {index.Version} (expected {SOZIP_VERSION})"
);
}
// Read reserved byte (padding)
stream.ReadByte();
// Read chunk size (2 bytes)
Span<byte> buf2 = stackalloc byte[2];
if (stream.Read(buf2) != 2)
{
throw new InvalidDataException("Invalid SOZip index: unable to read chunk size");
}
// Chunk size is stored as (actual_size / 1024) - 1
var chunkSizeEncoded = BinaryPrimitives.ReadUInt16LittleEndian(buf2);
index.ChunkSize = ((uint)chunkSizeEncoded + 1) * 1024;
// Read uncompressed size (8 bytes)
Span<byte> buf8 = stackalloc byte[8];
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException("Invalid SOZip index: unable to read uncompressed size");
}
index.UncompressedSize = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
// Read compressed size (8 bytes)
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException("Invalid SOZip index: unable to read compressed size");
}
index.CompressedSize = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
// Read offset count (4 bytes)
if (stream.Read(header) != 4)
{
throw new InvalidDataException("Invalid SOZip index: unable to read offset count");
}
index.OffsetCount = BinaryPrimitives.ReadUInt32LittleEndian(header);
// Read offsets
index.CompressedOffsets = new ulong[index.OffsetCount];
for (uint i = 0; i < index.OffsetCount; i++)
{
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException($"Invalid SOZip index: unable to read offset {i}");
}
index.CompressedOffsets[i] = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
}
return index;
}
/// <summary>
/// Reads a SOZip index from a byte array
/// </summary>
/// <param name="data">The byte array containing the index data</param>
/// <returns>A parsed SOZipIndex instance</returns>
public static SOZipIndex Read(byte[] data)
{
using var stream = new MemoryStream(data);
return Read(stream);
}
/// <summary>
/// Writes this SOZip index to a stream
/// </summary>
/// <param name="stream">The stream to write to</param>
public void Write(Stream stream)
{
Span<byte> buf8 = stackalloc byte[8];
// Write magic number
BinaryPrimitives.WriteUInt32LittleEndian(buf8, SOZIP_MAGIC);
stream.Write(buf8.Slice(0, 4));
// Write version
stream.WriteByte(SOZIP_VERSION);
// Write reserved byte (padding)
stream.WriteByte(0);
// Write chunk size (encoded as (size/1024)-1)
var chunkSizeEncoded = (ushort)((ChunkSize / 1024) - 1);
BinaryPrimitives.WriteUInt16LittleEndian(buf8, chunkSizeEncoded);
stream.Write(buf8.Slice(0, 2));
// Write uncompressed size
BinaryPrimitives.WriteUInt64LittleEndian(buf8, UncompressedSize);
stream.Write(buf8);
// Write compressed size
BinaryPrimitives.WriteUInt64LittleEndian(buf8, CompressedSize);
stream.Write(buf8);
// Write offset count
BinaryPrimitives.WriteUInt32LittleEndian(buf8, OffsetCount);
stream.Write(buf8.Slice(0, 4));
// Write offsets
foreach (var offset in CompressedOffsets)
{
BinaryPrimitives.WriteUInt64LittleEndian(buf8, offset);
stream.Write(buf8);
}
}
/// <summary>
/// Converts this SOZip index to a byte array
/// </summary>
/// <returns>Byte array containing the serialized index</returns>
public byte[] ToByteArray()
{
using var stream = new MemoryStream();
Write(stream);
return stream.ToArray();
}
/// <summary>
/// Gets the index of the chunk that contains the specified uncompressed offset
/// </summary>
/// <param name="uncompressedOffset">The uncompressed byte offset</param>
/// <returns>The chunk index</returns>
public int GetChunkIndex(long uncompressedOffset)
{
if (uncompressedOffset < 0 || (ulong)uncompressedOffset >= UncompressedSize)
{
throw new ArgumentOutOfRangeException(
nameof(uncompressedOffset),
"Offset is out of range"
);
}
return (int)((ulong)uncompressedOffset / ChunkSize);
}
/// <summary>
/// Gets the compressed offset for the specified chunk index
/// </summary>
/// <param name="chunkIndex">The chunk index</param>
/// <returns>The compressed byte offset for the start of the chunk</returns>
public ulong GetCompressedOffset(int chunkIndex)
{
if (chunkIndex < 0 || chunkIndex >= CompressedOffsets.Length)
{
throw new ArgumentOutOfRangeException(
nameof(chunkIndex),
"Chunk index is out of range"
);
}
return CompressedOffsets[chunkIndex];
}
/// <summary>
/// Gets the uncompressed offset for the start of the specified chunk
/// </summary>
/// <param name="chunkIndex">The chunk index</param>
/// <returns>The uncompressed byte offset for the start of the chunk</returns>
public ulong GetUncompressedOffset(int chunkIndex)
{
if (chunkIndex < 0 || chunkIndex >= CompressedOffsets.Length)
{
throw new ArgumentOutOfRangeException(
nameof(chunkIndex),
"Chunk index is out of range"
);
}
return (ulong)chunkIndex * ChunkSize;
}
/// <summary>
/// Gets the name of the SOZip index file for a given entry name
/// </summary>
/// <param name="entryName">The main entry name</param>
/// <returns>The index file name (hidden with .sozip.idx extension)</returns>
public static string GetIndexFileName(string entryName)
{
var directory = Path.GetDirectoryName(entryName);
var fileName = Path.GetFileName(entryName);
// The index file is hidden (prefixed with .)
var indexFileName = $".{fileName}{INDEX_EXTENSION}";
if (string.IsNullOrEmpty(directory))
{
return indexFileName;
}
return Path.Combine(directory, indexFileName).Replace('\\', '/');
}
/// <summary>
/// Checks if a file name is a SOZip index file
/// </summary>
/// <param name="fileName">The file name to check</param>
/// <returns>True if the file is a SOZip index file</returns>
public static bool IsIndexFile(string fileName)
{
if (string.IsNullOrEmpty(fileName))
{
return false;
}
var name = Path.GetFileName(fileName);
return name.StartsWith(".", StringComparison.Ordinal)
&& name.EndsWith(INDEX_EXTENSION, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Gets the main file name from a SOZip index file name
/// </summary>
/// <param name="indexFileName">The index file name</param>
/// <returns>The main file name, or null if not a valid index file</returns>
public static string? GetMainFileName(string indexFileName)
{
if (!IsIndexFile(indexFileName))
{
return null;
}
var directory = Path.GetDirectoryName(indexFileName);
var name = Path.GetFileName(indexFileName);
// Remove leading '.' and trailing '.sozip.idx'
var mainName = name.Substring(1, name.Length - 1 - INDEX_EXTENSION.Length);
if (string.IsNullOrEmpty(directory))
{
return mainName;
}
return Path.Combine(directory, mainName).Replace('\\', '/');
}
}

View File

@@ -2,6 +2,7 @@ using System;
using System.Collections.Generic;
using System.Linq;
using SharpCompress.Common.Zip.Headers;
using SharpCompress.Common.Zip.SOZip;
namespace SharpCompress.Common.Zip;
@@ -11,7 +12,7 @@ public class ZipEntry : Entry
internal ZipEntry(ZipFilePart? filePart)
{
if (filePart == null)
if (filePart is null)
{
return;
}
@@ -88,4 +89,24 @@ public class ZipEntry : Entry
public override int? Attrib => (int?)_filePart?.Header.ExternalFileAttributes;
public string? Comment => _filePart?.Header.Comment;
/// <summary>
/// Gets a value indicating whether this entry has SOZip (Seek-Optimized ZIP) support.
/// A SOZip entry has an associated index file that enables random access within
/// the compressed data.
/// </summary>
public bool IsSozip => _filePart?.Header.Extra.Any(e => e.Type == ExtraDataType.SOZip) ?? false;
/// <summary>
/// Gets a value indicating whether this entry is a SOZip index file.
/// Index files are hidden files with a .sozip.idx extension that contain
/// offsets into the main compressed file.
/// </summary>
public bool IsSozipIndexFile => Key is not null && SOZipIndex.IsIndexFile(Key);
/// <summary>
/// Gets the SOZip extra field data, if present.
/// </summary>
internal SOZipExtraField? SOZipExtra =>
_filePart?.Header.Extra.OfType<SOZipExtraField>().FirstOrDefault();
}

View File

@@ -134,7 +134,7 @@ internal class RarStream : Stream, IStreamStack
fetch = false;
}
_position += outTotal;
if (count > 0 && outTotal == 0 && _position != Length)
if (count > 0 && outTotal == 0 && _position < Length)
{
// sanity check, eg if we try to decompress a redir entry
throw new InvalidOperationException(
@@ -179,7 +179,7 @@ internal class RarStream : Stream, IStreamStack
fetch = false;
}
_position += outTotal;
if (count > 0 && outTotal == 0 && _position != Length)
if (count > 0 && outTotal == 0 && _position < Length)
{
// sanity check, eg if we try to decompress a redir entry
throw new InvalidOperationException(

View File

@@ -0,0 +1,37 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Common.Ace.Headers;
using SharpCompress.Readers;
using SharpCompress.Readers.Ace;
namespace SharpCompress.Factories
{
public class AceFactory : Factory, IReaderFactory
{
public override string Name => "Ace";
public override ArchiveType? KnownArchiveType => ArchiveType.Ace;
public override IEnumerable<string> GetSupportedExtensions()
{
yield return "ace";
}
public override bool IsArchive(
Stream stream,
string? password = null,
int bufferSize = ReaderOptions.DefaultBufferSize
)
{
return AceHeader.IsArchive(stream);
}
public IReader OpenReader(Stream stream, ReaderOptions? options) =>
AceReader.Open(stream, options);
}
}

View File

@@ -28,12 +28,7 @@ namespace SharpCompress.Factories
int bufferSize = ReaderOptions.DefaultBufferSize
)
{
var arjHeader = new ArjMainHeader(new ArchiveEncoding());
if (arjHeader.Read(stream) == null)
{
return false;
}
return true;
return ArjHeader.IsArchive(stream);
}
public IReader OpenReader(Stream stream, ReaderOptions? options) =>

View File

@@ -19,6 +19,7 @@ public abstract class Factory : IFactory
RegisterFactory(new TarFactory());
RegisterFactory(new ArcFactory());
RegisterFactory(new ArjFactory());
RegisterFactory(new AceFactory());
}
private static readonly HashSet<Factory> _factories = new();

View File

@@ -0,0 +1,115 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Common.Ace;
using SharpCompress.Common.Ace.Headers;
using SharpCompress.Common.Arj;
namespace SharpCompress.Readers.Ace
{
/// <summary>
/// Reader for ACE archives.
/// ACE is a proprietary archive format. This implementation supports both ACE 1.0 and ACE 2.0 formats
/// and can read archive metadata and extract uncompressed (stored) entries.
/// Compressed entries require proprietary decompression algorithms that are not publicly documented.
/// </summary>
/// <remarks>
/// ACE 2.0 additions over ACE 1.0:
/// - Improved LZ77 compression (compression type 2)
/// - Recovery record support
/// - Additional header flags
/// </remarks>
public abstract class AceReader : AbstractReader<AceEntry, AceVolume>
{
private readonly ArchiveEncoding _archiveEncoding;
internal AceReader(ReaderOptions options)
: base(options, ArchiveType.Ace)
{
_archiveEncoding = Options.ArchiveEncoding;
}
private AceReader(Stream stream, ReaderOptions options)
: this(options) { }
/// <summary>
/// Derived class must create or manage the Volume itself.
/// AbstractReader.Volume is get-only, so it cannot be set here.
/// </summary>
public override AceVolume? Volume => _volume;
private AceVolume? _volume;
/// <summary>
/// Opens an AceReader for non-seeking usage with a single volume.
/// </summary>
/// <param name="stream">The stream containing the ACE archive.</param>
/// <param name="options">Reader options.</param>
/// <returns>An AceReader instance.</returns>
public static AceReader Open(Stream stream, ReaderOptions? options = null)
{
stream.NotNull(nameof(stream));
return new SingleVolumeAceReader(stream, options ?? new ReaderOptions());
}
/// <summary>
/// Opens an AceReader for Non-seeking usage with multiple volumes
/// </summary>
/// <param name="streams"></param>
/// <param name="options"></param>
/// <returns></returns>
public static AceReader Open(IEnumerable<Stream> streams, ReaderOptions? options = null)
{
streams.NotNull(nameof(streams));
return new MultiVolumeAceReader(streams, options ?? new ReaderOptions());
}
protected abstract void ValidateArchive(AceVolume archive);
protected override IEnumerable<AceEntry> GetEntries(Stream stream)
{
var mainHeaderReader = new AceMainHeader(_archiveEncoding);
var mainHeader = mainHeaderReader.Read(stream);
if (mainHeader == null)
{
yield break;
}
if (mainHeader?.IsMultiVolume == true)
{
throw new MultiVolumeExtractionException(
"Multi volumes are currently not supported"
);
}
if (_volume == null)
{
_volume = new AceVolume(stream, Options, 0);
ValidateArchive(_volume);
}
var localHeaderReader = new AceFileHeader(_archiveEncoding);
while (true)
{
var localHeader = localHeaderReader.Read(stream);
if (localHeader?.IsFileEncrypted == true)
{
throw new CryptographicException(
"Password protected archives are currently not supported"
);
}
if (localHeader == null)
break;
yield return new AceEntry(new AceFilePart((AceFileHeader)localHeader, stream));
}
}
protected virtual IEnumerable<FilePart> CreateFilePartEnumerableForCurrentEntry() =>
Entry.Parts;
}
}

View File

@@ -0,0 +1,117 @@
#nullable disable
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Common.Ace;
namespace SharpCompress.Readers.Ace
{
internal class MultiVolumeAceReader : AceReader
{
private readonly IEnumerator<Stream> streams;
private Stream tempStream;
internal MultiVolumeAceReader(IEnumerable<Stream> streams, ReaderOptions options)
: base(options) => this.streams = streams.GetEnumerator();
protected override void ValidateArchive(AceVolume archive) { }
protected override Stream RequestInitialStream()
{
if (streams.MoveNext())
{
return streams.Current;
}
throw new MultiVolumeExtractionException(
"No stream provided when requested by MultiVolumeAceReader"
);
}
internal override bool NextEntryForCurrentStream()
{
if (!base.NextEntryForCurrentStream())
{
// if we're got another stream to try to process then do so
return streams.MoveNext() && LoadStreamForReading(streams.Current);
}
return true;
}
protected override IEnumerable<FilePart> CreateFilePartEnumerableForCurrentEntry()
{
var enumerator = new MultiVolumeStreamEnumerator(this, streams, tempStream);
tempStream = null;
return enumerator;
}
private class MultiVolumeStreamEnumerator : IEnumerable<FilePart>, IEnumerator<FilePart>
{
private readonly MultiVolumeAceReader reader;
private readonly IEnumerator<Stream> nextReadableStreams;
private Stream tempStream;
private bool isFirst = true;
internal MultiVolumeStreamEnumerator(
MultiVolumeAceReader r,
IEnumerator<Stream> nextReadableStreams,
Stream tempStream
)
{
reader = r;
this.nextReadableStreams = nextReadableStreams;
this.tempStream = tempStream;
}
public IEnumerator<FilePart> GetEnumerator() => this;
IEnumerator IEnumerable.GetEnumerator() => this;
public FilePart Current { get; private set; }
public void Dispose() { }
object IEnumerator.Current => Current;
public bool MoveNext()
{
if (isFirst)
{
Current = reader.Entry.Parts.First();
isFirst = false; //first stream already to go
return true;
}
if (!reader.Entry.IsSplitAfter)
{
return false;
}
if (tempStream != null)
{
reader.LoadStreamForReading(tempStream);
tempStream = null;
}
else if (!nextReadableStreams.MoveNext())
{
throw new MultiVolumeExtractionException(
"No stream provided when requested by MultiVolumeAceReader"
);
}
else
{
reader.LoadStreamForReading(nextReadableStreams.Current);
}
Current = reader.Entry.Parts.First();
return true;
}
public void Reset() { }
}
}
}

View File

@@ -0,0 +1,31 @@
using System;
using System.IO;
using SharpCompress.Common;
using SharpCompress.Common.Ace;
namespace SharpCompress.Readers.Ace
{
internal class SingleVolumeAceReader : AceReader
{
private readonly Stream _stream;
internal SingleVolumeAceReader(Stream stream, ReaderOptions options)
: base(options)
{
stream.NotNull(nameof(stream));
_stream = stream;
}
protected override Stream RequestInitialStream() => _stream;
protected override void ValidateArchive(AceVolume archive)
{
if (archive.IsMultiVolume)
{
throw new MultiVolumeExtractionException(
"Streamed archive is a Multi-volume archive. Use a different AceReader method to extract."
);
}
}
}
}

View File

@@ -70,7 +70,7 @@ public static class ReaderFactory
}
throw new InvalidFormatException(
"Cannot determine compressed stream type. Supported Reader Formats: Arc, Arj, Zip, GZip, BZip2, Tar, Rar, LZip, XZ, ZStandard"
"Cannot determine compressed stream type. Supported Reader Formats: Ace, Arc, Arj, Zip, GZip, BZip2, Tar, Rar, LZip, XZ, ZStandard"
);
}
}

View File

@@ -34,6 +34,7 @@ internal class ZipCentralDirectoryEntry
internal ulong Decompressed { get; set; }
internal ushort Zip64HeaderOffset { get; set; }
internal ulong HeaderOffset { get; }
internal string FileName => fileName;
internal uint Write(Stream outputStream)
{

View File

@@ -8,6 +8,7 @@ using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Common.Zip;
using SharpCompress.Common.Zip.Headers;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Compressors;
using SharpCompress.Compressors.BZip2;
using SharpCompress.Compressors.Deflate;
@@ -27,12 +28,19 @@ public class ZipWriter : AbstractWriter
private long streamPosition;
private PpmdProperties? ppmdProps;
private readonly bool isZip64;
private readonly bool enableSOZip;
private readonly int sozipChunkSize;
private readonly long sozipMinFileSize;
public ZipWriter(Stream destination, ZipWriterOptions zipWriterOptions)
: base(ArchiveType.Zip, zipWriterOptions)
{
zipComment = zipWriterOptions.ArchiveComment ?? string.Empty;
isZip64 = zipWriterOptions.UseZip64;
enableSOZip = zipWriterOptions.EnableSOZip;
sozipChunkSize = zipWriterOptions.SOZipChunkSize;
sozipMinFileSize = zipWriterOptions.SOZipMinFileSize;
if (destination.CanSeek)
{
streamPosition = destination.Position;
@@ -117,12 +125,21 @@ public class ZipWriter : AbstractWriter
var headersize = (uint)WriteHeader(entryPath, options, entry, useZip64);
streamPosition += headersize;
// Determine if SOZip should be used for this entry
var useSozip =
(options.EnableSOZip ?? enableSOZip)
&& compression == ZipCompressionMethod.Deflate
&& OutputStream.CanSeek;
return new ZipWritingStream(
this,
OutputStream.NotNull(),
entry,
compression,
options.CompressionLevel ?? compressionLevel
options.CompressionLevel ?? compressionLevel,
useSozip,
useSozip ? sozipChunkSize : 0
);
}
@@ -304,6 +321,64 @@ public class ZipWriter : AbstractWriter
OutputStream.Write(intBuf);
}
private void WriteSozipIndexFile(
ZipCentralDirectoryEntry dataEntry,
SOZipDeflateStream sozipStream
)
{
var indexFileName = SOZipIndex.GetIndexFileName(dataEntry.FileName);
// Create the SOZip index
var index = new SOZipIndex(
chunkSize: sozipStream.ChunkSize,
uncompressedSize: sozipStream.UncompressedBytesWritten,
compressedSize: sozipStream.CompressedBytesWritten,
compressedOffsets: sozipStream.CompressedOffsets
);
var indexBytes = index.ToByteArray();
// Calculate CRC for index data
var crc = new CRC32();
crc.SlurpBlock(indexBytes, 0, indexBytes.Length);
var indexCrc = (uint)crc.Crc32Result;
// Write the index file as a stored (uncompressed) entry
var indexEntry = new ZipCentralDirectoryEntry(
ZipCompressionMethod.None,
indexFileName,
(ulong)streamPosition,
WriterOptions.ArchiveEncoding
)
{
ModificationTime = DateTime.Now,
};
// Write the local file header for index
var indexOptions = new ZipWriterEntryOptions { CompressionType = CompressionType.None };
var headerSize = (uint)WriteHeader(indexFileName, indexOptions, indexEntry, isZip64);
streamPosition += headerSize;
// Write the index data directly
OutputStream.Write(indexBytes, 0, indexBytes.Length);
// Finalize the index entry
indexEntry.Crc = indexCrc;
indexEntry.Compressed = (ulong)indexBytes.Length;
indexEntry.Decompressed = (ulong)indexBytes.Length;
if (OutputStream.CanSeek)
{
// Update the header with sizes and CRC
OutputStream.Position = (long)(indexEntry.HeaderOffset + 14);
WriteFooter(indexCrc, (uint)indexBytes.Length, (uint)indexBytes.Length);
OutputStream.Position = streamPosition + indexBytes.Length;
}
streamPosition += indexBytes.Length;
entries.Add(indexEntry);
}
private void WriteEndRecord(ulong size)
{
var zip64EndOfCentralDirectoryNeeded =
@@ -385,7 +460,10 @@ public class ZipWriter : AbstractWriter
private readonly ZipWriter writer;
private readonly ZipCompressionMethod zipCompressionMethod;
private readonly int compressionLevel;
private readonly bool useSozip;
private readonly int sozipChunkSize;
private SharpCompressStream? counting;
private SOZipDeflateStream? sozipStream;
private ulong decompressed;
// Flag to prevent throwing exceptions on Dispose
@@ -397,7 +475,9 @@ public class ZipWriter : AbstractWriter
Stream originalStream,
ZipCentralDirectoryEntry entry,
ZipCompressionMethod zipCompressionMethod,
int compressionLevel
int compressionLevel,
bool useSozip = false,
int sozipChunkSize = 0
)
{
this.writer = writer;
@@ -406,6 +486,8 @@ public class ZipWriter : AbstractWriter
this.entry = entry;
this.zipCompressionMethod = zipCompressionMethod;
this.compressionLevel = compressionLevel;
this.useSozip = useSozip;
this.sozipChunkSize = sozipChunkSize;
writeStream = GetWriteStream(originalStream);
}
@@ -435,6 +517,15 @@ public class ZipWriter : AbstractWriter
}
case ZipCompressionMethod.Deflate:
{
if (useSozip && sozipChunkSize > 0)
{
sozipStream = new SOZipDeflateStream(
counting,
(CompressionLevel)compressionLevel,
sozipChunkSize
);
return sozipStream;
}
return new DeflateStream(
counting,
CompressionMode.Compress,
@@ -581,7 +672,18 @@ public class ZipWriter : AbstractWriter
writer.WriteFooter(entry.Crc, compressedvalue, decompressedvalue);
writer.streamPosition += (long)entry.Compressed + 16;
}
writer.entries.Add(entry);
// Write SOZip index file if SOZip was used and file meets minimum size
if (
useSozip
&& sozipStream is not null
&& entry.Decompressed >= (ulong)writer.sozipMinFileSize
)
{
writer.WriteSozipIndexFile(entry, sozipStream);
}
}
}

View File

@@ -49,4 +49,11 @@ public class ZipWriterEntryOptions
/// This option is not supported with non-seekable streams.
/// </summary>
public bool? EnableZip64 { get; set; }
/// <summary>
/// Enable or disable SOZip (Seek-Optimized ZIP) for this entry.
/// When null, uses the archive's default setting.
/// SOZip is only applicable to Deflate-compressed files on seekable streams.
/// </summary>
public bool? EnableSOZip { get; set; }
}

View File

@@ -1,5 +1,6 @@
using System;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Compressors.Deflate;
using D = SharpCompress.Compressors.Deflate;
@@ -24,6 +25,9 @@ public class ZipWriterOptions : WriterOptions
{
UseZip64 = writerOptions.UseZip64;
ArchiveComment = writerOptions.ArchiveComment;
EnableSOZip = writerOptions.EnableSOZip;
SOZipChunkSize = writerOptions.SOZipChunkSize;
SOZipMinFileSize = writerOptions.SOZipMinFileSize;
}
}
@@ -80,4 +84,27 @@ public class ZipWriterOptions : WriterOptions
/// are less than 4GiB in length.
/// </summary>
public bool UseZip64 { get; set; }
/// <summary>
/// Enables SOZip (Seek-Optimized ZIP) for Deflate-compressed files.
/// When enabled, files that meet the minimum size requirement will have
/// an accompanying index file that allows random access within the
/// compressed data. Requires a seekable output stream.
/// </summary>
public bool EnableSOZip { get; set; }
/// <summary>
/// The chunk size for SOZip index creation in bytes.
/// Must be a multiple of 1024 bytes. Default is 32KB (32768 bytes).
/// Smaller chunks allow for finer-grained random access but result
/// in larger index files and slightly less efficient compression.
/// </summary>
public int SOZipChunkSize { get; set; } = (int)SOZipIndex.DEFAULT_CHUNK_SIZE;
/// <summary>
/// Minimum file size (uncompressed) in bytes for SOZip optimization.
/// Files smaller than this size will not have SOZip index files created.
/// Default is 1MB (1048576 bytes).
/// </summary>
public long SOZipMinFileSize { get; set; } = 1048576;
}

View File

@@ -0,0 +1,61 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Readers;
using SharpCompress.Readers.Ace;
using Xunit;
namespace SharpCompress.Test.Ace
{
public class AceReaderTests : ReaderTests
{
public AceReaderTests()
{
UseExtensionInsteadOfNameToVerify = true;
UseCaseInsensitiveToVerify = true;
}
[Fact]
public void Ace_Uncompressed_Read() => Read("Ace.store.ace", CompressionType.None);
[Fact]
public void Ace_Encrypted_Read()
{
var exception = Assert.Throws<CryptographicException>(() => Read("Ace.encrypted.ace"));
}
[Theory]
[InlineData("Ace.method1.ace", CompressionType.AceLZ77)]
[InlineData("Ace.method1-solid.ace", CompressionType.AceLZ77)]
[InlineData("Ace.method2.ace", CompressionType.AceLZ77)]
[InlineData("Ace.method2-solid.ace", CompressionType.AceLZ77)]
public void Ace_Unsupported_ShouldThrow(string fileName, CompressionType compressionType)
{
var exception = Assert.Throws<NotSupportedException>(() =>
Read(fileName, compressionType)
);
}
[Theory]
[InlineData("Ace.store.largefile.ace", CompressionType.None)]
public void Ace_LargeFileTest_Read(string fileName, CompressionType compressionType)
{
ReadForBufferBoundaryCheck(fileName, compressionType);
}
[Fact]
public void Ace_Multi_Reader()
{
var exception = Assert.Throws<MultiVolumeExtractionException>(() =>
DoMultiReader(
["Ace.store.split.ace", "Ace.store.split.c01"],
streams => AceReader.Open(streams)
)
);
}
}
}

View File

@@ -45,14 +45,17 @@ namespace SharpCompress.Test.Arj
public void Arj_Multi_Reader()
{
var exception = Assert.Throws<MultiVolumeExtractionException>(() =>
DoArj_Multi_Reader([
"Arj.store.split.arj",
"Arj.store.split.a01",
"Arj.store.split.a02",
"Arj.store.split.a03",
"Arj.store.split.a04",
"Arj.store.split.a05",
])
DoMultiReader(
[
"Arj.store.split.arj",
"Arj.store.split.a01",
"Arj.store.split.a02",
"Arj.store.split.a03",
"Arj.store.split.a04",
"Arj.store.split.a05",
],
streams => ArjReader.Open(streams)
)
);
}
@@ -74,26 +77,5 @@ namespace SharpCompress.Test.Arj
{
ReadForBufferBoundaryCheck(fileName, compressionType);
}
private void DoArj_Multi_Reader(string[] archives)
{
using (
var reader = ArjReader.Open(
archives
.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s))
.Select(p => File.OpenRead(p))
)
)
{
while (reader.MoveToNextEntry())
{
reader.WriteEntryToDirectory(
SCRATCH_FILES_PATH,
new ExtractionOptions { ExtractFullPath = true, Overwrite = true }
);
}
}
VerifyFiles();
}
}
}

View File

@@ -0,0 +1,65 @@
using System;
using System.IO;
namespace SharpCompress.Test.Mocks;
/// <summary>
/// A stream wrapper that truncates the underlying stream after reading a specified number of bytes.
/// Used for testing error handling when streams end prematurely.
/// </summary>
public class TruncatedStream : Stream
{
private readonly Stream baseStream;
private readonly long truncateAfterBytes;
private long bytesRead;
public TruncatedStream(Stream baseStream, long truncateAfterBytes)
{
this.baseStream = baseStream ?? throw new ArgumentNullException(nameof(baseStream));
this.truncateAfterBytes = truncateAfterBytes;
bytesRead = 0;
}
public override bool CanRead => baseStream.CanRead;
public override bool CanSeek => baseStream.CanSeek;
public override bool CanWrite => false;
public override long Length => baseStream.Length;
public override long Position
{
get => baseStream.Position;
set => baseStream.Position = value;
}
public override int Read(byte[] buffer, int offset, int count)
{
if (bytesRead >= truncateAfterBytes)
{
// Simulate premature end of stream
return 0;
}
var maxBytesToRead = (int)Math.Min(count, truncateAfterBytes - bytesRead);
var actualBytesRead = baseStream.Read(buffer, offset, maxBytesToRead);
bytesRead += actualBytesRead;
return actualBytesRead;
}
public override long Seek(long offset, SeekOrigin origin) => baseStream.Seek(offset, origin);
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) =>
throw new NotSupportedException();
public override void Flush() => baseStream.Flush();
protected override void Dispose(bool disposing)
{
if (disposing)
{
baseStream?.Dispose();
}
base.Dispose(disposing);
}
}

View File

@@ -1,3 +1,4 @@
using System;
using System.IO;
using System.Linq;
using SharpCompress.Archives;
@@ -5,6 +6,7 @@ using SharpCompress.Archives.Rar;
using SharpCompress.Common;
using SharpCompress.Compressors.LZMA.Utilites;
using SharpCompress.Readers;
using SharpCompress.Test.Mocks;
using Xunit;
namespace SharpCompress.Test.Rar;
@@ -642,4 +644,77 @@ public class RarArchiveTests : ArchiveTests
);
Assert.True(passwordProtectedFilesArchive.IsEncrypted);
}
/// <summary>
/// Test for issue: InvalidOperationException when extracting RAR files.
/// This test verifies the fix for the validation logic that was changed from
/// (_position != Length) to (_position &lt; Length).
/// The old logic would throw an exception when position exceeded expected length,
/// but the new logic only throws when decompression ends prematurely (position &lt; expected).
/// </summary>
[Fact]
public void Rar_StreamValidation_OnlyThrowsOnPrematureEnd()
{
// Test normal extraction - should NOT throw InvalidOperationException
// even if actual decompressed size differs from header
var testFiles = new[] { "Rar.rar", "Rar5.rar", "Rar4.rar", "Rar2.rar" };
foreach (var testFile in testFiles)
{
using var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, testFile));
using var archive = RarArchive.Open(stream);
// Extract all entries and read them completely
foreach (var entry in archive.Entries.Where(e => !e.IsDirectory))
{
using var entryStream = entry.OpenEntryStream();
using var ms = new MemoryStream();
// This should complete without throwing InvalidOperationException
// The fix ensures we only throw when position &lt; expected length, not when position >= expected
entryStream.CopyTo(ms);
// Verify we read some data
Assert.True(
ms.Length > 0,
$"Failed to extract data from {entry.Key} in {testFile}"
);
}
}
}
/// <summary>
/// Negative test case: Verifies that InvalidOperationException IS thrown when
/// a RAR stream ends prematurely (position &lt; expected length).
/// This tests the validation condition (_position &lt; Length) works correctly.
/// </summary>
[Fact]
public void Rar_StreamValidation_ThrowsOnTruncatedStream()
{
// This test verifies the exception is thrown when decompression ends prematurely
// by using a truncated stream that stops reading after a small number of bytes
var testFile = "Rar.rar";
using var fileStream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, testFile));
// Wrap the file stream with a truncated stream that will stop reading early
// This simulates a corrupted or truncated RAR file
using var truncatedStream = new TruncatedStream(fileStream, 1000);
// Opening the archive should work, but extracting should throw
// when we try to read beyond the truncated data
var exception = Assert.Throws<InvalidOperationException>(() =>
{
using var archive = RarArchive.Open(truncatedStream);
foreach (var entry in archive.Entries.Where(e => !e.IsDirectory))
{
using var entryStream = entry.OpenEntryStream();
using var ms = new MemoryStream();
// This should throw InvalidOperationException when it can't read all expected bytes
entryStream.CopyTo(ms);
}
});
// Verify the exception message matches our expectation
Assert.Contains("unpacked file size does not match header", exception.Message);
}
}

View File

@@ -1,6 +1,7 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using SharpCompress.Common;
@@ -220,4 +221,26 @@ public abstract class ReaderTests : TestBase
Assert.Equal(expected.Pop(), reader.Entry.Key);
}
}
protected void DoMultiReader(
string[] archives,
Func<IEnumerable<Stream>, IDisposable> readerFactory
)
{
using var reader = readerFactory(
archives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)).Select(File.OpenRead)
);
dynamic dynReader = reader;
while (dynReader.MoveToNextEntry())
{
dynReader.WriteEntryToDirectory(
SCRATCH_FILES_PATH,
new ExtractionOptions { ExtractFullPath = true, Overwrite = true }
);
}
VerifyFiles();
}
}

View File

@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers;
using Xunit;
@@ -45,7 +46,7 @@ public class TestBase : IDisposable
public void Dispose() => Directory.Delete(SCRATCH_BASE_PATH, true);
public void VerifyFiles()
public void VerifyFiles(bool skipSoIndexes = false)
{
if (UseExtensionInsteadOfNameToVerify)
{
@@ -53,7 +54,7 @@ public class TestBase : IDisposable
}
else
{
VerifyFilesByName();
VerifyFilesByName(skipSoIndexes);
}
}
@@ -72,10 +73,23 @@ public class TestBase : IDisposable
}
}
protected void VerifyFilesByName()
private void VerifyFilesByName(bool skipSoIndexes)
{
var extracted = Directory
.EnumerateFiles(SCRATCH_FILES_PATH, "*.*", SearchOption.AllDirectories)
.Where(x =>
{
if (
skipSoIndexes
&& Path.GetFileName(x)
.EndsWith(SOZipIndex.INDEX_EXTENSION, StringComparison.OrdinalIgnoreCase)
)
{
return false;
}
return true;
})
.ToLookup(path => path.Substring(SCRATCH_FILES_PATH.Length));
var original = Directory
.EnumerateFiles(ORIGINAL_FILES_PATH, "*.*", SearchOption.AllDirectories)

View File

@@ -0,0 +1,257 @@
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Archives.Zip;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers.Zip;
using SharpCompress.Test.Mocks;
using SharpCompress.Writers;
using SharpCompress.Writers.Zip;
using Xunit;
namespace SharpCompress.Test.Zip;
public class SoZipReaderTests : TestBase
{
[Fact]
public async Task SOZip_Reader_RegularZip_NoSozipEntries()
{
// Regular zip files should not have SOZip entries
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ZipReader.Open(stream);
while (await reader.MoveToNextEntryAsync())
{
// Regular zip entries should NOT be SOZip
Assert.False(reader.Entry.IsSozip, $"Entry {reader.Entry.Key} should not be SOZip");
Assert.False(
reader.Entry.IsSozipIndexFile,
$"Entry {reader.Entry.Key} should not be a SOZip index file"
);
}
}
[Fact]
public void SOZip_Archive_RegularZip_NoSozipEntries()
{
// Regular zip files should not have SOZip entries
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.zip");
using Stream stream = File.OpenRead(path);
using var archive = ZipArchive.Open(stream);
foreach (var entry in archive.Entries)
{
// Regular zip entries should NOT be SOZip
Assert.False(entry.IsSozip, $"Entry {entry.Key} should not be SOZip");
Assert.False(
entry.IsSozipIndexFile,
$"Entry {entry.Key} should not be a SOZip index file"
);
}
}
[Fact]
public void SOZip_Archive_ReadSOZipFile()
{
// Read the SOZip test archive
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using Stream stream = File.OpenRead(path);
using var archive = ZipArchive.Open(stream);
var entries = archive.Entries.ToList();
// Should have 3 entries: data.txt, .data.txt.sozip.idx, and small.txt
Assert.Equal(3, entries.Count);
// Verify we have one SOZip index file
var indexFiles = entries.Where(e => e.IsSozipIndexFile).ToList();
Assert.Single(indexFiles);
Assert.Equal(".data.txt.sozip.idx", indexFiles[0].Key);
// Verify the index file is not compressed
Assert.Equal(CompressionType.None, indexFiles[0].CompressionType);
// Read and validate the index
using (var indexStream = indexFiles[0].OpenEntryStream())
{
using var memStream = new MemoryStream();
indexStream.CopyTo(memStream);
var indexBytes = memStream.ToArray();
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
Assert.Equal(1024u, index.ChunkSize); // As set in CreateSOZipTestArchive
Assert.True(index.UncompressedSize > 0);
Assert.True(index.OffsetCount > 0);
}
// Verify the data file can be read correctly
var dataEntry = entries.First(e => e.Key == "data.txt");
using (var dataStream = dataEntry.OpenEntryStream())
{
using var reader = new StreamReader(dataStream);
var content = reader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
// Verify the small file
var smallEntry = entries.First(e => e.Key == "small.txt");
Assert.False(smallEntry.IsSozipIndexFile);
using (var smallStream = smallEntry.OpenEntryStream())
{
using var reader = new StreamReader(smallStream);
var content = reader.ReadToEnd();
Assert.Equal("Small content", content);
}
}
[Fact]
public async Task SOZip_Reader_ReadSOZipFile()
{
// Read the SOZip test archive with ZipReader
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ZipReader.Open(stream);
var foundData = false;
var foundIndex = false;
var foundSmall = false;
while (await reader.MoveToNextEntryAsync())
{
if (reader.Entry.Key == "data.txt")
{
foundData = true;
Assert.False(reader.Entry.IsSozipIndexFile);
using var entryStream = reader.OpenEntryStream();
using var streamReader = new StreamReader(entryStream);
var content = streamReader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
else if (reader.Entry.Key == ".data.txt.sozip.idx")
{
foundIndex = true;
Assert.True(reader.Entry.IsSozipIndexFile);
using var indexStream = reader.OpenEntryStream();
using var memStream = new MemoryStream();
await indexStream.CopyToAsync(memStream);
var indexBytes = memStream.ToArray();
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
}
else if (reader.Entry.Key == "small.txt")
{
foundSmall = true;
Assert.False(reader.Entry.IsSozipIndexFile);
}
}
Assert.True(foundData, "data.txt entry not found");
Assert.True(foundIndex, ".data.txt.sozip.idx entry not found");
Assert.True(foundSmall, "small.txt entry not found");
}
[Fact]
public void SOZip_Archive_DetectsIndexFileByName()
{
// Create a zip with a SOZip index file (by name pattern)
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file that looks like a SOZip index (by name pattern)
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
// Test with ZipArchive
using var archive = ZipArchive.Open(memoryStream);
var entries = archive.Entries.ToList();
Assert.Equal(2, entries.Count);
var regularEntry = entries.First(e => e.Key == "test.txt");
Assert.False(regularEntry.IsSozipIndexFile);
Assert.False(regularEntry.IsSozip); // No SOZip extra field
var indexEntry = entries.First(e => e.Key == ".test.txt.sozip.idx");
Assert.True(indexEntry.IsSozipIndexFile);
}
[Fact]
public async Task SOZip_Reader_DetectsIndexFileByName()
{
// Create a zip with a SOZip index file (by name pattern)
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file that looks like a SOZip index (by name pattern)
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
// Test with ZipReader
using Stream stream = new ForwardOnlyStream(memoryStream);
using var reader = ZipReader.Open(stream);
var foundRegular = false;
var foundIndex = false;
while (await reader.MoveToNextEntryAsync())
{
if (reader.Entry.Key == "test.txt")
{
foundRegular = true;
Assert.False(reader.Entry.IsSozipIndexFile);
Assert.False(reader.Entry.IsSozip);
}
else if (reader.Entry.Key == ".test.txt.sozip.idx")
{
foundIndex = true;
Assert.True(reader.Entry.IsSozipIndexFile);
}
}
Assert.True(foundRegular, "Regular entry not found");
Assert.True(foundIndex, "Index entry not found");
}
}

View File

@@ -0,0 +1,358 @@
using System;
using System.IO;
using System.Linq;
using System.Text;
using SharpCompress.Archives.Zip;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers;
using SharpCompress.Writers;
using SharpCompress.Writers.Zip;
using Xunit;
namespace SharpCompress.Test.Zip;
public class SoZipWriterTests : TestBase
{
[Fact]
public void SOZipIndex_RoundTrip()
{
// Create an index
var offsets = new ulong[] { 0, 1024, 2048, 3072 };
var originalIndex = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100000,
compressedSize: 50000,
compressedOffsets: offsets
);
// Serialize to bytes
var bytes = originalIndex.ToByteArray();
// Deserialize back
var parsedIndex = SOZipIndex.Read(bytes);
// Verify all fields
Assert.Equal(SOZipIndex.SOZIP_VERSION, parsedIndex.Version);
Assert.Equal(32768u, parsedIndex.ChunkSize);
Assert.Equal(100000ul, parsedIndex.UncompressedSize);
Assert.Equal(50000ul, parsedIndex.CompressedSize);
Assert.Equal(4u, parsedIndex.OffsetCount);
Assert.Equal(offsets, parsedIndex.CompressedOffsets);
}
[Fact]
public void SOZipIndex_Read_InvalidMagic_ThrowsException()
{
var invalidData = new byte[] { 0x00, 0x00, 0x00, 0x00 };
var exception = Assert.Throws<InvalidDataException>(() => SOZipIndex.Read(invalidData));
Assert.Contains("magic number mismatch", exception.Message);
}
[Fact]
public void SOZipIndex_GetChunkIndex()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840, // 5 * 32768
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0, index.GetChunkIndex(0));
Assert.Equal(0, index.GetChunkIndex(32767));
Assert.Equal(1, index.GetChunkIndex(32768));
Assert.Equal(2, index.GetChunkIndex(65536));
Assert.Equal(4, index.GetChunkIndex(163839));
}
[Fact]
public void SOZipIndex_GetCompressedOffset()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840,
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0ul, index.GetCompressedOffset(0));
Assert.Equal(1000ul, index.GetCompressedOffset(1));
Assert.Equal(2000ul, index.GetCompressedOffset(2));
Assert.Equal(3000ul, index.GetCompressedOffset(3));
Assert.Equal(4000ul, index.GetCompressedOffset(4));
}
[Fact]
public void SOZipIndex_GetUncompressedOffset()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840,
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0ul, index.GetUncompressedOffset(0));
Assert.Equal(32768ul, index.GetUncompressedOffset(1));
Assert.Equal(65536ul, index.GetUncompressedOffset(2));
Assert.Equal(98304ul, index.GetUncompressedOffset(3));
Assert.Equal(131072ul, index.GetUncompressedOffset(4));
}
[Fact]
public void SOZipIndex_GetIndexFileName()
{
Assert.Equal(".file.txt.sozip.idx", SOZipIndex.GetIndexFileName("file.txt"));
Assert.Equal("dir/.file.txt.sozip.idx", SOZipIndex.GetIndexFileName("dir/file.txt"));
Assert.Equal("a/b/.file.txt.sozip.idx", SOZipIndex.GetIndexFileName("a/b/file.txt"));
}
[Fact]
public void SOZipIndex_IsIndexFile()
{
Assert.True(SOZipIndex.IsIndexFile(".file.txt.sozip.idx"));
Assert.True(SOZipIndex.IsIndexFile("dir/.file.txt.sozip.idx"));
Assert.True(SOZipIndex.IsIndexFile(".test.sozip.idx"));
Assert.False(SOZipIndex.IsIndexFile("file.txt"));
Assert.False(SOZipIndex.IsIndexFile("file.sozip.idx")); // Missing leading dot
Assert.False(SOZipIndex.IsIndexFile(".file.txt")); // Missing .sozip.idx
Assert.False(SOZipIndex.IsIndexFile(""));
Assert.False(SOZipIndex.IsIndexFile(null!));
}
[Fact]
public void SOZipIndex_GetMainFileName()
{
Assert.Equal("file.txt", SOZipIndex.GetMainFileName(".file.txt.sozip.idx"));
Assert.Equal("dir/file.txt", SOZipIndex.GetMainFileName("dir/.file.txt.sozip.idx"));
Assert.Equal("test", SOZipIndex.GetMainFileName(".test.sozip.idx"));
Assert.Null(SOZipIndex.GetMainFileName("file.txt"));
Assert.Null(SOZipIndex.GetMainFileName(""));
}
[Fact]
public void ZipEntry_IsSozipIndexFile_Detection()
{
// Create a zip with a file that has a SOZip index file name pattern
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file with SOZip index name pattern
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
using var archive = ZipArchive.Open(memoryStream);
var entries = archive.Entries.ToList();
Assert.Equal(2, entries.Count);
var regularEntry = entries.First(e => e.Key == "test.txt");
Assert.False(regularEntry.IsSozipIndexFile);
Assert.False(regularEntry.IsSozip); // No SOZip extra field
var indexEntry = entries.First(e => e.Key == ".test.txt.sozip.idx");
Assert.True(indexEntry.IsSozipIndexFile);
}
[Fact]
public void ZipWriterOptions_SOZipDefaults()
{
var options = new ZipWriterOptions(CompressionType.Deflate);
Assert.False(options.EnableSOZip);
Assert.Equal((int)SOZipIndex.DEFAULT_CHUNK_SIZE, options.SOZipChunkSize);
Assert.Equal(1048576L, options.SOZipMinFileSize); // 1MB
}
[Fact]
public void ZipWriterEntryOptions_SOZipDefaults()
{
var options = new ZipWriterEntryOptions();
Assert.Null(options.EnableSOZip);
}
[Fact]
public void SOZip_RoundTrip_CompressAndDecompress()
{
// Create a SOZip archive from Original files
var archivePath = Path.Combine(SCRATCH2_FILES_PATH, "test.sozip.zip");
using (var stream = File.Create(archivePath))
{
var options = new ZipWriterOptions(CompressionType.Deflate)
{
EnableSOZip = true,
SOZipMinFileSize = 1024, // 1KB to ensure test files qualify
LeaveStreamOpen = false,
};
using var writer = new ZipWriter(stream, options);
// Write all files from Original directory
var files = Directory.GetFiles(ORIGINAL_FILES_PATH, "*", SearchOption.AllDirectories);
foreach (var filePath in files)
{
var relativePath = filePath
.Substring(ORIGINAL_FILES_PATH.Length + 1)
.Replace('\\', '/');
using var fileStream = File.OpenRead(filePath);
writer.Write(relativePath, fileStream, new ZipWriterEntryOptions());
}
}
// Validate the archive was created and has files
Assert.True(File.Exists(archivePath));
// Validate the archive has SOZip entries
using (var stream = File.OpenRead(archivePath))
{
using var archive = ZipArchive.Open(stream);
var allEntries = archive.Entries.ToList();
// Archive should have files
Assert.NotEmpty(allEntries);
var sozipIndexEntries = allEntries.Where(e => e.IsSozipIndexFile).ToList();
// Should have at least one SOZip index file
Assert.NotEmpty(sozipIndexEntries);
// Verify index files have valid SOZip index data
foreach (var indexEntry in sozipIndexEntries)
{
// Check that the entry is stored (not compressed)
Assert.Equal(CompressionType.None, indexEntry.CompressionType);
using var indexStream = indexEntry.OpenEntryStream();
using var memStream = new MemoryStream();
indexStream.CopyTo(memStream);
var indexBytes = memStream.ToArray();
// Debug: Check first 4 bytes
Assert.True(
indexBytes.Length >= 4,
$"Index file too small: {indexBytes.Length} bytes"
);
// Should be able to parse the index without exception
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
Assert.True(index.ChunkSize > 0);
Assert.True(index.UncompressedSize > 0);
Assert.True(index.OffsetCount > 0);
// Verify there's a corresponding data file
var mainFileName = SOZipIndex.GetMainFileName(indexEntry.Key!);
Assert.NotNull(mainFileName);
Assert.Contains(allEntries, e => e.Key == mainFileName);
}
}
// Read and decompress the archive
using (var stream = File.OpenRead(archivePath))
{
using var reader = ReaderFactory.Open(stream);
reader.WriteAllToDirectory(
SCRATCH_FILES_PATH,
new ExtractionOptions { ExtractFullPath = true }
);
}
// Verify extracted files match originals
VerifyFiles(true);
}
[Fact]
public void CreateSOZipTestArchive()
{
// Create a SOZip test archive that can be committed to the repository
var archivePath = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using (var stream = File.Create(archivePath))
{
var options = new ZipWriterOptions(CompressionType.Deflate)
{
EnableSOZip = true,
SOZipMinFileSize = 100, // Low threshold to ensure test content is optimized
SOZipChunkSize = 1024, // Small chunks for testing
LeaveStreamOpen = false,
};
using var writer = new ZipWriter(stream, options);
// Create test content that's large enough to create multiple chunks
var largeContent = new string('A', 5000); // 5KB of 'A's
// Write a file with enough data to be SOZip-optimized
writer.Write(
"data.txt",
new MemoryStream(Encoding.UTF8.GetBytes(largeContent)),
new ZipWriterEntryOptions()
);
// Write a smaller file that won't be SOZip-optimized
writer.Write(
"small.txt",
new MemoryStream(Encoding.UTF8.GetBytes("Small content")),
new ZipWriterEntryOptions()
);
}
// Validate the archive was created
Assert.True(File.Exists(archivePath));
// Validate it's a valid SOZip archive
using (var stream = File.OpenRead(archivePath))
{
using var archive = ZipArchive.Open(stream);
var entries = archive.Entries.ToList();
// Should have data file, small file, and index file
Assert.Equal(3, entries.Count);
// Verify we have one SOZip index file
var indexFiles = entries.Where(e => e.IsSozipIndexFile).ToList();
Assert.Single(indexFiles);
// Verify the index file
var indexEntry = indexFiles.First();
Assert.Equal(".data.txt.sozip.idx", indexEntry.Key);
// Verify the data file can be read
var dataEntry = entries.First(e => e.Key == "data.txt");
using var dataStream = dataEntry.OpenEntryStream();
using var reader = new StreamReader(dataStream);
var content = reader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
}
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.