Compare commits

..

23 Commits

Author SHA1 Message Date
Adam Hathcock
d5a8c37113 Merge pull request #1154 from adamhathcock/adam/1151-release
Adam/1151 release cherry pick
2026-01-23 09:31:03 +00:00
Adam Hathcock
21ce9a38e6 fix up tests 2026-01-23 09:04:55 +00:00
Adam Hathcock
7732fbb698 Merge pull request #1151 from adamhathcock/copilot/fix-entrystream-flush-issue
Fix EntryStream.Dispose() throwing NotSupportedException on non-seekable streams
2026-01-23 08:59:56 +00:00
Adam Hathcock
97879f18b6 Merge pull request #1146 from adamhathcock/adam/pr-1145-release
Merge pull request #1145 from adamhathcock/copilot/add-leaveopen-para…
2026-01-19 10:35:33 +00:00
Adam Hathcock
d74454f7e9 Merge pull request #1145 from adamhathcock/copilot/add-leaveopen-parameter-lzipstream
Add leaveOpen parameter to LZipStream and BZip2Stream
2026-01-19 09:58:10 +00:00
Adam Hathcock
5c947bccc7 Merge branch 'adam/update-docs' 2026-01-07 16:18:51 +00:00
Adam Hathcock
fbdefc17c1 updates from review 2026-01-07 16:18:27 +00:00
Adam Hathcock
1425c6ff0d Merge pull request #1120 from adamhathcock/adam/update-docs
Update docs
2026-01-07 16:12:51 +00:00
Adam Hathcock
e038aea694 move old changelog 2026-01-07 16:10:55 +00:00
Adam Hathcock
87ccbf329d moved examples to USAGE 2026-01-07 15:56:38 +00:00
Adam Hathcock
9dcf384263 update for progress reporting 2026-01-07 15:30:26 +00:00
Adam Hathcock
be045c4f15 Merge pull request #1114 from adamhathcock/copilot/fix-7z-file-decompression-error
Fix async decompression of .7z files by implementing Memory<byte> ReadAsync overload
2026-01-07 08:16:51 +00:00
Adam Hathcock
fd968b3f78 Update src/SharpCompress/IO/ReadOnlySubStream.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-06 16:33:03 +00:00
Adam Hathcock
833dd7b3a2 fix tests and fmt 2026-01-06 15:33:43 +00:00
Adam Hathcock
b9258ad496 use more ValueTask methods but types are still created because of state machine suspension 2026-01-06 15:26:49 +00:00
copilot-swe-agent[bot]
0678318dde Fix async decompression by implementing Memory<byte> ReadAsync overload
The issue was that .NET 10's ReadExactlyAsync calls the Memory<byte> overload of ReadAsync, which wasn't implemented in BufferedSubStream. This caused it to fall back to the base Stream implementation that uses synchronous reads, leading to cache state corruption.

Solution: Added ValueTask<int> ReadAsync(Memory<byte>, CancellationToken) overload for modern .NET versions.

All tests now passing including LZMA2 and Solid archives.

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-06 14:18:15 +00:00
copilot-swe-agent[bot]
7116c0d098 Add async support to BufferedSubStream for 7zip decompression
- Implemented ReadAsync and RefillCacheAsync methods in BufferedSubStream
- Added async test cases for SevenZipArchive (LZMA, LZMA2, Solid, BZip2, PPMd)
- Tests show LZMA, BZip2, and PPMd working correctly
- LZMA2 and Solid archives still failing with Data Error - investigating cache state management

Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
2026-01-06 14:13:48 +00:00
copilot-swe-agent[bot]
2fde8436fb Initial plan 2026-01-06 14:00:03 +00:00
Adam Hathcock
61ecd6475f Merge pull request #1113 from adamhathcock/adam/fix-readonly-dispose
Fix a usage of ReadOnly that use dispose in 7Zip
2026-01-06 13:22:50 +00:00
Adam Hathcock
64b209a772 add disposal tests and fix lzipstream 2026-01-06 13:13:34 +00:00
Adam Hathcock
48dbdbfed5 fmt 2026-01-06 12:49:26 +00:00
Adam Hathcock
cf50311b9c Skip should use framework stuff 2026-01-06 12:45:13 +00:00
Adam Hathcock
e4d8582a2a 7zip streams always want to be disposed 2026-01-06 12:42:48 +00:00
35 changed files with 1167 additions and 1681 deletions

View File

@@ -25,7 +25,7 @@
| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Decompress | SevenZipArchive | N/A | N/A |
1. SOLID Rars are only supported in the RarReader API.
2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. Deflate64 is only supported for reading. SOZip (Seek-Optimized ZIP) detection is supported for reading. See [Zip Format Notes](#zip-format-notes) for details on multi-volume archives and streaming behavior.
2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. Deflate64 is only supported for reading. See [Zip Format Notes](#zip-format-notes) for details on multi-volume archives and streaming behavior.
3. The Tar format requires a file size in the header. If no size is specified to the TarWriter and the stream is not seekable, then an exception will be thrown.
4. The 7Zip format doesn't allow for reading as a forward-only stream so 7Zip is only supported through the Archive API. See [7Zip Format Notes](#7zip-format-notes) for details on async extraction behavior.
5. LZip has no support for extra data like the file name or timestamp. There is a default filename used when looking at the entry Key on the archive.

142
OLD_CHANGELOG.md Normal file
View File

@@ -0,0 +1,142 @@
# Version Log
* [Releases](https://github.com/adamhathcock/sharpcompress/releases)
## Version 0.18
* [Now on Github releases](https://github.com/adamhathcock/sharpcompress/releases/tag/0.18)
## Version 0.17.1
* Fix - [Bug Fix for .NET Core on Windows](https://github.com/adamhathcock/sharpcompress/pull/257)
## Version 0.17.0
* New - Full LZip support! Can read and write LZip files and Tars inside LZip files. [Make LZip a first class citizen. #241](https://github.com/adamhathcock/sharpcompress/issues/241)
* New - XZ read support! Can read XZ files and Tars inside XZ files. [XZ in SharpCompress #91](https://github.com/adamhathcock/sharpcompress/issues/94)
* Fix - [Regression - zip file writing on seekable streams always assumed stream start was 0. Introduced with Zip64 writing.](https://github.com/adamhathcock/sharpcompress/issues/244)
* Fix - [Zip files with post-data descriptors can be properly skipped via decompression](https://github.com/adamhathcock/sharpcompress/issues/162)
## Version 0.16.2
* Fix [.NET 3.5 should support files and cryptography (was a regression from 0.16.0)](https://github.com/adamhathcock/sharpcompress/pull/251)
* Fix [Zip per entry compression customization wrote the wrong method into the zip archive](https://github.com/adamhathcock/sharpcompress/pull/249)
## Version 0.16.1
* Fix [Preserve compression method when getting a compressed stream](https://github.com/adamhathcock/sharpcompress/pull/235)
* Fix [RAR entry key normalization fix](https://github.com/adamhathcock/sharpcompress/issues/201)
## Version 0.16.0
* Breaking - [Progress Event Tracking rethink](https://github.com/adamhathcock/sharpcompress/pull/226)
* Update to VS2017 - [VS2017](https://github.com/adamhathcock/sharpcompress/pull/231) - Framework targets have been changed.
* New - [Add Zip64 writing](https://github.com/adamhathcock/sharpcompress/pull/211)
* [Fix invalid/mismatching Zip version flags.](https://github.com/adamhathcock/sharpcompress/issues/164) - This allows nuget/System.IO.Packaging to read zip files generated by SharpCompress
* [Fix 7Zip directory hiding](https://github.com/adamhathcock/sharpcompress/pull/215/files)
* [Verify RAR CRC headers](https://github.com/adamhathcock/sharpcompress/pull/220)
## Version 0.15.2
* [Fix invalid headers](https://github.com/adamhathcock/sharpcompress/pull/210) - fixes an issue creating large-ish zip archives that was introduced with zip64 reading.
## Version 0.15.1
* [Zip64 extending information and ZipReader](https://github.com/adamhathcock/sharpcompress/pull/206)
## Version 0.15.0
* [Add zip64 support for ZipArchive extraction](https://github.com/adamhathcock/sharpcompress/pull/205)
## Version 0.14.1
* [.NET Assemblies aren't strong named](https://github.com/adamhathcock/sharpcompress/issues/158)
* [Pkware encryption for Zip files didn't allow for multiple reads of an entry](https://github.com/adamhathcock/sharpcompress/issues/197)
* [GZip Entry couldn't be read multiple times](https://github.com/adamhathcock/sharpcompress/issues/198)
## Version 0.14.0
* [Support for LZip reading in for Tars](https://github.com/adamhathcock/sharpcompress/pull/191)
## Version 0.13.1
* [Fix null password on ReaderFactory. Fix null options on SevenZipArchive](https://github.com/adamhathcock/sharpcompress/pull/188)
* [Make PpmdProperties lazy to avoid unnecessary allocations.](https://github.com/adamhathcock/sharpcompress/pull/185)
## Version 0.13.0
* Breaking change: Big refactor of Options on API.
* 7Zip supports Deflate
## Version 0.12.4
* Forward only zip issue fix https://github.com/adamhathcock/sharpcompress/issues/160
* Try to fix frameworks again by copying targets from JSON.NET
## Version 0.12.3
* 7Zip fixes https://github.com/adamhathcock/sharpcompress/issues/73
* Maybe all profiles will work with project.json now
## Version 0.12.2
* Support Profile 259 again
## Version 0.12.1
* Support Silverlight 5
## Version 0.12.0
* .NET Core RTM!
* Bug fix for Tar long paths
## Version 0.11.6
* Bug fix for global header in Tar
* Writers now have a leaveOpen `bool` overload. They won't close streams if not-requested to.
## Version 0.11.5
* Bug fix in Skip method
## Version 0.11.4
* SharpCompress is now endian neutral (matters for Mono platforms)
* Fix for Inflate (need to change implementation)
* Fixes for RAR detection
## Version 0.11.1
* Added Cancel on IReader
* Removed .NET 2.0 support and LinqBridge dependency
## Version 0.11
* Been over a year, contains mainly fixes from contributors!
* Possible breaking change: ArchiveEncoding is UTF8 by default now.
* TAR supports writing long names using longlink
* RAR Protect Header added
## Version 0.10.3
* Finally fixed Disposal issue when creating a new archive with the Archive API
## Version 0.10.2
* Fixed Rar Header reading for invalid extended time headers.
* Windows Store assembly is now strong named
* Known issues with Long Tar names being worked on
* Updated to VS2013
* Portable targets SL5 and Windows Phone 8 (up from SL4 and WP7)
## Version 0.10.1
* Fixed 7Zip extraction performance problem
## Version 0.10:
* Added support for RAR Decryption (thanks to https://github.com/hrasyid)
* Embedded some BouncyCastle crypto classes to allow RAR Decryption and Winzip AES Decryption in Portable and Windows Store DLLs
* Built in Release (I think)

228
README.md
View File

@@ -4,7 +4,7 @@ SharpCompress is a compression library in pure C# for .NET Framework 4.8, .NET 8
The major feature is support for non-seekable streams so large files can be processed on the fly (i.e. download stream).
**NEW:** All I/O operations now support async/await for improved performance and scalability. See the [Async Usage](#async-usage) section below.
**NEW:** All I/O operations now support async/await for improved performance and scalability. See the [USAGE.md](USAGE.md#async-examples) for examples.
GitHub Actions Build -
[![SharpCompress](https://github.com/adamhathcock/sharpcompress/actions/workflows/dotnetcore.yml/badge.svg)](https://github.com/adamhathcock/sharpcompress/actions/workflows/dotnetcore.yml)
@@ -34,235 +34,11 @@ Hi everyone. I hope you're using SharpCompress and finding it useful. Please giv
Please do not email me directly to ask for help. If you think there is a real issue, please report it here.
## Async Usage
SharpCompress now provides full async/await support for all I/O operations, allowing for better performance and scalability in modern applications.
### Async Reading Examples
Extract entries asynchronously:
```csharp
using (Stream stream = File.OpenRead("archive.zip"))
using (var reader = ReaderFactory.Open(stream))
{
while (reader.MoveToNextEntry())
{
if (!reader.Entry.IsDirectory)
{
// Async extraction
await reader.WriteEntryToDirectoryAsync(
@"C:\temp",
new ExtractionOptions() { ExtractFullPath = true, Overwrite = true },
cancellationToken
);
}
}
}
```
Extract all entries to directory asynchronously:
```csharp
using (Stream stream = File.OpenRead("archive.tar.gz"))
using (var reader = ReaderFactory.Open(stream))
{
await reader.WriteAllToDirectoryAsync(
@"C:\temp",
new ExtractionOptions() { ExtractFullPath = true, Overwrite = true },
cancellationToken
);
}
```
Open entry stream asynchronously:
```csharp
using (var archive = ZipArchive.Open("archive.zip"))
{
foreach (var entry in archive.Entries.Where(e => !e.IsDirectory))
{
using (var entryStream = await entry.OpenEntryStreamAsync(cancellationToken))
{
// Process stream asynchronously
await entryStream.CopyToAsync(outputStream, cancellationToken);
}
}
}
```
### Async Writing Examples
Write files asynchronously:
```csharp
using (Stream stream = File.OpenWrite("output.zip"))
using (var writer = WriterFactory.Open(stream, ArchiveType.Zip, CompressionType.Deflate))
{
await writer.WriteAsync("file1.txt", fileStream, DateTime.Now, cancellationToken);
}
```
Write all files from directory asynchronously:
```csharp
using (Stream stream = File.OpenWrite("output.tar.gz"))
using (var writer = WriterFactory.Open(stream, ArchiveType.Tar, new WriterOptions(CompressionType.GZip)))
{
await writer.WriteAllAsync(@"D:\files", "*", SearchOption.AllDirectories, cancellationToken);
}
```
All async methods support `CancellationToken` for graceful cancellation of long-running operations.
## Want to contribute?
I'm always looking for help or ideas. Please submit code or email with ideas. Unfortunately, just letting me know you'd like to help is not enough because I really have no overall plan of what needs to be done. I'll definitely accept code submissions and add you as a member of the project!
## TODOs (always lots)
* RAR 5 decryption crc check support
* 7Zip writing
* Zip64 (Need writing and extend Reading)
* Multi-volume Zip support.
* ZStandard writing
## Version Log
* [Releases](https://github.com/adamhathcock/sharpcompress/releases)
### Version 0.18
* [Now on Github releases](https://github.com/adamhathcock/sharpcompress/releases/tag/0.18)
### Version 0.17.1
* Fix - [Bug Fix for .NET Core on Windows](https://github.com/adamhathcock/sharpcompress/pull/257)
### Version 0.17.0
* New - Full LZip support! Can read and write LZip files and Tars inside LZip files. [Make LZip a first class citizen. #241](https://github.com/adamhathcock/sharpcompress/issues/241)
* New - XZ read support! Can read XZ files and Tars inside XZ files. [XZ in SharpCompress #91](https://github.com/adamhathcock/sharpcompress/issues/94)
* Fix - [Regression - zip file writing on seekable streams always assumed stream start was 0. Introduced with Zip64 writing.](https://github.com/adamhathcock/sharpcompress/issues/244)
* Fix - [Zip files with post-data descriptors can be properly skipped via decompression](https://github.com/adamhathcock/sharpcompress/issues/162)
### Version 0.16.2
* Fix [.NET 3.5 should support files and cryptography (was a regression from 0.16.0)](https://github.com/adamhathcock/sharpcompress/pull/251)
* Fix [Zip per entry compression customization wrote the wrong method into the zip archive](https://github.com/adamhathcock/sharpcompress/pull/249)
### Version 0.16.1
* Fix [Preserve compression method when getting a compressed stream](https://github.com/adamhathcock/sharpcompress/pull/235)
* Fix [RAR entry key normalization fix](https://github.com/adamhathcock/sharpcompress/issues/201)
### Version 0.16.0
* Breaking - [Progress Event Tracking rethink](https://github.com/adamhathcock/sharpcompress/pull/226)
* Update to VS2017 - [VS2017](https://github.com/adamhathcock/sharpcompress/pull/231) - Framework targets have been changed.
* New - [Add Zip64 writing](https://github.com/adamhathcock/sharpcompress/pull/211)
* [Fix invalid/mismatching Zip version flags.](https://github.com/adamhathcock/sharpcompress/issues/164) - This allows nuget/System.IO.Packaging to read zip files generated by SharpCompress
* [Fix 7Zip directory hiding](https://github.com/adamhathcock/sharpcompress/pull/215/files)
* [Verify RAR CRC headers](https://github.com/adamhathcock/sharpcompress/pull/220)
### Version 0.15.2
* [Fix invalid headers](https://github.com/adamhathcock/sharpcompress/pull/210) - fixes an issue creating large-ish zip archives that was introduced with zip64 reading.
### Version 0.15.1
* [Zip64 extending information and ZipReader](https://github.com/adamhathcock/sharpcompress/pull/206)
### Version 0.15.0
* [Add zip64 support for ZipArchive extraction](https://github.com/adamhathcock/sharpcompress/pull/205)
### Version 0.14.1
* [.NET Assemblies aren't strong named](https://github.com/adamhathcock/sharpcompress/issues/158)
* [Pkware encryption for Zip files didn't allow for multiple reads of an entry](https://github.com/adamhathcock/sharpcompress/issues/197)
* [GZip Entry couldn't be read multiple times](https://github.com/adamhathcock/sharpcompress/issues/198)
### Version 0.14.0
* [Support for LZip reading in for Tars](https://github.com/adamhathcock/sharpcompress/pull/191)
### Version 0.13.1
* [Fix null password on ReaderFactory. Fix null options on SevenZipArchive](https://github.com/adamhathcock/sharpcompress/pull/188)
* [Make PpmdProperties lazy to avoid unnecessary allocations.](https://github.com/adamhathcock/sharpcompress/pull/185)
### Version 0.13.0
* Breaking change: Big refactor of Options on API.
* 7Zip supports Deflate
### Version 0.12.4
* Forward only zip issue fix https://github.com/adamhathcock/sharpcompress/issues/160
* Try to fix frameworks again by copying targets from JSON.NET
### Version 0.12.3
* 7Zip fixes https://github.com/adamhathcock/sharpcompress/issues/73
* Maybe all profiles will work with project.json now
### Version 0.12.2
* Support Profile 259 again
### Version 0.12.1
* Support Silverlight 5
### Version 0.12.0
* .NET Core RTM!
* Bug fix for Tar long paths
### Version 0.11.6
* Bug fix for global header in Tar
* Writers now have a leaveOpen `bool` overload. They won't close streams if not-requested to.
### Version 0.11.5
* Bug fix in Skip method
### Version 0.11.4
* SharpCompress is now endian neutral (matters for Mono platforms)
* Fix for Inflate (need to change implementation)
* Fixes for RAR detection
### Version 0.11.1
* Added Cancel on IReader
* Removed .NET 2.0 support and LinqBridge dependency
### Version 0.11
* Been over a year, contains mainly fixes from contributors!
* Possible breaking change: ArchiveEncoding is UTF8 by default now.
* TAR supports writing long names using longlink
* RAR Protect Header added
### Version 0.10.3
* Finally fixed Disposal issue when creating a new archive with the Archive API
### Version 0.10.2
* Fixed Rar Header reading for invalid extended time headers.
* Windows Store assembly is now strong named
* Known issues with Long Tar names being worked on
* Updated to VS2013
* Portable targets SL5 and Windows Phone 8 (up from SL4 and WP7)
### Version 0.10.1
* Fixed 7Zip extraction performance problem
### Version 0.10:
* Added support for RAR Decryption (thanks to https://github.com/hrasyid)
* Embedded some BouncyCastle crypto classes to allow RAR Decryption and Winzip AES Decryption in Portable and Windows Store DLLs
* Built in Release (I think)
## Notes
XZ implementation based on: https://github.com/sambott/XZ.NET by @sambott

View File

@@ -113,38 +113,26 @@ using (var archive = RarArchive.Open("Test.rar"))
}
```
### Extract solid Rar or 7Zip archives with manual progress reporting
### Extract solid Rar or 7Zip archives with progress reporting
`ExtractAllEntries` only works for solid archives (Rar) or 7Zip archives. For optimal performance with these archive types, use this method:
```C#
using (var archive = RarArchive.Open("archive.rar")) // Must be solid Rar or 7Zip
using SharpCompress.Common;
using SharpCompress.Readers;
var progress = new Progress<ProgressReport>(report =>
{
if (archive.IsSolid || archive.Type == ArchiveType.SevenZip)
Console.WriteLine($"Extracting {report.EntryPath}: {report.PercentComplete}%");
});
using (var archive = RarArchive.Open("archive.rar", new ReaderOptions { Progress = progress })) // Must be solid Rar or 7Zip
{
archive.WriteToDirectory(@"D:\output", new ExtractionOptions()
{
// Calculate total size for progress reporting
double totalSize = archive.Entries.Where(e => !e.IsDirectory).Sum(e => e.Size);
long completed = 0;
using (var reader = archive.ExtractAllEntries())
{
while (reader.MoveToNextEntry())
{
if (!reader.Entry.IsDirectory)
{
reader.WriteEntryToDirectory(@"D:\output", new ExtractionOptions()
{
ExtractFullPath = true,
Overwrite = true
});
completed += reader.Entry.Size;
double progress = completed / totalSize;
Console.WriteLine($"Progress: {progress:P}");
}
}
}
}
ExtractFullPath = true,
Overwrite = true
});
}
```

View File

@@ -79,11 +79,25 @@ public class EntryStream : Stream, IStreamStack
{
if (ss.BaseStream() is SharpCompress.Compressors.Deflate.DeflateStream deflateStream)
{
deflateStream.Flush(); //Deflate over reads. Knock it back
try
{
deflateStream.Flush(); //Deflate over reads. Knock it back
}
catch (NotSupportedException)
{
// Ignore: underlying stream does not support required operations for Flush
}
}
else if (ss.BaseStream() is SharpCompress.Compressors.LZMA.LzmaStream lzmaStream)
{
lzmaStream.Flush(); //Lzma over reads. Knock it back
try
{
lzmaStream.Flush(); //Lzma over reads. Knock it back
}
catch (NotSupportedException)
{
// Ignore: underlying stream does not support required operations for Flush
}
}
}
#if DEBUG_STREAMS
@@ -111,11 +125,25 @@ public class EntryStream : Stream, IStreamStack
{
if (ss.BaseStream() is SharpCompress.Compressors.Deflate.DeflateStream deflateStream)
{
await deflateStream.FlushAsync().ConfigureAwait(false);
try
{
await deflateStream.FlushAsync().ConfigureAwait(false);
}
catch (NotSupportedException)
{
// Ignore: underlying stream does not support required operations for Flush
}
}
else if (ss.BaseStream() is SharpCompress.Compressors.LZMA.LzmaStream lzmaStream)
{
await lzmaStream.FlushAsync().ConfigureAwait(false);
try
{
await lzmaStream.FlushAsync().ConfigureAwait(false);
}
catch (NotSupportedException)
{
// Ignore: underlying stream does not support required operations for Flush
}
}
}
#if DEBUG_STREAMS

View File

@@ -55,7 +55,7 @@ internal class SevenZipFilePart : FilePart
{
folderStream.Skip(skipSize);
}
return new ReadOnlySubStream(folderStream, Header.Size);
return new ReadOnlySubStream(folderStream, Header.Size, leaveOpen: false);
}
public CompressionType CompressionType

View File

@@ -15,10 +15,6 @@ internal enum ExtraDataType : ushort
UnicodePathExtraField = 0x7075,
Zip64ExtendedInformationExtraField = 0x0001,
UnixTimeExtraField = 0x5455,
// SOZip (Seek-Optimized ZIP) extra field
// Used to link a main file to its SOZip index file
SOZip = 0x564B,
}
internal class ExtraData
@@ -237,44 +233,6 @@ internal sealed class UnixTimeExtraField : ExtraData
}
}
/// <summary>
/// SOZip (Seek-Optimized ZIP) extra field that links a main file to its index file.
/// The extra field contains the offset within the ZIP file where the index entry's
/// local header is located.
/// </summary>
internal sealed class SOZipExtraField : ExtraData
{
public SOZipExtraField(ExtraDataType type, ushort length, byte[] dataBytes)
: base(type, length, dataBytes) { }
/// <summary>
/// Gets the offset to the SOZip index file's local entry header within the ZIP archive.
/// </summary>
internal ulong IndexOffset
{
get
{
if (DataBytes is null || DataBytes.Length < 8)
{
return 0;
}
return BinaryPrimitives.ReadUInt64LittleEndian(DataBytes);
}
}
/// <summary>
/// Creates a SOZip extra field with the specified index offset
/// </summary>
/// <param name="indexOffset">The offset to the index file's local entry header</param>
/// <returns>A new SOZipExtraField instance</returns>
public static SOZipExtraField Create(ulong indexOffset)
{
var data = new byte[8];
BinaryPrimitives.WriteUInt64LittleEndian(data, indexOffset);
return new SOZipExtraField(ExtraDataType.SOZip, 8, data);
}
}
internal static class LocalEntryHeaderExtraFactory
{
internal static ExtraData Create(ExtraDataType type, ushort length, byte[] extraData) =>
@@ -288,7 +246,6 @@ internal static class LocalEntryHeaderExtraFactory
ExtraDataType.Zip64ExtendedInformationExtraField =>
new Zip64ExtendedInformationExtraField(type, length, extraData),
ExtraDataType.UnixTimeExtraField => new UnixTimeExtraField(type, length, extraData),
ExtraDataType.SOZip => new SOZipExtraField(type, length, extraData),
_ => new ExtraData(type, length, extraData),
};
}

View File

@@ -1,150 +0,0 @@
using System;
using System.Collections.Generic;
using System.IO;
using SharpCompress.Compressors;
using SharpCompress.Compressors.Deflate;
namespace SharpCompress.Common.Zip.SOZip;
/// <summary>
/// A Deflate stream that inserts sync flush points at regular intervals
/// to enable random access (SOZip optimization).
/// </summary>
internal sealed class SOZipDeflateStream : Stream
{
private readonly DeflateStream _deflateStream;
private readonly Stream _baseStream;
private readonly uint _chunkSize;
private readonly List<ulong> _compressedOffsets = new();
private readonly long _baseOffset;
private long _uncompressedBytesWritten;
private long _nextSyncPoint;
private bool _disposed;
/// <summary>
/// Creates a new SOZip Deflate stream
/// </summary>
/// <param name="baseStream">The underlying stream to write to</param>
/// <param name="compressionLevel">The compression level</param>
/// <param name="chunkSize">The chunk size for sync flush points</param>
public SOZipDeflateStream(Stream baseStream, CompressionLevel compressionLevel, int chunkSize)
{
_baseStream = baseStream;
_chunkSize = (uint)chunkSize;
_baseOffset = baseStream.Position;
_nextSyncPoint = chunkSize;
// Record the first offset (start of compressed data)
_compressedOffsets.Add(0);
_deflateStream = new DeflateStream(baseStream, CompressionMode.Compress, compressionLevel);
}
/// <summary>
/// Gets the array of compressed offsets recorded during writing
/// </summary>
public ulong[] CompressedOffsets => _compressedOffsets.ToArray();
/// <summary>
/// Gets the total number of uncompressed bytes written
/// </summary>
public ulong UncompressedBytesWritten => (ulong)_uncompressedBytesWritten;
/// <summary>
/// Gets the total number of compressed bytes written
/// </summary>
public ulong CompressedBytesWritten => (ulong)(_baseStream.Position - _baseOffset);
/// <summary>
/// Gets the chunk size being used
/// </summary>
public uint ChunkSize => _chunkSize;
public override bool CanRead => false;
public override bool CanSeek => false;
public override bool CanWrite => !_disposed && _deflateStream.CanWrite;
public override long Length => throw new NotSupportedException();
public override long Position
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
public override void Flush() => _deflateStream.Flush();
public override int Read(byte[] buffer, int offset, int count) =>
throw new NotSupportedException();
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count)
{
if (_disposed)
{
throw new ObjectDisposedException(nameof(SOZipDeflateStream));
}
var remaining = count;
var currentOffset = offset;
while (remaining > 0)
{
// Calculate how many bytes until the next sync point
var bytesUntilSync = (int)(_nextSyncPoint - _uncompressedBytesWritten);
if (bytesUntilSync <= 0)
{
// We've reached a sync point - perform sync flush
PerformSyncFlush();
continue;
}
// Write up to the next sync point
var bytesToWrite = Math.Min(remaining, bytesUntilSync);
_deflateStream.Write(buffer, currentOffset, bytesToWrite);
_uncompressedBytesWritten += bytesToWrite;
currentOffset += bytesToWrite;
remaining -= bytesToWrite;
}
}
private void PerformSyncFlush()
{
// Flush with Z_SYNC_FLUSH to create an independent block
var originalFlushMode = _deflateStream.FlushMode;
_deflateStream.FlushMode = FlushType.Sync;
_deflateStream.Flush();
_deflateStream.FlushMode = originalFlushMode;
// Record the compressed offset for this sync point
var compressedOffset = (ulong)(_baseStream.Position - _baseOffset);
_compressedOffsets.Add(compressedOffset);
// Set the next sync point
_nextSyncPoint += _chunkSize;
}
protected override void Dispose(bool disposing)
{
if (_disposed)
{
return;
}
_disposed = true;
if (disposing)
{
_deflateStream.Dispose();
}
base.Dispose(disposing);
}
}

View File

@@ -1,367 +0,0 @@
using System;
using System.Buffers.Binary;
using System.IO;
namespace SharpCompress.Common.Zip.SOZip;
/// <summary>
/// Represents a SOZip (Seek-Optimized ZIP) index that enables random access
/// within DEFLATE-compressed files by storing offsets to sync flush points.
/// </summary>
/// <remarks>
/// SOZip index files (.sozip.idx) contain a header followed by offset entries
/// that point to the beginning of independently decompressable DEFLATE blocks.
/// </remarks>
[CLSCompliant(false)]
public sealed class SOZipIndex
{
/// <summary>
/// SOZip index file magic number: "SOZo" (0x534F5A6F)
/// </summary>
public const uint SOZIP_MAGIC = 0x6F5A4F53; // "SOZo" little-endian
/// <summary>
/// Current SOZip specification version
/// </summary>
public const byte SOZIP_VERSION = 1;
/// <summary>
/// Index file extension suffix
/// </summary>
public const string INDEX_EXTENSION = ".sozip.idx";
/// <summary>
/// Default chunk size in bytes (32KB)
/// </summary>
public const uint DEFAULT_CHUNK_SIZE = 32768;
/// <summary>
/// The version of the SOZip index format
/// </summary>
public byte Version { get; private set; }
/// <summary>
/// Size of each uncompressed chunk in bytes
/// </summary>
public uint ChunkSize { get; private set; }
/// <summary>
/// Total uncompressed size of the file
/// </summary>
public ulong UncompressedSize { get; private set; }
/// <summary>
/// Total compressed size of the file
/// </summary>
public ulong CompressedSize { get; private set; }
/// <summary>
/// Number of offset entries in the index
/// </summary>
public uint OffsetCount { get; private set; }
/// <summary>
/// Array of compressed offsets for each chunk
/// </summary>
public ulong[] CompressedOffsets { get; private set; } = Array.Empty<ulong>();
/// <summary>
/// Creates a new empty SOZip index
/// </summary>
public SOZipIndex() { }
/// <summary>
/// Creates a new SOZip index with specified parameters
/// </summary>
/// <param name="chunkSize">Size of each uncompressed chunk</param>
/// <param name="uncompressedSize">Total uncompressed size</param>
/// <param name="compressedSize">Total compressed size</param>
/// <param name="compressedOffsets">Array of compressed offsets</param>
public SOZipIndex(
uint chunkSize,
ulong uncompressedSize,
ulong compressedSize,
ulong[] compressedOffsets
)
{
Version = SOZIP_VERSION;
ChunkSize = chunkSize;
UncompressedSize = uncompressedSize;
CompressedSize = compressedSize;
OffsetCount = (uint)compressedOffsets.Length;
CompressedOffsets = compressedOffsets;
}
/// <summary>
/// Reads a SOZip index from a stream
/// </summary>
/// <param name="stream">The stream containing the index data</param>
/// <returns>A parsed SOZipIndex instance</returns>
/// <exception cref="InvalidDataException">If the stream doesn't contain valid SOZip index data</exception>
public static SOZipIndex Read(Stream stream)
{
var index = new SOZipIndex();
Span<byte> header = stackalloc byte[4];
// Read magic number
if (stream.Read(header) != 4)
{
throw new InvalidDataException("Invalid SOZip index: unable to read magic number");
}
var magic = BinaryPrimitives.ReadUInt32LittleEndian(header);
if (magic != SOZIP_MAGIC)
{
throw new InvalidDataException(
$"Invalid SOZip index: magic number mismatch (expected 0x{SOZIP_MAGIC:X8}, got 0x{magic:X8})"
);
}
// Read version
var versionByte = stream.ReadByte();
if (versionByte < 0)
{
throw new InvalidDataException("Invalid SOZip index: unable to read version");
}
index.Version = (byte)versionByte;
if (index.Version != SOZIP_VERSION)
{
throw new InvalidDataException(
$"Unsupported SOZip index version: {index.Version} (expected {SOZIP_VERSION})"
);
}
// Read reserved byte (padding)
stream.ReadByte();
// Read chunk size (2 bytes)
Span<byte> buf2 = stackalloc byte[2];
if (stream.Read(buf2) != 2)
{
throw new InvalidDataException("Invalid SOZip index: unable to read chunk size");
}
// Chunk size is stored as (actual_size / 1024) - 1
var chunkSizeEncoded = BinaryPrimitives.ReadUInt16LittleEndian(buf2);
index.ChunkSize = ((uint)chunkSizeEncoded + 1) * 1024;
// Read uncompressed size (8 bytes)
Span<byte> buf8 = stackalloc byte[8];
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException("Invalid SOZip index: unable to read uncompressed size");
}
index.UncompressedSize = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
// Read compressed size (8 bytes)
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException("Invalid SOZip index: unable to read compressed size");
}
index.CompressedSize = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
// Read offset count (4 bytes)
if (stream.Read(header) != 4)
{
throw new InvalidDataException("Invalid SOZip index: unable to read offset count");
}
index.OffsetCount = BinaryPrimitives.ReadUInt32LittleEndian(header);
// Read offsets
index.CompressedOffsets = new ulong[index.OffsetCount];
for (uint i = 0; i < index.OffsetCount; i++)
{
if (stream.Read(buf8) != 8)
{
throw new InvalidDataException($"Invalid SOZip index: unable to read offset {i}");
}
index.CompressedOffsets[i] = BinaryPrimitives.ReadUInt64LittleEndian(buf8);
}
return index;
}
/// <summary>
/// Reads a SOZip index from a byte array
/// </summary>
/// <param name="data">The byte array containing the index data</param>
/// <returns>A parsed SOZipIndex instance</returns>
public static SOZipIndex Read(byte[] data)
{
using var stream = new MemoryStream(data);
return Read(stream);
}
/// <summary>
/// Writes this SOZip index to a stream
/// </summary>
/// <param name="stream">The stream to write to</param>
public void Write(Stream stream)
{
Span<byte> buf8 = stackalloc byte[8];
// Write magic number
BinaryPrimitives.WriteUInt32LittleEndian(buf8, SOZIP_MAGIC);
stream.Write(buf8.Slice(0, 4));
// Write version
stream.WriteByte(SOZIP_VERSION);
// Write reserved byte (padding)
stream.WriteByte(0);
// Write chunk size (encoded as (size/1024)-1)
var chunkSizeEncoded = (ushort)((ChunkSize / 1024) - 1);
BinaryPrimitives.WriteUInt16LittleEndian(buf8, chunkSizeEncoded);
stream.Write(buf8.Slice(0, 2));
// Write uncompressed size
BinaryPrimitives.WriteUInt64LittleEndian(buf8, UncompressedSize);
stream.Write(buf8);
// Write compressed size
BinaryPrimitives.WriteUInt64LittleEndian(buf8, CompressedSize);
stream.Write(buf8);
// Write offset count
BinaryPrimitives.WriteUInt32LittleEndian(buf8, OffsetCount);
stream.Write(buf8.Slice(0, 4));
// Write offsets
foreach (var offset in CompressedOffsets)
{
BinaryPrimitives.WriteUInt64LittleEndian(buf8, offset);
stream.Write(buf8);
}
}
/// <summary>
/// Converts this SOZip index to a byte array
/// </summary>
/// <returns>Byte array containing the serialized index</returns>
public byte[] ToByteArray()
{
using var stream = new MemoryStream();
Write(stream);
return stream.ToArray();
}
/// <summary>
/// Gets the index of the chunk that contains the specified uncompressed offset
/// </summary>
/// <param name="uncompressedOffset">The uncompressed byte offset</param>
/// <returns>The chunk index</returns>
public int GetChunkIndex(long uncompressedOffset)
{
if (uncompressedOffset < 0 || (ulong)uncompressedOffset >= UncompressedSize)
{
throw new ArgumentOutOfRangeException(
nameof(uncompressedOffset),
"Offset is out of range"
);
}
return (int)((ulong)uncompressedOffset / ChunkSize);
}
/// <summary>
/// Gets the compressed offset for the specified chunk index
/// </summary>
/// <param name="chunkIndex">The chunk index</param>
/// <returns>The compressed byte offset for the start of the chunk</returns>
public ulong GetCompressedOffset(int chunkIndex)
{
if (chunkIndex < 0 || chunkIndex >= CompressedOffsets.Length)
{
throw new ArgumentOutOfRangeException(
nameof(chunkIndex),
"Chunk index is out of range"
);
}
return CompressedOffsets[chunkIndex];
}
/// <summary>
/// Gets the uncompressed offset for the start of the specified chunk
/// </summary>
/// <param name="chunkIndex">The chunk index</param>
/// <returns>The uncompressed byte offset for the start of the chunk</returns>
public ulong GetUncompressedOffset(int chunkIndex)
{
if (chunkIndex < 0 || chunkIndex >= CompressedOffsets.Length)
{
throw new ArgumentOutOfRangeException(
nameof(chunkIndex),
"Chunk index is out of range"
);
}
return (ulong)chunkIndex * ChunkSize;
}
/// <summary>
/// Gets the name of the SOZip index file for a given entry name
/// </summary>
/// <param name="entryName">The main entry name</param>
/// <returns>The index file name (hidden with .sozip.idx extension)</returns>
public static string GetIndexFileName(string entryName)
{
var directory = Path.GetDirectoryName(entryName);
var fileName = Path.GetFileName(entryName);
// The index file is hidden (prefixed with .)
var indexFileName = $".{fileName}{INDEX_EXTENSION}";
if (string.IsNullOrEmpty(directory))
{
return indexFileName;
}
return Path.Combine(directory, indexFileName).Replace('\\', '/');
}
/// <summary>
/// Checks if a file name is a SOZip index file
/// </summary>
/// <param name="fileName">The file name to check</param>
/// <returns>True if the file is a SOZip index file</returns>
public static bool IsIndexFile(string fileName)
{
if (string.IsNullOrEmpty(fileName))
{
return false;
}
var name = Path.GetFileName(fileName);
return name.StartsWith(".", StringComparison.Ordinal)
&& name.EndsWith(INDEX_EXTENSION, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Gets the main file name from a SOZip index file name
/// </summary>
/// <param name="indexFileName">The index file name</param>
/// <returns>The main file name, or null if not a valid index file</returns>
public static string? GetMainFileName(string indexFileName)
{
if (!IsIndexFile(indexFileName))
{
return null;
}
var directory = Path.GetDirectoryName(indexFileName);
var name = Path.GetFileName(indexFileName);
// Remove leading '.' and trailing '.sozip.idx'
var mainName = name.Substring(1, name.Length - 1 - INDEX_EXTENSION.Length);
if (string.IsNullOrEmpty(directory))
{
return mainName;
}
return Path.Combine(directory, mainName).Replace('\\', '/');
}
}

View File

@@ -2,7 +2,6 @@ using System;
using System.Collections.Generic;
using System.Linq;
using SharpCompress.Common.Zip.Headers;
using SharpCompress.Common.Zip.SOZip;
namespace SharpCompress.Common.Zip;
@@ -12,7 +11,7 @@ public class ZipEntry : Entry
internal ZipEntry(ZipFilePart? filePart)
{
if (filePart is null)
if (filePart == null)
{
return;
}
@@ -89,24 +88,4 @@ public class ZipEntry : Entry
public override int? Attrib => (int?)_filePart?.Header.ExternalFileAttributes;
public string? Comment => _filePart?.Header.Comment;
/// <summary>
/// Gets a value indicating whether this entry has SOZip (Seek-Optimized ZIP) support.
/// A SOZip entry has an associated index file that enables random access within
/// the compressed data.
/// </summary>
public bool IsSozip => _filePart?.Header.Extra.Any(e => e.Type == ExtraDataType.SOZip) ?? false;
/// <summary>
/// Gets a value indicating whether this entry is a SOZip index file.
/// Index files are hidden files with a .sozip.idx extension that contain
/// offsets into the main compressed file.
/// </summary>
public bool IsSozipIndexFile => Key is not null && SOZipIndex.IsIndexFile(Key);
/// <summary>
/// Gets the SOZip extra field data, if present.
/// </summary>
internal SOZipExtraField? SOZipExtra =>
_filePart?.Header.Extra.OfType<SOZipExtraField>().FirstOrDefault();
}

View File

@@ -30,6 +30,7 @@ public sealed class BZip2Stream : Stream, IStreamStack
private readonly Stream stream;
private bool isDisposed;
private readonly bool leaveOpen;
/// <summary>
/// Create a BZip2Stream
@@ -37,19 +38,30 @@ public sealed class BZip2Stream : Stream, IStreamStack
/// <param name="stream">The stream to read from</param>
/// <param name="compressionMode">Compression Mode</param>
/// <param name="decompressConcatenated">Decompress Concatenated</param>
public BZip2Stream(Stream stream, CompressionMode compressionMode, bool decompressConcatenated)
/// <param name="leaveOpen">Leave the stream open after disposing</param>
public BZip2Stream(
Stream stream,
CompressionMode compressionMode,
bool decompressConcatenated,
bool leaveOpen = false
)
{
#if DEBUG_STREAMS
this.DebugConstruct(typeof(BZip2Stream));
#endif
this.leaveOpen = leaveOpen;
Mode = compressionMode;
if (Mode == CompressionMode.Compress)
{
this.stream = new CBZip2OutputStream(stream);
this.stream = new CBZip2OutputStream(stream, 9, leaveOpen);
}
else
{
this.stream = new CBZip2InputStream(stream, decompressConcatenated);
this.stream = new CBZip2InputStream(
stream,
decompressConcatenated,
leaveOpen: leaveOpen
);
}
}

View File

@@ -168,6 +168,7 @@ internal class CBZip2InputStream : Stream, IStreamStack
private int computedBlockCRC,
computedCombinedCRC;
private readonly bool decompressConcatenated;
private readonly bool leaveOpen;
private int i2,
count,
@@ -181,9 +182,10 @@ internal class CBZip2InputStream : Stream, IStreamStack
private char z;
private bool isDisposed;
public CBZip2InputStream(Stream zStream, bool decompressConcatenated)
public CBZip2InputStream(Stream zStream, bool decompressConcatenated, bool leaveOpen = false)
{
this.decompressConcatenated = decompressConcatenated;
this.leaveOpen = leaveOpen;
ll8 = null;
tt = null;
BsSetStream(zStream);
@@ -207,7 +209,10 @@ internal class CBZip2InputStream : Stream, IStreamStack
this.DebugDispose(typeof(CBZip2InputStream));
#endif
base.Dispose(disposing);
bsStream?.Dispose();
if (!leaveOpen)
{
bsStream?.Dispose();
}
}
internal static int[][] InitIntArray(int n1, int n2)
@@ -398,7 +403,10 @@ internal class CBZip2InputStream : Stream, IStreamStack
private void BsFinishedWithStream()
{
bsStream?.Dispose();
if (!leaveOpen)
{
bsStream?.Dispose();
}
bsStream = null;
}

View File

@@ -341,12 +341,14 @@ internal sealed class CBZip2OutputStream : Stream, IStreamStack
private int currentChar = -1;
private int runLength;
private readonly bool leaveOpen;
public CBZip2OutputStream(Stream inStream)
: this(inStream, 9) { }
public CBZip2OutputStream(Stream inStream, bool leaveOpen = false)
: this(inStream, 9, leaveOpen) { }
public CBZip2OutputStream(Stream inStream, int inBlockSize)
public CBZip2OutputStream(Stream inStream, int inBlockSize, bool leaveOpen = false)
{
this.leaveOpen = leaveOpen;
block = null;
quadrant = null;
zptr = null;
@@ -481,7 +483,10 @@ internal sealed class CBZip2OutputStream : Stream, IStreamStack
this.DebugDispose(typeof(CBZip2OutputStream));
#endif
Dispose();
bsStream?.Dispose();
if (!leaveOpen)
{
bsStream?.Dispose();
}
bsStream = null;
}
}

View File

@@ -153,7 +153,7 @@ internal class OutWindow : IDisposable
_pendingLen = rem;
}
public async Task CopyPendingAsync(CancellationToken cancellationToken = default)
public async ValueTask CopyPendingAsync(CancellationToken cancellationToken = default)
{
if (_pendingLen < 1)
{
@@ -206,7 +206,7 @@ internal class OutWindow : IDisposable
_pendingDist = distance;
}
public async Task CopyBlockAsync(
public async ValueTask CopyBlockAsync(
int distance,
int len,
CancellationToken cancellationToken = default
@@ -253,7 +253,7 @@ internal class OutWindow : IDisposable
}
}
public async Task PutByteAsync(byte b, CancellationToken cancellationToken = default)
public async ValueTask PutByteAsync(byte b, CancellationToken cancellationToken = default)
{
_buffer[_pos++] = b;
_total++;
@@ -369,6 +369,28 @@ internal class OutWindow : IDisposable
return size;
}
public int Read(Memory<byte> buffer, int offset, int count)
{
if (_streamPos >= _pos)
{
return 0;
}
var size = _pos - _streamPos;
if (size > count)
{
size = count;
}
_buffer.AsMemory(_streamPos, size).CopyTo(buffer.Slice(offset, size));
_streamPos += size;
if (_streamPos >= _windowSize)
{
_pos = 0;
_streamPos = 0;
}
return size;
}
public int ReadByte()
{
if (_streamPos >= _pos)

View File

@@ -45,10 +45,14 @@ public sealed class LZipStream : Stream, IStreamStack
private bool _finished;
private long _writeCount;
private readonly Stream? _originalStream;
private readonly bool _leaveOpen;
public LZipStream(Stream stream, CompressionMode mode)
public LZipStream(Stream stream, CompressionMode mode, bool leaveOpen = false)
{
Mode = mode;
_originalStream = stream;
_leaveOpen = leaveOpen;
if (mode == CompressionMode.Decompress)
{
@@ -58,7 +62,7 @@ public sealed class LZipStream : Stream, IStreamStack
throw new InvalidFormatException("Not an LZip stream");
}
var properties = GetProperties(dSize);
_stream = new LzmaStream(properties, stream);
_stream = new LzmaStream(properties, stream, leaveOpen: leaveOpen);
}
else
{
@@ -125,6 +129,10 @@ public sealed class LZipStream : Stream, IStreamStack
{
Finish();
_stream.Dispose();
if (Mode == CompressionMode.Compress && !_leaveOpen)
{
_originalStream?.Dispose();
}
}
}

View File

@@ -3,6 +3,7 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Threading.Tasks;
using SharpCompress.Compressors.LZMA.LZ;
using SharpCompress.Compressors.LZMA.RangeCoder;
@@ -475,7 +476,7 @@ public class Decoder : ICoder, ISetDecoderProperties // ,System.IO.Stream
return false;
}
internal async System.Threading.Tasks.Task<bool> CodeAsync(
internal async ValueTask<bool> CodeAsync(
int dictionarySize,
OutWindow outWindow,
RangeCoder.Decoder rangeDecoder,

View File

@@ -35,6 +35,7 @@ public class LzmaStream : Stream, IStreamStack
private readonly Stream _inputStream;
private readonly long _inputSize;
private readonly long _outputSize;
private readonly bool _leaveOpen;
private readonly int _dictionarySize;
private readonly OutWindow _outWindow = new();
@@ -56,14 +57,28 @@ public class LzmaStream : Stream, IStreamStack
private readonly Encoder _encoder;
private bool _isDisposed;
public LzmaStream(byte[] properties, Stream inputStream)
: this(properties, inputStream, -1, -1, null, properties.Length < 5) { }
public LzmaStream(byte[] properties, Stream inputStream, bool leaveOpen = false)
: this(properties, inputStream, -1, -1, null, properties.Length < 5, leaveOpen) { }
public LzmaStream(byte[] properties, Stream inputStream, long inputSize)
: this(properties, inputStream, inputSize, -1, null, properties.Length < 5) { }
public LzmaStream(byte[] properties, Stream inputStream, long inputSize, bool leaveOpen = false)
: this(properties, inputStream, inputSize, -1, null, properties.Length < 5, leaveOpen) { }
public LzmaStream(byte[] properties, Stream inputStream, long inputSize, long outputSize)
: this(properties, inputStream, inputSize, outputSize, null, properties.Length < 5) { }
public LzmaStream(
byte[] properties,
Stream inputStream,
long inputSize,
long outputSize,
bool leaveOpen = false
)
: this(
properties,
inputStream,
inputSize,
outputSize,
null,
properties.Length < 5,
leaveOpen
) { }
public LzmaStream(
byte[] properties,
@@ -71,13 +86,15 @@ public class LzmaStream : Stream, IStreamStack
long inputSize,
long outputSize,
Stream presetDictionary,
bool isLzma2
bool isLzma2,
bool leaveOpen = false
)
{
_inputStream = inputStream;
_inputSize = inputSize;
_outputSize = outputSize;
_isLzma2 = isLzma2;
_leaveOpen = leaveOpen;
#if DEBUG_STREAMS
this.DebugConstruct(typeof(LzmaStream));
@@ -179,7 +196,10 @@ public class LzmaStream : Stream, IStreamStack
{
_position = _encoder.Code(null, true);
}
_inputStream?.Dispose();
if (!_leaveOpen)
{
_inputStream?.Dispose();
}
_outWindow.Dispose();
}
base.Dispose(disposing);
@@ -425,7 +445,7 @@ public class LzmaStream : Stream, IStreamStack
}
}
private async Task DecodeChunkHeaderAsync(CancellationToken cancellationToken = default)
private async ValueTask DecodeChunkHeaderAsync(CancellationToken cancellationToken = default)
{
var controlBuffer = new byte[1];
await _inputStream
@@ -632,6 +652,119 @@ public class LzmaStream : Stream, IStreamStack
return total;
}
#if !NETFRAMEWORK && !NETSTANDARD2_0
public override async ValueTask<int> ReadAsync(
Memory<byte> buffer,
CancellationToken cancellationToken = default
)
{
if (_endReached)
{
return 0;
}
var total = 0;
var offset = 0;
var count = buffer.Length;
while (total < count)
{
cancellationToken.ThrowIfCancellationRequested();
if (_availableBytes == 0)
{
if (_isLzma2)
{
await DecodeChunkHeaderAsync(cancellationToken).ConfigureAwait(false);
}
else
{
_endReached = true;
}
if (_endReached)
{
break;
}
}
var toProcess = count - total;
if (toProcess > _availableBytes)
{
toProcess = (int)_availableBytes;
}
_outWindow.SetLimit(toProcess);
if (_uncompressedChunk)
{
_inputPosition += await _outWindow
.CopyStreamAsync(_inputStream, toProcess, cancellationToken)
.ConfigureAwait(false);
}
else if (
await _decoder
.CodeAsync(_dictionarySize, _outWindow, _rangeDecoder, cancellationToken)
.ConfigureAwait(false)
&& _outputSize < 0
)
{
_availableBytes = _outWindow.AvailableBytes;
}
var read = _outWindow.Read(buffer, offset, toProcess);
total += read;
offset += read;
_position += read;
_availableBytes -= read;
if (_availableBytes == 0 && !_uncompressedChunk)
{
if (
!_rangeDecoder.IsFinished
|| (_rangeDecoderLimit >= 0 && _rangeDecoder._total != _rangeDecoderLimit)
)
{
_outWindow.SetLimit(toProcess + 1);
if (
!await _decoder
.CodeAsync(
_dictionarySize,
_outWindow,
_rangeDecoder,
cancellationToken
)
.ConfigureAwait(false)
)
{
_rangeDecoder.ReleaseStream();
throw new DataErrorException();
}
}
_rangeDecoder.ReleaseStream();
_inputPosition += _rangeDecoder._total;
if (_outWindow.HasPending)
{
throw new DataErrorException();
}
}
}
if (_endReached)
{
if (_inputSize >= 0 && _inputPosition != _inputSize)
{
throw new DataErrorException();
}
if (_outputSize >= 0 && _position != _outputSize)
{
throw new DataErrorException();
}
}
return total;
}
#endif
public override Task WriteAsync(
byte[] buffer,
int offset,

View File

@@ -1,5 +1,7 @@
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace SharpCompress.IO;
@@ -68,6 +70,23 @@ internal class BufferedSubStream : SharpCompressStream, IStreamStack
BytesLeftToRead -= _cacheLength;
}
private async ValueTask RefillCacheAsync(CancellationToken cancellationToken)
{
var count = (int)Math.Min(BytesLeftToRead, _cache.Length);
_cacheOffset = 0;
if (count == 0)
{
_cacheLength = 0;
return;
}
Stream.Position = origin;
_cacheLength = await Stream
.ReadAsync(_cache, 0, count, cancellationToken)
.ConfigureAwait(false);
origin += _cacheLength;
BytesLeftToRead -= _cacheLength;
}
public override int Read(byte[] buffer, int offset, int count)
{
if (count > Length)
@@ -104,6 +123,61 @@ internal class BufferedSubStream : SharpCompressStream, IStreamStack
return _cache[_cacheOffset++];
}
public override async Task<int> ReadAsync(
byte[] buffer,
int offset,
int count,
CancellationToken cancellationToken
)
{
if (count > Length)
{
count = (int)Length;
}
if (count > 0)
{
if (_cacheOffset == _cacheLength)
{
await RefillCacheAsync(cancellationToken).ConfigureAwait(false);
}
count = Math.Min(count, _cacheLength - _cacheOffset);
Buffer.BlockCopy(_cache, _cacheOffset, buffer, offset, count);
_cacheOffset += count;
}
return count;
}
#if !NETFRAMEWORK && !NETSTANDARD2_0
public override async ValueTask<int> ReadAsync(
Memory<byte> buffer,
CancellationToken cancellationToken = default
)
{
var count = buffer.Length;
if (count > Length)
{
count = (int)Length;
}
if (count > 0)
{
if (_cacheOffset == _cacheLength)
{
await RefillCacheAsync(cancellationToken).ConfigureAwait(false);
}
count = Math.Min(count, _cacheLength - _cacheOffset);
_cache.AsSpan(_cacheOffset, count).CopyTo(buffer.Span);
_cacheOffset += count;
}
return count;
}
#endif
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();

View File

@@ -16,11 +16,11 @@ internal class ReadOnlySubStream : SharpCompressStream, IStreamStack
private long _position;
public ReadOnlySubStream(Stream stream, long bytesToRead)
: this(stream, null, bytesToRead) { }
public ReadOnlySubStream(Stream stream, long bytesToRead, bool leaveOpen = true)
: this(stream, null, bytesToRead, leaveOpen) { }
public ReadOnlySubStream(Stream stream, long? origin, long bytesToRead)
: base(stream, leaveOpen: true, throwOnDispose: false)
public ReadOnlySubStream(Stream stream, long? origin, long bytesToRead, bool leaveOpen = true)
: base(stream, leaveOpen, throwOnDispose: false)
{
if (origin != null && stream.Position != origin.Value)
{

View File

@@ -138,8 +138,6 @@ public class SharpCompressStream : Stream, IStreamStack
#endif
}
internal bool IsRecording { get; private set; }
protected override void Dispose(bool disposing)
{
#if DEBUG_STREAMS

View File

@@ -71,48 +71,16 @@ internal static class Utility
return;
}
using var buffer = MemoryPool<byte>.Shared.Rent(TEMP_BUFFER_SIZE);
while (advanceAmount > 0)
{
var toRead = (int)Math.Min(buffer.Memory.Length, advanceAmount);
var read = source.Read(buffer.Memory.Slice(0, toRead).Span);
if (read <= 0)
{
break;
}
advanceAmount -= read;
}
using var readOnlySubStream = new IO.ReadOnlySubStream(source, advanceAmount);
readOnlySubStream.CopyTo(Stream.Null);
}
public static void Skip(this Stream source)
{
using var buffer = MemoryPool<byte>.Shared.Rent(TEMP_BUFFER_SIZE);
while (source.Read(buffer.Memory.Span) > 0) { }
}
public static void Skip(this Stream source) => source.CopyTo(Stream.Null);
public static async Task SkipAsync(
this Stream source,
CancellationToken cancellationToken = default
)
public static Task SkipAsync(this Stream source, CancellationToken cancellationToken = default)
{
var array = ArrayPool<byte>.Shared.Rent(TEMP_BUFFER_SIZE);
try
{
while (true)
{
var read = await source
.ReadAsync(array, 0, array.Length, cancellationToken)
.ConfigureAwait(false);
if (read <= 0)
{
break;
}
}
}
finally
{
ArrayPool<byte>.Shared.Return(array);
}
cancellationToken.ThrowIfCancellationRequested();
return source.CopyToAsync(Stream.Null);
}
public static DateTime DosDateToDateTime(ushort iDate, ushort iTime)

View File

@@ -34,7 +34,6 @@ internal class ZipCentralDirectoryEntry
internal ulong Decompressed { get; set; }
internal ushort Zip64HeaderOffset { get; set; }
internal ulong HeaderOffset { get; }
internal string FileName => fileName;
internal uint Write(Stream outputStream)
{

View File

@@ -8,7 +8,6 @@ using System.Threading.Tasks;
using SharpCompress.Common;
using SharpCompress.Common.Zip;
using SharpCompress.Common.Zip.Headers;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Compressors;
using SharpCompress.Compressors.BZip2;
using SharpCompress.Compressors.Deflate;
@@ -28,19 +27,12 @@ public class ZipWriter : AbstractWriter
private long streamPosition;
private PpmdProperties? ppmdProps;
private readonly bool isZip64;
private readonly bool enableSOZip;
private readonly int sozipChunkSize;
private readonly long sozipMinFileSize;
public ZipWriter(Stream destination, ZipWriterOptions zipWriterOptions)
: base(ArchiveType.Zip, zipWriterOptions)
{
zipComment = zipWriterOptions.ArchiveComment ?? string.Empty;
isZip64 = zipWriterOptions.UseZip64;
enableSOZip = zipWriterOptions.EnableSOZip;
sozipChunkSize = zipWriterOptions.SOZipChunkSize;
sozipMinFileSize = zipWriterOptions.SOZipMinFileSize;
if (destination.CanSeek)
{
streamPosition = destination.Position;
@@ -125,21 +117,12 @@ public class ZipWriter : AbstractWriter
var headersize = (uint)WriteHeader(entryPath, options, entry, useZip64);
streamPosition += headersize;
// Determine if SOZip should be used for this entry
var useSozip =
(options.EnableSOZip ?? enableSOZip)
&& compression == ZipCompressionMethod.Deflate
&& OutputStream.CanSeek;
return new ZipWritingStream(
this,
OutputStream.NotNull(),
entry,
compression,
options.CompressionLevel ?? compressionLevel,
useSozip,
useSozip ? sozipChunkSize : 0
options.CompressionLevel ?? compressionLevel
);
}
@@ -321,64 +304,6 @@ public class ZipWriter : AbstractWriter
OutputStream.Write(intBuf);
}
private void WriteSozipIndexFile(
ZipCentralDirectoryEntry dataEntry,
SOZipDeflateStream sozipStream
)
{
var indexFileName = SOZipIndex.GetIndexFileName(dataEntry.FileName);
// Create the SOZip index
var index = new SOZipIndex(
chunkSize: sozipStream.ChunkSize,
uncompressedSize: sozipStream.UncompressedBytesWritten,
compressedSize: sozipStream.CompressedBytesWritten,
compressedOffsets: sozipStream.CompressedOffsets
);
var indexBytes = index.ToByteArray();
// Calculate CRC for index data
var crc = new CRC32();
crc.SlurpBlock(indexBytes, 0, indexBytes.Length);
var indexCrc = (uint)crc.Crc32Result;
// Write the index file as a stored (uncompressed) entry
var indexEntry = new ZipCentralDirectoryEntry(
ZipCompressionMethod.None,
indexFileName,
(ulong)streamPosition,
WriterOptions.ArchiveEncoding
)
{
ModificationTime = DateTime.Now,
};
// Write the local file header for index
var indexOptions = new ZipWriterEntryOptions { CompressionType = CompressionType.None };
var headerSize = (uint)WriteHeader(indexFileName, indexOptions, indexEntry, isZip64);
streamPosition += headerSize;
// Write the index data directly
OutputStream.Write(indexBytes, 0, indexBytes.Length);
// Finalize the index entry
indexEntry.Crc = indexCrc;
indexEntry.Compressed = (ulong)indexBytes.Length;
indexEntry.Decompressed = (ulong)indexBytes.Length;
if (OutputStream.CanSeek)
{
// Update the header with sizes and CRC
OutputStream.Position = (long)(indexEntry.HeaderOffset + 14);
WriteFooter(indexCrc, (uint)indexBytes.Length, (uint)indexBytes.Length);
OutputStream.Position = streamPosition + indexBytes.Length;
}
streamPosition += indexBytes.Length;
entries.Add(indexEntry);
}
private void WriteEndRecord(ulong size)
{
var zip64EndOfCentralDirectoryNeeded =
@@ -460,10 +385,7 @@ public class ZipWriter : AbstractWriter
private readonly ZipWriter writer;
private readonly ZipCompressionMethod zipCompressionMethod;
private readonly int compressionLevel;
private readonly bool useSozip;
private readonly int sozipChunkSize;
private SharpCompressStream? counting;
private SOZipDeflateStream? sozipStream;
private ulong decompressed;
// Flag to prevent throwing exceptions on Dispose
@@ -475,9 +397,7 @@ public class ZipWriter : AbstractWriter
Stream originalStream,
ZipCentralDirectoryEntry entry,
ZipCompressionMethod zipCompressionMethod,
int compressionLevel,
bool useSozip = false,
int sozipChunkSize = 0
int compressionLevel
)
{
this.writer = writer;
@@ -486,8 +406,6 @@ public class ZipWriter : AbstractWriter
this.entry = entry;
this.zipCompressionMethod = zipCompressionMethod;
this.compressionLevel = compressionLevel;
this.useSozip = useSozip;
this.sozipChunkSize = sozipChunkSize;
writeStream = GetWriteStream(originalStream);
}
@@ -517,15 +435,6 @@ public class ZipWriter : AbstractWriter
}
case ZipCompressionMethod.Deflate:
{
if (useSozip && sozipChunkSize > 0)
{
sozipStream = new SOZipDeflateStream(
counting,
(CompressionLevel)compressionLevel,
sozipChunkSize
);
return sozipStream;
}
return new DeflateStream(
counting,
CompressionMode.Compress,
@@ -672,18 +581,7 @@ public class ZipWriter : AbstractWriter
writer.WriteFooter(entry.Crc, compressedvalue, decompressedvalue);
writer.streamPosition += (long)entry.Compressed + 16;
}
writer.entries.Add(entry);
// Write SOZip index file if SOZip was used and file meets minimum size
if (
useSozip
&& sozipStream is not null
&& entry.Decompressed >= (ulong)writer.sozipMinFileSize
)
{
writer.WriteSozipIndexFile(entry, sozipStream);
}
}
}

View File

@@ -49,11 +49,4 @@ public class ZipWriterEntryOptions
/// This option is not supported with non-seekable streams.
/// </summary>
public bool? EnableZip64 { get; set; }
/// <summary>
/// Enable or disable SOZip (Seek-Optimized ZIP) for this entry.
/// When null, uses the archive's default setting.
/// SOZip is only applicable to Deflate-compressed files on seekable streams.
/// </summary>
public bool? EnableSOZip { get; set; }
}

View File

@@ -1,6 +1,5 @@
using System;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Compressors.Deflate;
using D = SharpCompress.Compressors.Deflate;
@@ -25,9 +24,6 @@ public class ZipWriterOptions : WriterOptions
{
UseZip64 = writerOptions.UseZip64;
ArchiveComment = writerOptions.ArchiveComment;
EnableSOZip = writerOptions.EnableSOZip;
SOZipChunkSize = writerOptions.SOZipChunkSize;
SOZipMinFileSize = writerOptions.SOZipMinFileSize;
}
}
@@ -84,27 +80,4 @@ public class ZipWriterOptions : WriterOptions
/// are less than 4GiB in length.
/// </summary>
public bool UseZip64 { get; set; }
/// <summary>
/// Enables SOZip (Seek-Optimized ZIP) for Deflate-compressed files.
/// When enabled, files that meet the minimum size requirement will have
/// an accompanying index file that allows random access within the
/// compressed data. Requires a seekable output stream.
/// </summary>
public bool EnableSOZip { get; set; }
/// <summary>
/// The chunk size for SOZip index creation in bytes.
/// Must be a multiple of 1024 bytes. Default is 32KB (32768 bytes).
/// Smaller chunks allow for finer-grained random access but result
/// in larger index files and slightly less efficient compression.
/// </summary>
public int SOZipChunkSize { get; set; } = (int)SOZipIndex.DEFAULT_CHUNK_SIZE;
/// <summary>
/// Minimum file size (uncompressed) in bytes for SOZip optimization.
/// Files smaller than this size will not have SOZip index files created.
/// Default is 1MB (1048576 bytes).
/// </summary>
public long SOZipMinFileSize { get; set; } = 1048576;
}

View File

@@ -0,0 +1,139 @@
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using SharpCompress.Archives;
using SharpCompress.Common;
using Xunit;
namespace SharpCompress.Test.SevenZip;
#if !NETFRAMEWORK
public class SevenZipArchiveAsyncTests : ArchiveTests
{
[Fact]
public async Task SevenZipArchive_LZMA_AsyncStreamExtraction()
{
var testArchive = Path.Combine(TEST_ARCHIVES_PATH, "7Zip.LZMA.7z");
using var stream = File.OpenRead(testArchive);
using var archive = ArchiveFactory.Open(stream);
foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory))
{
var targetPath = Path.Combine(SCRATCH_FILES_PATH, entry.Key!);
var targetDir = Path.GetDirectoryName(targetPath);
if (!string.IsNullOrEmpty(targetDir) && !Directory.Exists(targetDir))
{
Directory.CreateDirectory(targetDir);
}
using var sourceStream = await entry.OpenEntryStreamAsync(CancellationToken.None);
await using var targetStream = File.Create(targetPath);
await sourceStream.CopyToAsync(targetStream, CancellationToken.None);
}
VerifyFiles();
}
[Fact]
public async Task SevenZipArchive_LZMA2_AsyncStreamExtraction()
{
var testArchive = Path.Combine(TEST_ARCHIVES_PATH, "7Zip.LZMA2.7z");
using var stream = File.OpenRead(testArchive);
using var archive = ArchiveFactory.Open(stream);
foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory))
{
var targetPath = Path.Combine(SCRATCH_FILES_PATH, entry.Key!);
var targetDir = Path.GetDirectoryName(targetPath);
if (!string.IsNullOrEmpty(targetDir) && !Directory.Exists(targetDir))
{
Directory.CreateDirectory(targetDir);
}
using var sourceStream = await entry.OpenEntryStreamAsync(CancellationToken.None);
await using var targetStream = File.Create(targetPath);
await sourceStream.CopyToAsync(targetStream, CancellationToken.None);
}
VerifyFiles();
}
[Fact]
public async Task SevenZipArchive_Solid_AsyncStreamExtraction()
{
var testArchive = Path.Combine(TEST_ARCHIVES_PATH, "7Zip.solid.7z");
using var stream = File.OpenRead(testArchive);
using var archive = ArchiveFactory.Open(stream);
foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory))
{
var targetPath = Path.Combine(SCRATCH_FILES_PATH, entry.Key!);
var targetDir = Path.GetDirectoryName(targetPath);
if (!string.IsNullOrEmpty(targetDir) && !Directory.Exists(targetDir))
{
Directory.CreateDirectory(targetDir);
}
using var sourceStream = await entry.OpenEntryStreamAsync(CancellationToken.None);
await using var targetStream = File.Create(targetPath);
await sourceStream.CopyToAsync(targetStream, CancellationToken.None);
}
VerifyFiles();
}
[Fact]
public async Task SevenZipArchive_BZip2_AsyncStreamExtraction()
{
var testArchive = Path.Combine(TEST_ARCHIVES_PATH, "7Zip.BZip2.7z");
using var stream = File.OpenRead(testArchive);
using var archive = ArchiveFactory.Open(stream);
foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory))
{
var targetPath = Path.Combine(SCRATCH_FILES_PATH, entry.Key!);
var targetDir = Path.GetDirectoryName(targetPath);
if (!string.IsNullOrEmpty(targetDir) && !Directory.Exists(targetDir))
{
Directory.CreateDirectory(targetDir);
}
using var sourceStream = await entry.OpenEntryStreamAsync(CancellationToken.None);
await using var targetStream = File.Create(targetPath);
await sourceStream.CopyToAsync(targetStream, CancellationToken.None);
}
VerifyFiles();
}
[Fact]
public async Task SevenZipArchive_PPMd_AsyncStreamExtraction()
{
var testArchive = Path.Combine(TEST_ARCHIVES_PATH, "7Zip.PPMd.7z");
using var stream = File.OpenRead(testArchive);
using var archive = ArchiveFactory.Open(stream);
foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory))
{
var targetPath = Path.Combine(SCRATCH_FILES_PATH, entry.Key!);
var targetDir = Path.GetDirectoryName(targetPath);
if (!string.IsNullOrEmpty(targetDir) && !Directory.Exists(targetDir))
{
Directory.CreateDirectory(targetDir);
}
using var sourceStream = await entry.OpenEntryStreamAsync(CancellationToken.None);
await using var targetStream = File.Create(targetPath);
await sourceStream.CopyToAsync(targetStream, CancellationToken.None);
}
VerifyFiles();
}
}
#endif

View File

@@ -9,6 +9,9 @@
<PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Debug|net10.0|AnyCPU'">
<DefineConstants>$(DefineConstants);DEBUG_STREAMS</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition=" '$(TargetFramework)' == 'net48' ">
<DefineConstants>$(DefineConstants);LEGACY_DOTNET</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Windows)))">
<DefineConstants>$(DefineConstants);WINDOWS</DefineConstants>
</PropertyGroup>
@@ -25,7 +28,7 @@
<PackageReference Include="xunit" />
<PackageReference Include="Microsoft.NETFramework.ReferenceAssemblies" PrivateAssets="All" />
</ItemGroup>
<ItemGroup Condition=" '$(VersionlessImplicitFrameworkDefine)' != 'NETFRAMEWORK' ">
<ItemGroup Condition="$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))">
<PackageReference Include="Mono.Posix.NETStandard" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,202 @@
using System;
using System.IO;
using SharpCompress.Common;
using SharpCompress.Compressors;
using SharpCompress.Compressors.BZip2;
using SharpCompress.Compressors.Deflate;
using SharpCompress.Compressors.LZMA;
using SharpCompress.Compressors.Lzw;
using SharpCompress.Compressors.PPMd;
using SharpCompress.Compressors.Reduce;
using SharpCompress.Compressors.ZStandard;
using SharpCompress.IO;
using SharpCompress.Readers;
using SharpCompress.Test.Mocks;
using Xunit;
namespace SharpCompress.Test.Streams;
public class DisposalTests
{
private void VerifyStreamDisposal(
Func<Stream, bool, Stream> createStream,
bool supportsLeaveOpen = true
)
{
// 1. Test Dispose behavior (should dispose inner stream)
{
using var innerStream = new TestStream(new MemoryStream());
// createStream(stream, leaveOpen: false)
var stream = createStream(innerStream, false);
stream.Dispose();
// Some streams might not support disposal of inner stream (e.g. PpmdStream apparently)
// But for those that satisfy the pattern, we assert true.
Assert.True(
innerStream.IsDisposed,
"Stream should have been disposed when leaveOpen=false"
);
}
// 2. Test LeaveOpen behavior (should NOT dispose inner stream)
if (supportsLeaveOpen)
{
using var innerStream = new TestStream(new MemoryStream());
// createStream(stream, leaveOpen: true)
var stream = createStream(innerStream, true);
stream.Dispose();
Assert.False(
innerStream.IsDisposed,
"Stream should NOT have been disposed when leaveOpen=true"
);
}
}
private void VerifyAlwaysDispose(Func<Stream, Stream> createStream)
{
using var innerStream = new TestStream(new MemoryStream());
var stream = createStream(innerStream);
stream.Dispose();
Assert.True(innerStream.IsDisposed, "Stream should have been disposed (AlwaysDispose)");
}
private void VerifyNeverDispose(Func<Stream, Stream> createStream)
{
using var innerStream = new TestStream(new MemoryStream());
var stream = createStream(innerStream);
stream.Dispose();
Assert.False(innerStream.IsDisposed, "Stream should NOT have been disposed (NeverDispose)");
}
[Fact]
public void SourceStream_Disposal()
{
VerifyStreamDisposal(
(stream, leaveOpen) =>
new SourceStream(
stream,
i => null,
new ReaderOptions { LeaveStreamOpen = leaveOpen }
)
);
}
[Fact]
public void ProgressReportingStream_Disposal()
{
VerifyStreamDisposal(
(stream, leaveOpen) =>
new ProgressReportingStream(
stream,
new Progress<ProgressReport>(),
"",
0,
leaveOpen: leaveOpen
)
);
}
[Fact]
public void DataDescriptorStream_Disposal()
{
// DataDescriptorStream DOES dispose inner stream
VerifyAlwaysDispose(stream => new DataDescriptorStream(stream));
}
[Fact]
public void DeflateStream_Disposal()
{
// DeflateStream in SharpCompress always disposes inner stream
VerifyAlwaysDispose(stream => new DeflateStream(stream, CompressionMode.Compress));
}
[Fact]
public void GZipStream_Disposal()
{
// GZipStream in SharpCompress always disposes inner stream
VerifyAlwaysDispose(stream => new GZipStream(stream, CompressionMode.Compress));
}
[Fact]
public void LzwStream_Disposal()
{
VerifyStreamDisposal(
(stream, leaveOpen) =>
{
var lzw = new LzwStream(stream);
lzw.IsStreamOwner = !leaveOpen;
return lzw;
}
);
}
[Fact]
public void PpmdStream_Disposal()
{
// PpmdStream seems to not dispose inner stream based on code analysis
// It takes PpmdProperties which we need to mock or create.
var props = new PpmdProperties();
VerifyNeverDispose(stream => new PpmdStream(props, stream, false));
}
[Fact]
public void LzmaStream_Disposal()
{
// LzmaStream always disposes inner stream
// Need to provide valid properties to avoid crash in constructor (invalid window size)
// 5 bytes: 1 byte properties + 4 bytes dictionary size (little endian)
// Dictionary size = 1024 (0x400) -> 00 04 00 00
var lzmaProps = new byte[] { 0, 0, 4, 0, 0 };
VerifyAlwaysDispose(stream => new LzmaStream(lzmaProps, stream));
}
[Fact]
public void LZipStream_Disposal()
{
// LZipStream now supports leaveOpen parameter
// Use Compress mode to avoid need for valid input header
VerifyStreamDisposal(
(stream, leaveOpen) => new LZipStream(stream, CompressionMode.Compress, leaveOpen)
);
}
[Fact]
public void BZip2Stream_Disposal()
{
// BZip2Stream now supports leaveOpen parameter
VerifyStreamDisposal(
(stream, leaveOpen) =>
new BZip2Stream(stream, CompressionMode.Compress, false, leaveOpen)
);
}
[Fact]
public void ReduceStream_Disposal()
{
// ReduceStream does not dispose inner stream
VerifyNeverDispose(stream => new ReduceStream(stream, 0, 0, 1));
}
[Fact]
public void ZStandard_CompressionStream_Disposal()
{
VerifyStreamDisposal(
(stream, leaveOpen) =>
new CompressionStream(stream, level: 0, bufferSize: 0, leaveOpen: leaveOpen)
);
}
[Fact]
public void ZStandard_DecompressionStream_Disposal()
{
VerifyStreamDisposal(
(stream, leaveOpen) =>
new DecompressionStream(
stream,
bufferSize: 0,
checkEndOfStream: false,
leaveOpen: leaveOpen
)
);
}
}

View File

@@ -0,0 +1,226 @@
using System;
using System.IO;
using System.Text;
using SharpCompress.Compressors;
using SharpCompress.Compressors.BZip2;
using SharpCompress.Compressors.LZMA;
using SharpCompress.Test.Mocks;
using Xunit;
namespace SharpCompress.Test.Streams;
public class LeaveOpenBehaviorTests
{
private static byte[] CreateTestData() =>
Encoding.UTF8.GetBytes("The quick brown fox jumps over the lazy dog");
[Fact]
public void BZip2Stream_Compress_LeaveOpen_False()
{
using var innerStream = new TestStream(new MemoryStream());
using (
var bzip2 = new BZip2Stream(
innerStream,
CompressionMode.Compress,
false,
leaveOpen: false
)
)
{
bzip2.Write(CreateTestData(), 0, CreateTestData().Length);
bzip2.Finish();
}
Assert.True(innerStream.IsDisposed, "Inner stream should be disposed when leaveOpen=false");
}
[Fact]
public void BZip2Stream_Compress_LeaveOpen_True()
{
using var innerStream = new TestStream(new MemoryStream());
byte[] compressed;
using (
var bzip2 = new BZip2Stream(
innerStream,
CompressionMode.Compress,
false,
leaveOpen: true
)
)
{
bzip2.Write(CreateTestData(), 0, CreateTestData().Length);
bzip2.Finish();
}
Assert.False(
innerStream.IsDisposed,
"Inner stream should NOT be disposed when leaveOpen=true"
);
// Should be able to read the compressed data
innerStream.Position = 0;
compressed = new byte[innerStream.Length];
innerStream.Read(compressed, 0, compressed.Length);
Assert.True(compressed.Length > 0);
}
[Fact]
public void BZip2Stream_Decompress_LeaveOpen_False()
{
// First compress some data
var memStream = new MemoryStream();
using (var bzip2 = new BZip2Stream(memStream, CompressionMode.Compress, false, true))
{
bzip2.Write(CreateTestData(), 0, CreateTestData().Length);
bzip2.Finish();
}
memStream.Position = 0;
using var innerStream = new TestStream(memStream);
var decompressed = new byte[CreateTestData().Length];
using (
var bzip2 = new BZip2Stream(
innerStream,
CompressionMode.Decompress,
false,
leaveOpen: false
)
)
{
bzip2.Read(decompressed, 0, decompressed.Length);
}
Assert.True(innerStream.IsDisposed, "Inner stream should be disposed when leaveOpen=false");
Assert.Equal(CreateTestData(), decompressed);
}
[Fact]
public void BZip2Stream_Decompress_LeaveOpen_True()
{
// First compress some data
var memStream = new MemoryStream();
using (var bzip2 = new BZip2Stream(memStream, CompressionMode.Compress, false, true))
{
bzip2.Write(CreateTestData(), 0, CreateTestData().Length);
bzip2.Finish();
}
memStream.Position = 0;
using var innerStream = new TestStream(memStream);
var decompressed = new byte[CreateTestData().Length];
using (
var bzip2 = new BZip2Stream(
innerStream,
CompressionMode.Decompress,
false,
leaveOpen: true
)
)
{
bzip2.Read(decompressed, 0, decompressed.Length);
}
Assert.False(
innerStream.IsDisposed,
"Inner stream should NOT be disposed when leaveOpen=true"
);
Assert.Equal(CreateTestData(), decompressed);
// Should still be able to use the stream
innerStream.Position = 0;
Assert.True(innerStream.CanRead);
}
[Fact]
public void LZipStream_Compress_LeaveOpen_False()
{
using var innerStream = new TestStream(new MemoryStream());
using (var lzip = new LZipStream(innerStream, CompressionMode.Compress, leaveOpen: false))
{
lzip.Write(CreateTestData(), 0, CreateTestData().Length);
lzip.Finish();
}
Assert.True(innerStream.IsDisposed, "Inner stream should be disposed when leaveOpen=false");
}
[Fact]
public void LZipStream_Compress_LeaveOpen_True()
{
using var innerStream = new TestStream(new MemoryStream());
byte[] compressed;
using (var lzip = new LZipStream(innerStream, CompressionMode.Compress, leaveOpen: true))
{
lzip.Write(CreateTestData(), 0, CreateTestData().Length);
lzip.Finish();
}
Assert.False(
innerStream.IsDisposed,
"Inner stream should NOT be disposed when leaveOpen=true"
);
// Should be able to read the compressed data
innerStream.Position = 0;
compressed = new byte[innerStream.Length];
innerStream.Read(compressed, 0, compressed.Length);
Assert.True(compressed.Length > 0);
}
[Fact]
public void LZipStream_Decompress_LeaveOpen_False()
{
// First compress some data
var memStream = new MemoryStream();
using (var lzip = new LZipStream(memStream, CompressionMode.Compress, true))
{
lzip.Write(CreateTestData(), 0, CreateTestData().Length);
lzip.Finish();
}
memStream.Position = 0;
using var innerStream = new TestStream(memStream);
var decompressed = new byte[CreateTestData().Length];
using (var lzip = new LZipStream(innerStream, CompressionMode.Decompress, leaveOpen: false))
{
lzip.Read(decompressed, 0, decompressed.Length);
}
Assert.True(innerStream.IsDisposed, "Inner stream should be disposed when leaveOpen=false");
Assert.Equal(CreateTestData(), decompressed);
}
[Fact]
public void LZipStream_Decompress_LeaveOpen_True()
{
// First compress some data
var memStream = new MemoryStream();
using (var lzip = new LZipStream(memStream, CompressionMode.Compress, true))
{
lzip.Write(CreateTestData(), 0, CreateTestData().Length);
lzip.Finish();
}
memStream.Position = 0;
using var innerStream = new TestStream(memStream);
var decompressed = new byte[CreateTestData().Length];
using (var lzip = new LZipStream(innerStream, CompressionMode.Decompress, leaveOpen: true))
{
lzip.Read(decompressed, 0, decompressed.Length);
}
Assert.False(
innerStream.IsDisposed,
"Inner stream should NOT be disposed when leaveOpen=true"
);
Assert.Equal(CreateTestData(), decompressed);
// Should still be able to use the stream
innerStream.Position = 0;
Assert.True(innerStream.CanRead);
}
}

View File

@@ -3,7 +3,6 @@ using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers;
using Xunit;
@@ -46,7 +45,7 @@ public class TestBase : IDisposable
public void Dispose() => Directory.Delete(SCRATCH_BASE_PATH, true);
public void VerifyFiles(bool skipSoIndexes = false)
public void VerifyFiles()
{
if (UseExtensionInsteadOfNameToVerify)
{
@@ -54,7 +53,7 @@ public class TestBase : IDisposable
}
else
{
VerifyFilesByName(skipSoIndexes);
VerifyFilesByName();
}
}
@@ -73,23 +72,10 @@ public class TestBase : IDisposable
}
}
private void VerifyFilesByName(bool skipSoIndexes)
protected void VerifyFilesByName()
{
var extracted = Directory
.EnumerateFiles(SCRATCH_FILES_PATH, "*.*", SearchOption.AllDirectories)
.Where(x =>
{
if (
skipSoIndexes
&& Path.GetFileName(x)
.EndsWith(SOZipIndex.INDEX_EXTENSION, StringComparison.OrdinalIgnoreCase)
)
{
return false;
}
return true;
})
.ToLookup(path => path.Substring(SCRATCH_FILES_PATH.Length));
var original = Directory
.EnumerateFiles(ORIGINAL_FILES_PATH, "*.*", SearchOption.AllDirectories)

View File

@@ -1,257 +0,0 @@
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SharpCompress.Archives.Zip;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers.Zip;
using SharpCompress.Test.Mocks;
using SharpCompress.Writers;
using SharpCompress.Writers.Zip;
using Xunit;
namespace SharpCompress.Test.Zip;
public class SoZipReaderTests : TestBase
{
[Fact]
public async Task SOZip_Reader_RegularZip_NoSozipEntries()
{
// Regular zip files should not have SOZip entries
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ZipReader.Open(stream);
while (await reader.MoveToNextEntryAsync())
{
// Regular zip entries should NOT be SOZip
Assert.False(reader.Entry.IsSozip, $"Entry {reader.Entry.Key} should not be SOZip");
Assert.False(
reader.Entry.IsSozipIndexFile,
$"Entry {reader.Entry.Key} should not be a SOZip index file"
);
}
}
[Fact]
public void SOZip_Archive_RegularZip_NoSozipEntries()
{
// Regular zip files should not have SOZip entries
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.zip");
using Stream stream = File.OpenRead(path);
using var archive = ZipArchive.Open(stream);
foreach (var entry in archive.Entries)
{
// Regular zip entries should NOT be SOZip
Assert.False(entry.IsSozip, $"Entry {entry.Key} should not be SOZip");
Assert.False(
entry.IsSozipIndexFile,
$"Entry {entry.Key} should not be a SOZip index file"
);
}
}
[Fact]
public void SOZip_Archive_ReadSOZipFile()
{
// Read the SOZip test archive
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using Stream stream = File.OpenRead(path);
using var archive = ZipArchive.Open(stream);
var entries = archive.Entries.ToList();
// Should have 3 entries: data.txt, .data.txt.sozip.idx, and small.txt
Assert.Equal(3, entries.Count);
// Verify we have one SOZip index file
var indexFiles = entries.Where(e => e.IsSozipIndexFile).ToList();
Assert.Single(indexFiles);
Assert.Equal(".data.txt.sozip.idx", indexFiles[0].Key);
// Verify the index file is not compressed
Assert.Equal(CompressionType.None, indexFiles[0].CompressionType);
// Read and validate the index
using (var indexStream = indexFiles[0].OpenEntryStream())
{
using var memStream = new MemoryStream();
indexStream.CopyTo(memStream);
var indexBytes = memStream.ToArray();
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
Assert.Equal(1024u, index.ChunkSize); // As set in CreateSOZipTestArchive
Assert.True(index.UncompressedSize > 0);
Assert.True(index.OffsetCount > 0);
}
// Verify the data file can be read correctly
var dataEntry = entries.First(e => e.Key == "data.txt");
using (var dataStream = dataEntry.OpenEntryStream())
{
using var reader = new StreamReader(dataStream);
var content = reader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
// Verify the small file
var smallEntry = entries.First(e => e.Key == "small.txt");
Assert.False(smallEntry.IsSozipIndexFile);
using (var smallStream = smallEntry.OpenEntryStream())
{
using var reader = new StreamReader(smallStream);
var content = reader.ReadToEnd();
Assert.Equal("Small content", content);
}
}
[Fact]
public async Task SOZip_Reader_ReadSOZipFile()
{
// Read the SOZip test archive with ZipReader
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ZipReader.Open(stream);
var foundData = false;
var foundIndex = false;
var foundSmall = false;
while (await reader.MoveToNextEntryAsync())
{
if (reader.Entry.Key == "data.txt")
{
foundData = true;
Assert.False(reader.Entry.IsSozipIndexFile);
using var entryStream = reader.OpenEntryStream();
using var streamReader = new StreamReader(entryStream);
var content = streamReader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
else if (reader.Entry.Key == ".data.txt.sozip.idx")
{
foundIndex = true;
Assert.True(reader.Entry.IsSozipIndexFile);
using var indexStream = reader.OpenEntryStream();
using var memStream = new MemoryStream();
await indexStream.CopyToAsync(memStream);
var indexBytes = memStream.ToArray();
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
}
else if (reader.Entry.Key == "small.txt")
{
foundSmall = true;
Assert.False(reader.Entry.IsSozipIndexFile);
}
}
Assert.True(foundData, "data.txt entry not found");
Assert.True(foundIndex, ".data.txt.sozip.idx entry not found");
Assert.True(foundSmall, "small.txt entry not found");
}
[Fact]
public void SOZip_Archive_DetectsIndexFileByName()
{
// Create a zip with a SOZip index file (by name pattern)
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file that looks like a SOZip index (by name pattern)
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
// Test with ZipArchive
using var archive = ZipArchive.Open(memoryStream);
var entries = archive.Entries.ToList();
Assert.Equal(2, entries.Count);
var regularEntry = entries.First(e => e.Key == "test.txt");
Assert.False(regularEntry.IsSozipIndexFile);
Assert.False(regularEntry.IsSozip); // No SOZip extra field
var indexEntry = entries.First(e => e.Key == ".test.txt.sozip.idx");
Assert.True(indexEntry.IsSozipIndexFile);
}
[Fact]
public async Task SOZip_Reader_DetectsIndexFileByName()
{
// Create a zip with a SOZip index file (by name pattern)
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file that looks like a SOZip index (by name pattern)
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
// Test with ZipReader
using Stream stream = new ForwardOnlyStream(memoryStream);
using var reader = ZipReader.Open(stream);
var foundRegular = false;
var foundIndex = false;
while (await reader.MoveToNextEntryAsync())
{
if (reader.Entry.Key == "test.txt")
{
foundRegular = true;
Assert.False(reader.Entry.IsSozipIndexFile);
Assert.False(reader.Entry.IsSozip);
}
else if (reader.Entry.Key == ".test.txt.sozip.idx")
{
foundIndex = true;
Assert.True(reader.Entry.IsSozipIndexFile);
}
}
Assert.True(foundRegular, "Regular entry not found");
Assert.True(foundIndex, "Index entry not found");
}
}

View File

@@ -1,358 +0,0 @@
using System;
using System.IO;
using System.Linq;
using System.Text;
using SharpCompress.Archives.Zip;
using SharpCompress.Common;
using SharpCompress.Common.Zip.SOZip;
using SharpCompress.Readers;
using SharpCompress.Writers;
using SharpCompress.Writers.Zip;
using Xunit;
namespace SharpCompress.Test.Zip;
public class SoZipWriterTests : TestBase
{
[Fact]
public void SOZipIndex_RoundTrip()
{
// Create an index
var offsets = new ulong[] { 0, 1024, 2048, 3072 };
var originalIndex = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100000,
compressedSize: 50000,
compressedOffsets: offsets
);
// Serialize to bytes
var bytes = originalIndex.ToByteArray();
// Deserialize back
var parsedIndex = SOZipIndex.Read(bytes);
// Verify all fields
Assert.Equal(SOZipIndex.SOZIP_VERSION, parsedIndex.Version);
Assert.Equal(32768u, parsedIndex.ChunkSize);
Assert.Equal(100000ul, parsedIndex.UncompressedSize);
Assert.Equal(50000ul, parsedIndex.CompressedSize);
Assert.Equal(4u, parsedIndex.OffsetCount);
Assert.Equal(offsets, parsedIndex.CompressedOffsets);
}
[Fact]
public void SOZipIndex_Read_InvalidMagic_ThrowsException()
{
var invalidData = new byte[] { 0x00, 0x00, 0x00, 0x00 };
var exception = Assert.Throws<InvalidDataException>(() => SOZipIndex.Read(invalidData));
Assert.Contains("magic number mismatch", exception.Message);
}
[Fact]
public void SOZipIndex_GetChunkIndex()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840, // 5 * 32768
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0, index.GetChunkIndex(0));
Assert.Equal(0, index.GetChunkIndex(32767));
Assert.Equal(1, index.GetChunkIndex(32768));
Assert.Equal(2, index.GetChunkIndex(65536));
Assert.Equal(4, index.GetChunkIndex(163839));
}
[Fact]
public void SOZipIndex_GetCompressedOffset()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840,
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0ul, index.GetCompressedOffset(0));
Assert.Equal(1000ul, index.GetCompressedOffset(1));
Assert.Equal(2000ul, index.GetCompressedOffset(2));
Assert.Equal(3000ul, index.GetCompressedOffset(3));
Assert.Equal(4000ul, index.GetCompressedOffset(4));
}
[Fact]
public void SOZipIndex_GetUncompressedOffset()
{
var offsets = new ulong[] { 0, 1000, 2000, 3000, 4000 };
var index = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 163840,
compressedSize: 5000,
compressedOffsets: offsets
);
Assert.Equal(0ul, index.GetUncompressedOffset(0));
Assert.Equal(32768ul, index.GetUncompressedOffset(1));
Assert.Equal(65536ul, index.GetUncompressedOffset(2));
Assert.Equal(98304ul, index.GetUncompressedOffset(3));
Assert.Equal(131072ul, index.GetUncompressedOffset(4));
}
[Fact]
public void SOZipIndex_GetIndexFileName()
{
Assert.Equal(".file.txt.sozip.idx", SOZipIndex.GetIndexFileName("file.txt"));
Assert.Equal("dir/.file.txt.sozip.idx", SOZipIndex.GetIndexFileName("dir/file.txt"));
Assert.Equal("a/b/.file.txt.sozip.idx", SOZipIndex.GetIndexFileName("a/b/file.txt"));
}
[Fact]
public void SOZipIndex_IsIndexFile()
{
Assert.True(SOZipIndex.IsIndexFile(".file.txt.sozip.idx"));
Assert.True(SOZipIndex.IsIndexFile("dir/.file.txt.sozip.idx"));
Assert.True(SOZipIndex.IsIndexFile(".test.sozip.idx"));
Assert.False(SOZipIndex.IsIndexFile("file.txt"));
Assert.False(SOZipIndex.IsIndexFile("file.sozip.idx")); // Missing leading dot
Assert.False(SOZipIndex.IsIndexFile(".file.txt")); // Missing .sozip.idx
Assert.False(SOZipIndex.IsIndexFile(""));
Assert.False(SOZipIndex.IsIndexFile(null!));
}
[Fact]
public void SOZipIndex_GetMainFileName()
{
Assert.Equal("file.txt", SOZipIndex.GetMainFileName(".file.txt.sozip.idx"));
Assert.Equal("dir/file.txt", SOZipIndex.GetMainFileName("dir/.file.txt.sozip.idx"));
Assert.Equal("test", SOZipIndex.GetMainFileName(".test.sozip.idx"));
Assert.Null(SOZipIndex.GetMainFileName("file.txt"));
Assert.Null(SOZipIndex.GetMainFileName(""));
}
[Fact]
public void ZipEntry_IsSozipIndexFile_Detection()
{
// Create a zip with a file that has a SOZip index file name pattern
using var memoryStream = new MemoryStream();
using (
var writer = WriterFactory.Open(
memoryStream,
ArchiveType.Zip,
new ZipWriterOptions(CompressionType.Deflate) { LeaveStreamOpen = true }
)
)
{
// Write a regular file
writer.Write("test.txt", new MemoryStream(Encoding.UTF8.GetBytes("Hello World")));
// Write a file with SOZip index name pattern
var indexData = new SOZipIndex(
chunkSize: 32768,
uncompressedSize: 100,
compressedSize: 50,
compressedOffsets: new ulong[] { 0 }
);
writer.Write(".test.txt.sozip.idx", new MemoryStream(indexData.ToByteArray()));
}
memoryStream.Position = 0;
using var archive = ZipArchive.Open(memoryStream);
var entries = archive.Entries.ToList();
Assert.Equal(2, entries.Count);
var regularEntry = entries.First(e => e.Key == "test.txt");
Assert.False(regularEntry.IsSozipIndexFile);
Assert.False(regularEntry.IsSozip); // No SOZip extra field
var indexEntry = entries.First(e => e.Key == ".test.txt.sozip.idx");
Assert.True(indexEntry.IsSozipIndexFile);
}
[Fact]
public void ZipWriterOptions_SOZipDefaults()
{
var options = new ZipWriterOptions(CompressionType.Deflate);
Assert.False(options.EnableSOZip);
Assert.Equal((int)SOZipIndex.DEFAULT_CHUNK_SIZE, options.SOZipChunkSize);
Assert.Equal(1048576L, options.SOZipMinFileSize); // 1MB
}
[Fact]
public void ZipWriterEntryOptions_SOZipDefaults()
{
var options = new ZipWriterEntryOptions();
Assert.Null(options.EnableSOZip);
}
[Fact]
public void SOZip_RoundTrip_CompressAndDecompress()
{
// Create a SOZip archive from Original files
var archivePath = Path.Combine(SCRATCH2_FILES_PATH, "test.sozip.zip");
using (var stream = File.Create(archivePath))
{
var options = new ZipWriterOptions(CompressionType.Deflate)
{
EnableSOZip = true,
SOZipMinFileSize = 1024, // 1KB to ensure test files qualify
LeaveStreamOpen = false,
};
using var writer = new ZipWriter(stream, options);
// Write all files from Original directory
var files = Directory.GetFiles(ORIGINAL_FILES_PATH, "*", SearchOption.AllDirectories);
foreach (var filePath in files)
{
var relativePath = filePath
.Substring(ORIGINAL_FILES_PATH.Length + 1)
.Replace('\\', '/');
using var fileStream = File.OpenRead(filePath);
writer.Write(relativePath, fileStream, new ZipWriterEntryOptions());
}
}
// Validate the archive was created and has files
Assert.True(File.Exists(archivePath));
// Validate the archive has SOZip entries
using (var stream = File.OpenRead(archivePath))
{
using var archive = ZipArchive.Open(stream);
var allEntries = archive.Entries.ToList();
// Archive should have files
Assert.NotEmpty(allEntries);
var sozipIndexEntries = allEntries.Where(e => e.IsSozipIndexFile).ToList();
// Should have at least one SOZip index file
Assert.NotEmpty(sozipIndexEntries);
// Verify index files have valid SOZip index data
foreach (var indexEntry in sozipIndexEntries)
{
// Check that the entry is stored (not compressed)
Assert.Equal(CompressionType.None, indexEntry.CompressionType);
using var indexStream = indexEntry.OpenEntryStream();
using var memStream = new MemoryStream();
indexStream.CopyTo(memStream);
var indexBytes = memStream.ToArray();
// Debug: Check first 4 bytes
Assert.True(
indexBytes.Length >= 4,
$"Index file too small: {indexBytes.Length} bytes"
);
// Should be able to parse the index without exception
var index = SOZipIndex.Read(indexBytes);
Assert.Equal(SOZipIndex.SOZIP_VERSION, index.Version);
Assert.True(index.ChunkSize > 0);
Assert.True(index.UncompressedSize > 0);
Assert.True(index.OffsetCount > 0);
// Verify there's a corresponding data file
var mainFileName = SOZipIndex.GetMainFileName(indexEntry.Key!);
Assert.NotNull(mainFileName);
Assert.Contains(allEntries, e => e.Key == mainFileName);
}
}
// Read and decompress the archive
using (var stream = File.OpenRead(archivePath))
{
using var reader = ReaderFactory.Open(stream);
reader.WriteAllToDirectory(
SCRATCH_FILES_PATH,
new ExtractionOptions { ExtractFullPath = true }
);
}
// Verify extracted files match originals
VerifyFiles(true);
}
[Fact]
public void CreateSOZipTestArchive()
{
// Create a SOZip test archive that can be committed to the repository
var archivePath = Path.Combine(TEST_ARCHIVES_PATH, "Zip.sozip.zip");
using (var stream = File.Create(archivePath))
{
var options = new ZipWriterOptions(CompressionType.Deflate)
{
EnableSOZip = true,
SOZipMinFileSize = 100, // Low threshold to ensure test content is optimized
SOZipChunkSize = 1024, // Small chunks for testing
LeaveStreamOpen = false,
};
using var writer = new ZipWriter(stream, options);
// Create test content that's large enough to create multiple chunks
var largeContent = new string('A', 5000); // 5KB of 'A's
// Write a file with enough data to be SOZip-optimized
writer.Write(
"data.txt",
new MemoryStream(Encoding.UTF8.GetBytes(largeContent)),
new ZipWriterEntryOptions()
);
// Write a smaller file that won't be SOZip-optimized
writer.Write(
"small.txt",
new MemoryStream(Encoding.UTF8.GetBytes("Small content")),
new ZipWriterEntryOptions()
);
}
// Validate the archive was created
Assert.True(File.Exists(archivePath));
// Validate it's a valid SOZip archive
using (var stream = File.OpenRead(archivePath))
{
using var archive = ZipArchive.Open(stream);
var entries = archive.Entries.ToList();
// Should have data file, small file, and index file
Assert.Equal(3, entries.Count);
// Verify we have one SOZip index file
var indexFiles = entries.Where(e => e.IsSozipIndexFile).ToList();
Assert.Single(indexFiles);
// Verify the index file
var indexEntry = indexFiles.First();
Assert.Equal(".data.txt.sozip.idx", indexEntry.Key);
// Verify the data file can be read
var dataEntry = entries.First(e => e.Key == "data.txt");
using var dataStream = dataEntry.OpenEntryStream();
using var reader = new StreamReader(dataStream);
var content = reader.ReadToEnd();
Assert.Equal(5000, content.Length);
Assert.True(content.All(c => c == 'A'));
}
}
}

View File

@@ -251,4 +251,58 @@ public class ZipReaderAsyncTests : ReaderTests
}
Assert.Equal(8, count);
}
[Fact]
public async ValueTask EntryStream_Dispose_DoesNotThrow_OnNonSeekableStream_Deflate_Async()
{
// Since version 0.41.0: EntryStream.DisposeAsync() should not throw NotSupportedException
// when FlushAsync() fails on non-seekable streams (Deflate compression)
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.dd.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ReaderFactory.Open(stream);
// This should not throw, even if internal FlushAsync() fails
while (await reader.MoveToNextEntryAsync())
{
if (!reader.Entry.IsDirectory)
{
#if LEGACY_DOTNET
using var entryStream = await reader.OpenEntryStreamAsync();
#else
await using var entryStream = await reader.OpenEntryStreamAsync();
#endif
// Read some data
var buffer = new byte[1024];
await entryStream.ReadAsync(buffer, 0, buffer.Length);
// DisposeAsync should not throw NotSupportedException
}
}
}
[Fact]
public async ValueTask EntryStream_Dispose_DoesNotThrow_OnNonSeekableStream_LZMA_Async()
{
// Since version 0.41.0: EntryStream.DisposeAsync() should not throw NotSupportedException
// when FlushAsync() fails on non-seekable streams (LZMA compression)
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.lzma.dd.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ReaderFactory.Open(stream);
// This should not throw, even if internal FlushAsync() fails
while (await reader.MoveToNextEntryAsync())
{
if (!reader.Entry.IsDirectory)
{
#if LEGACY_DOTNET
using var entryStream = await reader.OpenEntryStreamAsync();
#else
await using var entryStream = await reader.OpenEntryStreamAsync();
#endif
// Read some data
var buffer = new byte[1024];
await entryStream.ReadAsync(buffer, 0, buffer.Length);
// DisposeAsync should not throw NotSupportedException
}
}
}
}

View File

@@ -436,4 +436,50 @@ public class ZipReaderTests : ReaderTests
Assert.Equal(archiveKeys.OrderBy(k => k), readerKeys.OrderBy(k => k));
}
}
[Fact]
public void EntryStream_Dispose_DoesNotThrow_OnNonSeekableStream_Deflate()
{
// Since version 0.41.0: EntryStream.Dispose() should not throw NotSupportedException
// when Flush() fails on non-seekable streams (Deflate compression)
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.dd.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ReaderFactory.Open(stream);
// This should not throw, even if internal Flush() fails
while (reader.MoveToNextEntry())
{
if (!reader.Entry.IsDirectory)
{
using var entryStream = reader.OpenEntryStream();
// Read some data
var buffer = new byte[1024];
entryStream.Read(buffer, 0, buffer.Length);
// Dispose should not throw NotSupportedException
}
}
}
[Fact]
public void EntryStream_Dispose_DoesNotThrow_OnNonSeekableStream_LZMA()
{
// Since version 0.41.0: EntryStream.Dispose() should not throw NotSupportedException
// when Flush() fails on non-seekable streams (LZMA compression)
var path = Path.Combine(TEST_ARCHIVES_PATH, "Zip.lzma.dd.zip");
using Stream stream = new ForwardOnlyStream(File.OpenRead(path));
using var reader = ReaderFactory.Open(stream);
// This should not throw, even if internal Flush() fails
while (reader.MoveToNextEntry())
{
if (!reader.Entry.IsDirectory)
{
using var entryStream = reader.OpenEntryStream();
// Read some data
var buffer = new byte[1024];
entryStream.Read(buffer, 0, buffer.Length);
// Dispose should not throw NotSupportedException
}
}
}
}