starred/sharpcompress
1
0
Fork 0
mirror of https://github.com/adamhathcock/sharpcompress.git synced 2026-02-10 21:22:11 +00:00
Code Issues 975 Packages Projects Releases 20 Wiki Activity

[PR #1131] [MERGED] More test fixes and some perf changes #1561

New Issue
Open
opened 2026-01-29 22:21:09 +00:00 by claunia · 0 comments
claunia commented 2026-01-29 22:21:09 +00:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/adamhathcock/sharpcompress/pull/1131
Author: @adamhathcock
Created: 1/15/2026
Status: ✅ Merged
Merged: 1/15/2026
Merged by: @adamhathcock

Base: master ← Head: adam/async-again

📝 Commits (9)

📊 Changes

24 files changed (+488 additions, -163 deletions)

View changed files

📝 src/SharpCompress/Archives/Rar/RarArchive.cs (+14 -0)
📝 src/SharpCompress/Common/AsyncBinaryReader.cs (+12 -4)
📝 src/SharpCompress/Common/Zip/Headers/DirectoryEndHeader.cs (+2 -1)
📝 src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs (+6 -4)
📝 src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs (+4 -2)
📝 src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs (+4 -5)
📝 src/SharpCompress/Common/Zip/SeekableZipHeaderFactory.cs (+25 -86)
📝 src/SharpCompress/Common/Zip/ZipHeaderFactory.cs (+1 -1)
📝 src/SharpCompress/Compressors/Deflate/Inflate.cs (+27 -17)
📝 src/SharpCompress/Compressors/Rar/RarStream.cs (+1 -1)
📝 src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs (+4 -9)
📝 src/SharpCompress/IO/BufferedSubStream.cs (+6 -2)
📝 src/SharpCompress/IO/SharpCompressStream.cs (+0 -2)
➕ tests/SharpCompress.Test/Ace/AceReaderAsyncTests.cs (+145 -0)
➕ tests/SharpCompress.Test/Arj/ArjReaderAsyncTests.cs (+176 -0)
📝 tests/SharpCompress.Test/BZip2/BZip2StreamAsyncTests.cs (+9 -8)
📝 tests/SharpCompress.Test/GZip/GZipReaderAsyncTests.cs (+7 -1)
📝 tests/SharpCompress.Test/Rar/RarArchiveAsyncTests.cs (+4 -1)
📝 tests/SharpCompress.Test/SevenZip/SevenZipArchiveAsyncTests.cs (+21 -10)
📝 tests/SharpCompress.Test/Tar/TarArchiveAsyncTests.cs (+5 -3)

...and 4 more files

📄 Description

This pull request introduces several improvements and refactorings to asynchronous ZIP archive handling and Deflate decompression. The main changes include refactoring buffer management for memory efficiency, updating async reading APIs for consistency, and improving resource cleanup. These updates enhance performance, reduce allocations, and make the codebase easier to maintain.

Async ZIP reading and buffer management improvements

  • Refactored SeekableZipHeaderFactory to use ArrayPool<byte> for buffer allocation in async header seeking, reducing memory allocations and improving performance. Also, updated the header search logic to use spans for efficient data access. [1] [2]
  • Updated async ZIP header reading methods to use explicit buffer arrays and offsets instead of returning new arrays, improving memory management and consistency across DirectoryEndHeader, DirectoryEntryHeader, LocalEntryHeader, and Zip64DirectoryEndHeader. [1] [2] [3] [4]
  • Added a SkipAsync method to AsyncBinaryReader and replaced unnecessary buffer reads with skips in ZIP header parsing, improving efficiency. [1] [2]

Deflate decompression memory optimization

  • Changed the Deflate decompression window from a raw byte[] to an IMemoryOwner<byte> from MemoryPool<byte>, ensuring pooled buffer usage and proper disposal for reduced GC pressure and memory leaks. All array copy operations were updated to use spans. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Resource cleanup and async disposal

  • Implemented DisposeAsync in RarArchive to support proper async resource cleanup, including disposal of unpacker resources.
  • Ensured the Deflate window buffer is disposed correctly in InflateBlocks.Free() to prevent memory leaks when decompressing data.

These changes collectively improve performance, resource management, and maintainability of the archive and compression codebase.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/adamhathcock/sharpcompress/pull/1131 **Author:** [@adamhathcock](https://github.com/adamhathcock) **Created:** 1/15/2026 **Status:** ✅ Merged **Merged:** 1/15/2026 **Merged by:** [@adamhathcock](https://github.com/adamhathcock) **Base:** `master` ← **Head:** `adam/async-again` --- ### 📝 Commits (9) - [`c1d240b`](https://github.com/adamhathcock/sharpcompress/commit/c1d240b516abddd11437673c2a9237e4e46581b1) Fix more tests - [`ebfa16f`](https://github.com/adamhathcock/sharpcompress/commit/ebfa16f09f5253d12a2cd97b5dbfcf74cc9a776f) more test fixes - [`84b5b5a`](https://github.com/adamhathcock/sharpcompress/commit/84b5b5a717dfc0772b6a3f4a7c84a65d3666048a) add more tests - [`81a2060`](https://github.com/adamhathcock/sharpcompress/commit/81a2060c7560d250aa853ce7105d02ad1530e76b) reduce memory usage on headers - [`437271c`](https://github.com/adamhathcock/sharpcompress/commit/437271c6a24c054c43211b17e35c4e742dcfc626) change byte[] to memory using pool - [`38eec23`](https://github.com/adamhathcock/sharpcompress/commit/38eec23e079ff26ac6ddeaff7fa971d5c123bce8) rar byte[] better - [`227fec6`](https://github.com/adamhathcock/sharpcompress/commit/227fec66ad7403a1bf4650f552d1cbac3cef6280) more pooling - [`c55a383`](https://github.com/adamhathcock/sharpcompress/commit/c55a3831121477da8d876f9e97a88aa08e23740c) Merge remote-tracking branch 'origin/master' into adam/async-again - [`ec310c8`](https://github.com/adamhathcock/sharpcompress/commit/ec310c87de7ed03efa4d018e20638ea81bb40dcb) merge fixes and fmt ### 📊 Changes **24 files changed** (+488 additions, -163 deletions) <details> <summary>View changed files</summary> 📝 `src/SharpCompress/Archives/Rar/RarArchive.cs` (+14 -0) 📝 `src/SharpCompress/Common/AsyncBinaryReader.cs` (+12 -4) 📝 `src/SharpCompress/Common/Zip/Headers/DirectoryEndHeader.cs` (+2 -1) 📝 `src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs` (+6 -4) 📝 `src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs` (+4 -2) 📝 `src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs` (+4 -5) 📝 `src/SharpCompress/Common/Zip/SeekableZipHeaderFactory.cs` (+25 -86) 📝 `src/SharpCompress/Common/Zip/ZipHeaderFactory.cs` (+1 -1) 📝 `src/SharpCompress/Compressors/Deflate/Inflate.cs` (+27 -17) 📝 `src/SharpCompress/Compressors/Rar/RarStream.cs` (+1 -1) 📝 `src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs` (+4 -9) 📝 `src/SharpCompress/IO/BufferedSubStream.cs` (+6 -2) 📝 `src/SharpCompress/IO/SharpCompressStream.cs` (+0 -2) ➕ `tests/SharpCompress.Test/Ace/AceReaderAsyncTests.cs` (+145 -0) ➕ `tests/SharpCompress.Test/Arj/ArjReaderAsyncTests.cs` (+176 -0) 📝 `tests/SharpCompress.Test/BZip2/BZip2StreamAsyncTests.cs` (+9 -8) 📝 `tests/SharpCompress.Test/GZip/GZipReaderAsyncTests.cs` (+7 -1) 📝 `tests/SharpCompress.Test/Rar/RarArchiveAsyncTests.cs` (+4 -1) 📝 `tests/SharpCompress.Test/SevenZip/SevenZipArchiveAsyncTests.cs` (+21 -10) 📝 `tests/SharpCompress.Test/Tar/TarArchiveAsyncTests.cs` (+5 -3) _...and 4 more files_ </details> ### 📄 Description This pull request introduces several improvements and refactorings to asynchronous ZIP archive handling and Deflate decompression. The main changes include refactoring buffer management for memory efficiency, updating async reading APIs for consistency, and improving resource cleanup. These updates enhance performance, reduce allocations, and make the codebase easier to maintain. ### Async ZIP reading and buffer management improvements * Refactored `SeekableZipHeaderFactory` to use `ArrayPool<byte>` for buffer allocation in async header seeking, reducing memory allocations and improving performance. Also, updated the header search logic to use spans for efficient data access. [[1]](diffhunk://#diff-e4bf6d19a79bff81ad297d9d193c48548686e39806ed8e0f928a6afd99f7f23aL250-R202) [[2]](diffhunk://#diff-e4bf6d19a79bff81ad297d9d193c48548686e39806ed8e0f928a6afd99f7f23aR211-R215) * Updated async ZIP header reading methods to use explicit buffer arrays and offsets instead of returning new arrays, improving memory management and consistency across `DirectoryEndHeader`, `DirectoryEntryHeader`, `LocalEntryHeader`, and `Zip64DirectoryEndHeader`. [[1]](diffhunk://#diff-c07e955791385b6bc8e78f729f4bf89c594ffb7a2a8e10e39a095b5a7e6405dbL32-R33) [[2]](diffhunk://#diff-115c96a4f555b3918f7c4e399e83ca7d8a9eede91e237aa989246b6bbec61761L56-R61) [[3]](diffhunk://#diff-f37cd5cf680b244796a45c6316c30e473b6c39ca38b5924b5c9aab0fe201e00eL40-R43) [[4]](diffhunk://#diff-0c1915b061ec322d6ba9aa9e751546b3a1e9505847c0b711ccbc03334252f0b8L41-R45) * Added a `SkipAsync` method to `AsyncBinaryReader` and replaced unnecessary buffer reads with skips in ZIP header parsing, improving efficiency. [[1]](diffhunk://#diff-fb995dad22a6c5e6b7fbcd992f623685cd43e387568a8b2ead2581b11ba708a5L54-R66) [[2]](diffhunk://#diff-8f0c2dac486475c7cff3782cf74b4fcd7f8f62962203af2fb7b89143d42eefb7L82-R82) ### Deflate decompression memory optimization * Changed the Deflate decompression window from a raw `byte[]` to an `IMemoryOwner<byte>` from `MemoryPool<byte>`, ensuring pooled buffer usage and proper disposal for reduced GC pressure and memory leaks. All array copy operations were updated to use spans. [[1]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL119-R127) [[2]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL343-R344) [[3]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acR719-R726) [[4]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1216-R1223) [[5]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1262-R1269) [[6]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1399-R1406) [[7]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1464-R1478) [[8]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1493-R1507) [[9]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1512-R1527) [[10]](diffhunk://#diff-8f4178f88d6ead96112102c2843a01cd5e609a6beae3debf67796e074de896acL1563-R1573) ### Resource cleanup and async disposal * Implemented `DisposeAsync` in `RarArchive` to support proper async resource cleanup, including disposal of unpacker resources. * Ensured the Deflate window buffer is disposed correctly in `InflateBlocks.Free()` to prevent memory leaks when decompressing data. These changes collectively improve performance, resource management, and maintainability of the archive and compression codebase. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-29 22:21:09 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
feedback requested
not bug
pull-request
Mirrored from GitHub Pull Request
 question
up for grabs
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#1561
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?