Commit Graph

1001 Commits

Author SHA1 Message Date
Adam Hathcock
deaabb5504 Merge pull request #1469 from adamhathcock/dependabot/nuget/build/all-dependencies-232486072c
Bump the all-dependencies group with 5 updates
2026-10-26 16:35:15 +00:00
Adam Hathcock
dd6d09c6d2 Reduce allocations in 7z encryption and codec lookup 2026-10-26 13:45:21 +00:00
Adam Hathcock
fa96277b96 Merge pull request #1476 from jaysanati/feature/7z-write-encryption-master
SevenZipWriter: password protection (7zAES) and encrypted headers
2026-10-26 13:17:32 +00:00
Adam Hathcock
2fa1dec513 Merge pull request #1473 from adamhathcock/review/fix-zip64-descriptor-width
Fix ZIP64 streaming data descriptor width parsing
2026-10-26 11:36:16 +00:00
Adam Hathcock
8fa17607d8 Merge pull request #1477 from adamhathcock/perf/xz-async-decoding
Optimize XZ async reads and buffered LZMA2 decoding
2026-10-26 08:27:00 +00:00
Adam Hathcock
0a716d4236 Merge pull request #1475 from jaysanati/fix/7z-encrypted-compression-type
Report the codec, not 7zAES, as an encrypted 7z entry's CompressionType
2026-10-26 08:24:34 +00:00
Jay Sanati
521a33688a Add 7z write encryption (7zAES) and encrypted headers
SevenZipWriterOptions gains Password and EncryptHeader (#1472). With a
password, each folder is written as 7zAES -> LZMA/LZMA2 in the same
layout 7-Zip uses: AES is coder 0 and reads the packed stream, the
codec is coder 1 and reads AES's output. AES properties match 7-Zip's
defaults (NumCyclesPower 19, no salt, random 16-byte IV per folder), so
the key is derived once per archive. EncryptHeader runs the header
through the same chain; an encrypted header is always LZMA-compressed.
Empty files have no stream and stay unencrypted, as in 7-Zip.

- SevenZipAesKey: the 7zAES KDF, moved unchanged from AesDecoderStream
  so the encoder and decoder share it.
- AesEncoderStream: AES-256-CBC, zero-fills the last block on Finish.
- SevenZipAesEncryption: per-archive key and per-folder IV/properties.
- EncryptHeader without a password, or an empty password, throws
  ArgumentException.

Uses only Aes (CBC) and SHA256, available on every target framework and
Native AOT safe. Verified with 7-Zip 26.05 (7z t, 7z x, 7z l -slt) and
WinRAR for LZMA and LZMA2, with and without header encryption.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 13:27:20 -07:00
Jay Sanati
a7d3b100a0 Report the codec, not 7zAES, as an encrypted 7z entry's CompressionType
7-Zip writes 7zAES as the first coder of an encrypted folder, in front
of the codec. SevenZipFilePart.GetCompression() looked at the first
coder only, so CompressionType threw InvalidFormatException for every
entry of a 7-Zip-made encrypted archive. Skip the AES filter and report
the first real coder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 13:27:20 -07:00
Jay Sanati
6d51b39230 Report the codec, not 7zAES, as an encrypted 7z entry's CompressionType
7-Zip writes 7zAES as the first coder of an encrypted folder, in front
of the codec. SevenZipFilePart.GetCompression() looked at the first
coder only, so CompressionType threw InvalidFormatException for every
entry of a 7-Zip-made encrypted archive. Skip the AES filter and report
the first real coder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 12:03:46 -07:00
Adam Hathcock
ba248dc01e Decode asynchronously buffered LZMA2 chunks with the fast CPU path 2026-10-24 13:37:15 +01:00
Adam Hathcock
31f3476fc9 Add native ValueTask memory reads through XZ filter chains 2026-10-24 13:09:51 +01:00
Adam Hathcock
de7a5d9ad4 Avoid duplicate task results in XZ BCJ array reads 2026-10-24 12:54:39 +01:00
Adam Hathcock
1fc761d5a5 Update dependencies and migrate tests to Microsoft Testing Platform 2026-10-24 12:33:38 +01:00
Adam Hathcock
5c46363f26 Pool XZ BCJ buffers and dispose owned filter chains 2026-10-24 12:24:58 +01:00
Adam Hathcock
0aaf9b92aa Fix streaming XZ BCJ decoding across read boundaries 2026-10-24 12:08:18 +01:00
Adam Hathcock
9c555c41cc Preserve legacy ZIP descriptor width probe 2026-10-24 12:02:27 +01:00
Adam Hathcock
ee30126370 Fix ZIP64 streaming data descriptor width parsing 2026-10-24 11:50:31 +01:00
dependabot[bot]
3a964e3816 Bump the all-dependencies group with 5 updates
Bumps JetBrains.Profiler.SelfApi from 2.5.18 to 2.5.19
Bumps Microsoft.NET.Test.Sdk from 18.10.1 to 18.11.0
Bumps Microsoft.SourceLink.GitHub from 10.0.401 to 10.0.402
Bumps Pure.DI from 2.5.4 to 2.5.5
Bumps System.Text.Encoding.CodePages from 10.0.12 to 10.0.13

---
updated-dependencies:
- dependency-name: JetBrains.Profiler.SelfApi
  dependency-version: 2.5.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: Microsoft.NET.Test.Sdk
  dependency-version: 18.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-dependencies
- dependency-name: Microsoft.SourceLink.GitHub
  dependency-version: 10.0.402
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: Pure.DI
  dependency-version: 2.5.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: System.Text.Encoding.CodePages
  dependency-version: 10.0.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-22 09:40:12 +00:00
Adam Hathcock
b8e81bd3f5 Split ArchiveFactory into focused public entry points 2026-10-21 11:47:47 +01:00
Adam Hathcock
ed29acda67 Remove redundant partial modifiers 2026-10-21 09:53:07 +01:00
Adam Hathcock
068da202e4 Expose default services directly through LibraryDefaults 2026-10-21 09:46:38 +01:00
Adam Hathcock
e21da75de2 Split DI services into focused workflow components 2026-10-21 09:27:47 +01:00
Adam Hathcock
4cc8d6b257 Use generated internal interfaces for DI services 2026-10-21 09:12:42 +01:00
Adam Hathcock
b578325982 Move extension workflows into internal services 2026-10-21 08:57:36 +01:00
Adam Hathcock
740ae8fa89 Refactor factory workflows into internal Pure.DI services 2026-10-21 08:42:00 +01:00
Adam Hathcock
2463488de7 Merge remote-tracking branch 'origin/master' into feature/non-seekable-zip64-writing
# Conflicts:
#	src/SharpCompress/Factories/TarFactory.cs
#	src/SharpCompress/Readers/Tar/TarReader.Factory.cs
#	src/SharpCompress/Readers/Tar/TarReader.cs
#	tests/SharpCompress.Test/Streams/RarLifetimeTests.cs
#	tests/SharpCompress.Test/Streams/SevenZipLifetimeTests.cs
#	tests/SharpCompress.Test/Streams/TarLifetimeTests.cs
2026-10-20 11:41:45 +01:00
Adam Hathcock
2d9508755e Merge remote-tracking branch 'origin/master' into adam/release-to-master 2026-10-20 11:21:00 +01:00
Adam Hathcock
714a27fd40 Fix RAR checksum handling for empty reads and checksum-free entries 2026-10-20 11:11:43 +01:00
Adam Hathcock
5efd391725 Merge remote-tracking branch 'origin/master' into perf/rar-profile-pooling 2026-10-20 11:07:23 +01:00
Adam Hathcock
bae154799d Merge pull request #1461 from adamhathcock/perf/zip-profile-pooling
Reduce Deflate read and Huffman scratch allocations
2026-10-20 11:05:49 +01:00
Adam Hathcock
c78cc86e04 Avoid temporary allocations in ring-buffered async reads 2026-10-20 10:52:15 +01:00
Adam Hathcock
f63cd81a65 Reduce allocations in encrypted RAR reading
Reuse bounded buffers and transform AES blocks in place. Remove per-block arrays and plaintext queues, avoid per-byte header allocations, and dispose owned crypto transforms.

Add encrypted RAR benchmarks, regression tests, and before/after profiling results. Full .NET 10 suite passes (2,446 tests); all library targets build and net48 tests compile.
2026-10-20 10:43:28 +01:00
Adam Hathcock
e61a335d54 Reduce allocations in RAR5 password key derivation
Reuse keyed HMAC state and digest buffers on modern .NET targets while preserving RAR5's three PBKDF2 snapshots. Add independent derivation checks and an allocation regression test.

Profiling on macOS ARM64 with .NET 10 reduced allocations by 99.96%, but showed slower throughput; Alpine-specific timing remains unverified. Full .NET 10 test suite passes.
2026-10-20 10:10:29 +01:00
Adam Hathcock
7731843e25 Reduce Deflate read and Huffman scratch allocations
Use a ValueTask read core and memory-based compressed-input refills on modern targets, replace per-block Huffman output arrays with scalar parameters, and decode directly into array-backed caller memory with a pooled fallback.

Preserve generated sync array-read semantics and clear codec output references after reads. Add slice, fallback, cancellation, failure, tree-validation, and read-ahead coverage; document staged ZIP profiles.

Validation: 2,989 .NET 10 tests passed; all library targets, net48 tests, and performance project compile; formatting checks pass.
2026-10-20 09:56:54 +01:00
Adam Hathcock
23ac770e6a Merge remote-tracking branch 'origin/master' into perf/rar-profile-pooling 2026-10-20 08:45:43 +01:00
Adam Hathcock
1ad052edcc Merge remote-tracking branch 'origin/master' into perf/7zip-profile-pooling
# Conflicts:
#	src/SharpCompress/Compressors/LZMA/Lzma2EncoderStream.cs
#	src/SharpCompress/Compressors/LZMA/LzmaStream.Async.cs
#	src/SharpCompress/IO/ReadOnlyMemoryStream.cs
2026-10-20 08:31:32 +01:00
Adam Hathcock
f41cc208cc Pool LZMA workspaces and streamline 7z metadata writing
Reuse LZMA2 trial decoders and pooled compressed output, pool and consolidate decoder probability banks, lazily rent encoder optimization scratch, and avoid copying direct async memory writes.

Use bounded stack scratch and stream 7z metadata without temporary property buffers. Add ownership, failure, context-shrink, Unicode, and block-boundary regressions.

Document allocation profiles, the small-file LZMA2 throughput tradeoff, and rejection of exact-size match-finder arrays. Validation: 3,016 .NET 10 tests passed; all library targets, net48 tests, and performance project compile; formatting checks pass.
2026-10-20 08:28:23 +01:00
Adam Hathcock
030569611e Reduce 7z copy and LZMA decoder allocations
Pool writer CRC-copy buffers, use memory-based async read overloads on modern targets, reuse LZMA2 decoders across resets, and initialize probability-model banks lazily.

Add reset/property-change, memory-read, writer boundary/CRC, and growth regression coverage. Document staged profiling results and rejection of dictionary-growth policies that regress small-file allocations.

Validation: 2,997 .NET 10 Release tests passed, all library targets built, net48 tests compiled, and formatting checks passed.
2026-10-19 15:11:47 +01:00
Adam Hathcock
14d77b671c Release pooled 7z decoder buffers across async and partial reads
Dispose cached folder decoders during async archive cleanup and scope temporary encoded-header decoders in both read paths. Release LZMA range-decoder input buffers on partial reads and failures while preserving borrowed source ownership.

Correct the synchronous extraction-reader benchmark and expand lifetime and checksum coverage. Document profiling showing async Archive allocations falling from 8.95 MiB to 0.70 MiB per extraction, with metadata-only allocations reduced by 70%.

Validation: 2,989 .NET 10 Release tests passed; all library targets, the performance project, and net48 test compilation succeeded. Formatting and whitespace checks passed.
2026-10-19 14:43:51 +01:00
Adam Hathcock
81879e9de4 Pool RAR Huffman scratch buffers and validate async checksums
Reuse unpacker-owned pooled scratch arrays for async RAR5 and RAR2 Huffman tables, preserving alphabet bounds and zeroed history tails. Match async Archive CRC32/BLAKE2sp verification to the synchronous path and dispose entry streams when initialization fails.

Add regression tests and profiling documentation showing approximately 3.13 MiB fewer managed allocations per 200 MiB extraction. Keep this change based directly on master without unrelated branch commits.

Validation: 2,983 .NET 10 Release tests passed, all six library targets built, net48 tests compiled, and formatting checks passed.
2026-10-19 14:10:30 +01:00
Adam Hathcock
78006f94f7 More pooled or readonly memory stream usage 2026-10-19 12:07:29 +01:00
Adam Hathcock
1414e17f6f Avoid copying pooled 7z headers when computing async CRCs 2026-10-19 12:01:46 +01:00
Adam Hathcock
700ae66e0a Fix usage of DisposeAsyncScope 2026-10-19 11:49:08 +01:00
Adam Hathcock
0fbf36d602 Merge remote-tracking branch 'origin/release'
# Conflicts:
#	src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs
#	src/SharpCompress/Compressors/LZMA/LzmaStream.cs
2026-10-19 11:15:09 +01:00
Adam Hathcock
2168385f2e fixes for release 2026-10-19 10:52:07 +01:00
Adam Hathcock
560a148ddb Merge pull request #1455 from romaklayt/perf/source-stream-cache-length
Read each SourceStream part's length once
2026-10-19 10:36:22 +01:00
Adam Hathcock
999676d9c9 Merge pull request #1457 from romaklayt/perf/zip-winzip-aes-reader
Speed up reading WinZip AES zip entries
2026-10-19 10:34:55 +01:00
Adam Hathcock
7642f76f38 changes fixes to use pooled arrays 2026-10-19 10:10:09 +01:00
Adam Hathcock
4966fc186b Merge branch 'release' into fix/lzma2-truncated-header
# Conflicts:
#	src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs
2026-10-19 09:59:33 +01:00
Adam Hathcock
c6d8c76b14 Review changes around disposal 2026-10-19 09:43:44 +01:00