2863 Commits

Author SHA1 Message Date
Adam Hathcock
35aa0ca4ad Merge pull request #1362 from LANCommander/fix-zip64-streaming-corrupted-entry-sizes
Zip: fix Zip64 streaming reader corrupting entry size/CRC and failing on non-seekable streams
2026-07-02 11:44:20 +01:00
Pat Hartl
c59aa78928 Zip: fix Zip64 streaming reader corrupting entry size/CRC and failing on non-seekable streams
The Zip64 branch of the streaming header reader assumed a data descriptor always follows a >=4GB entry. When the entry instead has back-patched sizes (no descriptor), the following header was parsed as descriptor fields, overwriting the entry's correct size/CRC with central-directory 0xFFFFFFFF sentinels and, on non-seekable async streams, leaving the reader misaligned so the next entry threw.

Now it detects a header signature after the entry data and leaves the already-correct metadata untouched, parsing that header normally. The fix has been applied to both the sync and async readers.
2026-06-29 21:35:55 -05:00
Adam Hathcock
0e20a10961 Merge pull request #1358 from fiddyschmitt/tolerate-truncated-bzip2-stream
BZip2: add opt-in tolerateTruncatedStream to decode a footerless/partial stream
2026-06-23 12:04:30 +01:00
Fidel
de8ee30b17 BZip2: add opt-in tolerateTruncatedStream to decode a footerless/partial stream
Add a `tolerateTruncatedStream` option (default false) to `BZip2Stream.Create`/`CreateAsync`,
threaded through to `CBZip2InputStream`. When enabled, the decoder accepts a stream that has no
trailing footer - e.g. a truncated stream, or a sub-range of blocks extracted for random access:

- An end-of-input reached while reading a block/footer header (a true block boundary, tracked by
  `expectingBlockStart`) is treated as a normal end of stream instead of throwing
  `ArchiveOperationException("BZip2 compressed file ends unexpectedly")`. EOF in the middle of a
  block, the block CRC, or the Huffman tables still throws.
- The whole-stream combined CRC in the footer is not verified, since a partial decode's running
  combined CRC won't match the stored whole-stream value. Per-block CRCs are still enforced.

Default behaviour is unchanged (the flag defaults to false) and the change is applied symmetrically
to the sync and async read paths.

Tests (BZip2StreamTests):
- a footerless header-only stream decodes to empty with the flag and throws without it;
- a real block followed by end-of-input at the next block boundary decodes with the flag and throws
  without it;
- a corrupted whole-stream combined CRC is tolerated with the flag and fatal without it;
- a complete, well-formed stream still round-trips with the flag set.
All existing BZip2 tests continue to pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 13:08:02 +10:00
Adam Hathcock
37a22a8697 Merge pull request #1357 from adamhathcock/adam/crc-impl
more complete CRC checking
2026-06-19 11:58:58 +01:00
Adam Hathcock
49d00034e4 update tests 2026-06-19 11:21:39 +01:00
Adam Hathcock
727d2a79f9 updates from review 2026-06-18 17:06:16 +01:00
Adam Hathcock
e8ee054757 rest of the formats 2026-06-15 17:02:31 +01:00
Adam Hathcock
808524d917 BZip2, GZip and XZ done 2026-06-15 09:59:48 +01:00
Adam Hathcock
e9e05520d6 7Zip and Rar 2026-06-15 09:29:50 +01:00
Adam Hathcock
2866bfbd54 First pass with zip 2026-06-15 09:17:05 +01:00
Adam Hathcock
c36a916c84 add XZ LZMA skill 2026-06-13 15:11:52 +01:00
Adam Hathcock
78a2218982 Handle lzma end marker correctly 2026-06-13 15:09:13 +01:00
Adam Hathcock
2d041e5c76 Fixed XZ block CRC64 computation and added notes on XZ format specifics 2026-06-13 14:31:36 +01:00
Adam Hathcock
160ba0938a review fixes 2026-06-13 10:00:39 +01:00
Adam Hathcock
7814013995 Merge remote-tracking branch 'origin/master' into adam/xz-crc-check 2026-06-13 09:40:39 +01:00
Adam Hathcock
668d5999f0 Merge pull request #1354 from adamhathcock/adam/instance-buffer-size
Instanced buffer size
2026-06-13 09:40:18 +01:00
Adam Hathcock
59d6b82ac5 Cleanup: use some arraypool buffers 2026-06-13 09:37:21 +01:00
Adam Hathcock
4e3dcabbd0 Add basic crc checking for xz 2026-06-13 09:24:04 +01:00
copilot-swe-agent[bot]
323831cfa7 chore: update NuGet lock files to match CI SDK versions (ILLink.Tasks 8.0.27, 10.0.8) 2026-06-12 14:14:22 +00:00
Adam Hathcock
9684e53084 Merge remote-tracking branch 'origin/adam/instance-buffer-size' into adam/instance-buffer-size 2026-06-12 15:10:56 +01:00
Adam Hathcock
68a74e0f45 Try to make things an exact match 2026-06-12 15:10:15 +01:00
Adam Hathcock
61ae3f21c7 fix package lock 2026-06-12 15:01:10 +01:00
Adam Hathcock
0fdd9e6fc6 update csharpier 2026-06-12 14:54:14 +01:00
Adam Hathcock
21b9c33c95 Fix tests 2026-06-12 14:51:56 +01:00
Adam Hathcock
de6e2bfee2 Add WriterOptions.BufferSize property 2026-06-12 14:41:05 +01:00
Adam Hathcock
3a73cfe925 Add instance based extraction size instead of just static 2026-06-11 17:24:42 +01:00
Adam Hathcock
5de1193aab Merge pull request #1349 from adamhathcock/adam/update-docs
updates docs for latest changes and ArchiveInfo
2026-06-07 11:26:25 +01:00
Adam Hathcock
31fcb8526c add note about providers 2026-06-07 11:20:34 +01:00
Adam Hathcock
2c9f7650a0 updates docs for latest changes and ArchiveInfo 2026-06-06 15:50:49 +01:00
Adam Hathcock
cd55ed2744 Merge pull request #1347 from adamhathcock/adam/deflate-allocations
Add pooling for arrays for deflate
2026-06-04 14:50:20 +01:00
Adam Hathcock
27cc668147 Merge pull request #1346 from adamhathcock/adam/gzip-leave-open
LeaveOpen wasn't respected by GZip
2026-06-02 14:59:23 +01:00
Adam Hathcock
b4b13b7c08 Add pooling for arrays for deflate 2026-06-02 14:02:26 +01:00
Adam Hathcock
a14a5c7afd Really fix constructor 2026-06-02 13:47:14 +01:00
Adam Hathcock
4034a5471b LeaveOpen wasn't respected by GZip 2026-06-02 13:42:40 +01:00
Adam Hathcock
051b302def Merge pull request #1336 from adamhathcock/adam/rar-allocations
Reduce rar allocations
2026-06-02 13:29:58 +01:00
Adam Hathcock
4c6f773d2b more allocation reductions 2026-06-02 13:15:16 +01:00
Adam Hathcock
dbfe9a1d61 Merge remote-tracking branch 'origin/master' into adam/rar-allocations 2026-06-02 10:37:12 +01:00
Adam Hathcock
aec708361a Merge pull request #1345 from adamhathcock/adam/minor-updates
Minor updates
2026-06-02 09:27:10 +01:00
Adam Hathcock
5bfb67e177 Update deps 2026-06-02 09:13:00 +01:00
Adam Hathcock
fbfbd81fe4 fmt 2026-06-02 09:07:57 +01:00
Adam Hathcock
f2c42487f3 use slnx 2026-06-02 09:06:18 +01:00
Adam Hathcock
697703c4b6 dependabot updates 2026-06-02 09:06:09 +01:00
Adam Hathcock
22413268ec Merge pull request #1335 from adamhathcock/adam/lzma-allocations
Reduce LZMA Allocations
2026-06-01 15:51:04 +01:00
copilot-swe-agent[bot]
4d89d856b1 Regenerate AotSmoke packages.lock.json with net10.0/linux-x64 section 2026-06-01 13:27:30 +00:00
Adam Hathcock
98b9d86a5c Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-01 14:25:38 +01:00
Adam Hathcock
07319f61bf Minor changes 2026-06-01 13:41:17 +01:00
Adam Hathcock
c83ab22098 Merge remote-tracking branch 'origin/adam/lzma-allocations' into adam/lzma-allocations 2026-06-01 13:40:24 +01:00
copilot-swe-agent[bot]
4ed093a492 Finalize review comment fix 2026-06-01 12:37:31 +00:00
copilot-swe-agent[bot]
02c6123825 Use fixed LZMA2 chunk capacity in write buffering 2026-06-01 12:34:27 +00:00