mirror of
https://github.com/adamhathcock/sharpcompress.git
synced 2026-09-23 23:45:14 +00:00
Merge pull request #1288 from adamhathcock/adam/tar-pax
update docs for tar gap analysis and XZ usage
This commit is contained in:
Adam Hathcock
GitHub
@@ -20,7 +20,7 @@
|
||||
| Tar.BZip2 | BZip2 | Both | TarArchive | TarReader | TarWriter (3) |
|
||||
| Tar.Zstandard | ZStandard | Decompress | TarArchive | TarReader | N/A |
|
||||
| Tar.LZip | LZMA | Both | TarArchive | TarReader | TarWriter (3) |
|
||||
| Tar.XZ | LZMA2 | Decompress | TarArchive | TarReader | TarWriter (3) |
|
||||
| Tar.XZ | LZMA2 | Decompress | TarArchive | TarReader | N/A |
|
||||
| GZip (single file) | DEFLATE | Both | GZipArchive | GZipReader | GZipWriter |
|
||||
| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Both | SevenZipArchive | N/A | SevenZipWriter |
|
||||
|
||||
|
||||
340
docs/TAR_GAP_ANALYSIS.md
Normal file
340
docs/TAR_GAP_ANALYSIS.md
Normal file
@@ -0,0 +1,340 @@
|
||||
# Tar Gap Analysis
|
||||
|
||||
## Scope
|
||||
|
||||
This document compares the current Tar documentation, tests, and code paths in SharpCompress.
|
||||
|
||||
It is intentionally implementation-focused. The goal is to identify mismatches, omissions, and incomplete areas in the current SharpCompress Tar support.
|
||||
|
||||
Primary references:
|
||||
|
||||
- `docs/FORMATS.md`
|
||||
- `src/SharpCompress/Factories/TarFactory.cs`
|
||||
- `src/SharpCompress/Factories/TarWrapper.cs`
|
||||
- `src/SharpCompress/Archives/Tar/`
|
||||
- `src/SharpCompress/Readers/Tar/`
|
||||
- `src/SharpCompress/Writers/Tar/`
|
||||
- `src/SharpCompress/Common/Tar/`
|
||||
- `tests/SharpCompress.Test/Tar/`
|
||||
|
||||
## Claimed vs Actual Support
|
||||
|
||||
### `Tar.XZ` is read-only
|
||||
|
||||
`tar.xz` is supported for reading, but not for writing.
|
||||
|
||||
Actual implementation in `src/SharpCompress/Writers/Tar/TarWriter.cs` does not support `CompressionType.Xz`. The writer throws `InvalidFormatException` for any compression type outside:
|
||||
|
||||
- `None`
|
||||
- `GZip`
|
||||
- `BZip2`
|
||||
- `LZip`
|
||||
|
||||
Impact:
|
||||
|
||||
- Tar write support is narrower than Tar read support
|
||||
- `tar.xz` creation is not available through the built-in Tar writer
|
||||
|
||||
Recommended action:
|
||||
|
||||
- keep the format table marked `N/A` for Tar.XZ writer support
|
||||
|
||||
## Read-Path Gaps
|
||||
|
||||
### PAX headers are not implemented
|
||||
|
||||
There is no explicit support for POSIX PAX local extended headers.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `EntryType` does not define the usual local PAX header type value
|
||||
- `TarHeader.Read` handles `LongName` and `LongLink`, but does not implement PAX record parsing
|
||||
- there are no tests or test archives covering PAX behavior
|
||||
|
||||
Impact:
|
||||
|
||||
- archives relying on PAX for long names, metadata, or timestamps may not be interpreted correctly
|
||||
|
||||
Recommended action:
|
||||
|
||||
- decide whether PAX is intentionally unsupported or should be implemented
|
||||
- document that decision explicitly
|
||||
|
||||
### Sparse files are not semantically implemented
|
||||
|
||||
`EntryType` defines `SparseFile`, but the read path does not contain sparse map handling or sparse reconstruction logic.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/SharpCompress/Common/Tar/Headers/EntryType.cs`
|
||||
- no sparse-specific code in `TarHeader`, `TarEntry`, `TarFilePart`, or `TarArchive`
|
||||
- no sparse tests
|
||||
|
||||
Impact:
|
||||
|
||||
- sparse entries may be treated as ordinary entries rather than sparse files with holes
|
||||
|
||||
Recommended action:
|
||||
|
||||
- document sparse support as unsupported or partial
|
||||
- add explicit tests if future support is added
|
||||
|
||||
### Global extended headers are not semantically implemented
|
||||
|
||||
`EntryType` defines `GlobalExtendedHeader`, but no semantic handling exists in the read pipeline.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `TarHeader.Read` does not special-case `GlobalExtendedHeader`
|
||||
- `TarEntry` does not surface a global-header model
|
||||
- no tests cover this case
|
||||
|
||||
Impact:
|
||||
|
||||
- global metadata records are not applied in a defined way
|
||||
|
||||
Recommended action:
|
||||
|
||||
- document as unsupported until explicit behavior exists
|
||||
|
||||
### Device and FIFO semantics are not surfaced
|
||||
|
||||
The entry type enum includes `CharDevice`, `BlockDevice`, and `Fifo`, but the public tar model does not expose device metadata semantics.
|
||||
|
||||
Impact:
|
||||
|
||||
- such entries may not round-trip meaningfully through the API
|
||||
- behavior is undocumented and untested
|
||||
|
||||
Recommended action:
|
||||
|
||||
- either document them as raw/unmodeled entry types or add dedicated support
|
||||
|
||||
## Write-Path Gaps
|
||||
|
||||
### `HeaderFormat` is not honored consistently
|
||||
|
||||
`TarWriterOptions.HeaderFormat` exists and defaults to `GNU_TAR_LONG_LINK`, but the configured value is not consistently applied.
|
||||
|
||||
### Sync directory write path
|
||||
|
||||
`TarWriter.WriteDirectory` creates headers using:
|
||||
|
||||
- `new TarHeader(WriterOptions.ArchiveEncoding)`
|
||||
|
||||
This uses the default tar header format rather than the writer's configured `headerFormat` field.
|
||||
|
||||
Impact:
|
||||
|
||||
- directory entries written through the sync path do not follow `TarWriterOptions.HeaderFormat`
|
||||
|
||||
### Async write path
|
||||
|
||||
`TarWriter.WriteAsync` and `WriteDirectoryAsync` also create headers using the default constructor rather than the configured header format.
|
||||
|
||||
Impact:
|
||||
|
||||
- async writes ignore `TarWriterOptions.HeaderFormat` for both file and directory entries
|
||||
|
||||
Recommended action:
|
||||
|
||||
- pass the configured header format to all `TarHeader` constructions in sync and async write paths
|
||||
- add tests for both `GNU_TAR_LONG_LINK` and `USTAR`
|
||||
|
||||
### No public link-writing support
|
||||
|
||||
The read path supports symbolic and hard link targets through `TarEntry.LinkTarget`, but the write API exposes only regular file and directory creation.
|
||||
|
||||
Impact:
|
||||
|
||||
- symlink and hardlink tar archives cannot be created through the current public Tar writer API
|
||||
|
||||
Recommended action:
|
||||
|
||||
- either document this as a deliberate limitation or add link-writing APIs
|
||||
|
||||
### Metadata round-trip support is incomplete
|
||||
|
||||
The writer does not round-trip rich tar metadata beyond the basic fields needed for file and directory entries.
|
||||
|
||||
Current write behavior sets fixed defaults for some fields such as mode, owner id, and group id.
|
||||
|
||||
Impact:
|
||||
|
||||
- modified or newly created tar archives may lose metadata fidelity relative to the original archive
|
||||
|
||||
Recommended action:
|
||||
|
||||
- document current metadata write behavior clearly
|
||||
- expand metadata support only if needed by consumers
|
||||
|
||||
### No write support for some detected wrappers
|
||||
|
||||
The detection and read path supports wrappers that the write path does not support.
|
||||
|
||||
| Wrapper | Read support | Write support |
|
||||
| ------- | ------------ | ------------- |
|
||||
| `tar.xz` | Yes | No |
|
||||
| `tar.zst` | Yes | No |
|
||||
| `tar.Z` | Yes | No |
|
||||
|
||||
This is not inherently wrong, but it should be clearly documented everywhere support is summarized.
|
||||
|
||||
## Sync and Async API Inconsistencies
|
||||
|
||||
### Seekability requirements differ at the API boundary
|
||||
|
||||
Synchronous `TarArchive.OpenArchive(Stream)` explicitly throws if the stream is not seekable.
|
||||
|
||||
Asynchronous `TarArchive.OpenAsyncArchive(Stream)` does not perform the same public guard.
|
||||
|
||||
Impact:
|
||||
|
||||
- callers do not see the same contract from sync and async overloads
|
||||
- behavior is harder to reason about from API docs alone
|
||||
|
||||
Recommended action:
|
||||
|
||||
- either align the contracts or document the difference explicitly
|
||||
|
||||
### Async and sync write behavior do not align on header format handling
|
||||
|
||||
This is the most visible sync/async inconsistency in the current Tar writer implementation.
|
||||
|
||||
Recommended action:
|
||||
|
||||
- fix the implementation first
|
||||
- add matching sync and async tests to keep the behavior aligned
|
||||
|
||||
## Test Coverage Gaps
|
||||
|
||||
### Symlink coverage exists in test data but not in assertions
|
||||
|
||||
There is a tar archive containing symlinks:
|
||||
|
||||
- `tests/TestArchives/Archives/TarWithSymlink.tar.gz`
|
||||
|
||||
Current Tar tests do not assert tar symlink behavior against that fixture.
|
||||
|
||||
Impact:
|
||||
|
||||
- the code claims practical read support for link targets, but coverage does not verify it
|
||||
|
||||
Recommended action:
|
||||
|
||||
- add reader and archive tests asserting `EntryType`-derived behavior and `LinkTarget`
|
||||
|
||||
### No tests for `HeaderFormat`
|
||||
|
||||
There are currently no tests covering:
|
||||
|
||||
- `TarWriterOptions.HeaderFormat = USTAR`
|
||||
- `TarWriterOptions.HeaderFormat = GNU_TAR_LONG_LINK`
|
||||
- long-name failures in USTAR mode
|
||||
- long-name success in GNU mode through the async writer path
|
||||
|
||||
Impact:
|
||||
|
||||
- the current header-format regressions were able to exist without test coverage
|
||||
|
||||
Recommended action:
|
||||
|
||||
- add dedicated sync and async writer tests for header format selection
|
||||
|
||||
### No tests for PAX, sparse, or global headers
|
||||
|
||||
There is no evidence of coverage for:
|
||||
|
||||
- PAX local headers
|
||||
- global extended headers
|
||||
- sparse tar entries
|
||||
|
||||
Impact:
|
||||
|
||||
- unsupported or partial behavior is neither documented by tests nor protected from regression
|
||||
|
||||
Recommended action:
|
||||
|
||||
- either add fixtures and tests or document these as unsupported with no test coverage
|
||||
|
||||
### No tests for unsupported write wrappers
|
||||
|
||||
There are negative tests for an invalid `Rar` compression type, but not for unsupported tar wrappers that a user might reasonably infer from read support.
|
||||
|
||||
Missing negative cases include:
|
||||
|
||||
- `CompressionType.Xz`
|
||||
- `CompressionType.ZStandard`
|
||||
- `CompressionType.Lzw`
|
||||
|
||||
Recommended action:
|
||||
|
||||
- add explicit negative tests so the supported write matrix stays intentional
|
||||
|
||||
## Documentation Gaps
|
||||
|
||||
### Current format documentation is too coarse for Tar
|
||||
|
||||
`docs/FORMATS.md` summarizes support at the wrapper level, but Tar behavior depends on more than wrapper compression.
|
||||
|
||||
Missing implementation-specific details include:
|
||||
|
||||
- GNU long-name and long-link support
|
||||
- USTAR prefix handling
|
||||
- oldgnu numeric quirk handling
|
||||
- missing PAX support
|
||||
- missing sparse support
|
||||
- reader vs archive behavior differences for compressed tar
|
||||
- file-size requirements for writing from non-seekable sources
|
||||
|
||||
Recommended action:
|
||||
|
||||
- keep `docs/FORMATS.md` high-level
|
||||
- add and maintain a dedicated Tar spec document for details
|
||||
|
||||
### The current docs do not call out partial support clearly
|
||||
|
||||
The codebase supports some tar dialect features and not others, but the docs do not separate:
|
||||
|
||||
- fully supported
|
||||
- partially supported
|
||||
- unsupported
|
||||
|
||||
Recommended action:
|
||||
|
||||
- use an explicit feature matrix in the Tar documentation
|
||||
|
||||
## Recommended Follow-Ups
|
||||
|
||||
### Priority 0
|
||||
|
||||
- Correct `docs/FORMATS.md` for `Tar.XZ` write support
|
||||
|
||||
### Priority 1
|
||||
|
||||
- Fix `TarWriterOptions.HeaderFormat` handling in sync and async writer paths
|
||||
- Add tests for header-format behavior
|
||||
- Add symlink coverage using `TarWithSymlink.tar.gz`
|
||||
|
||||
### Priority 2
|
||||
|
||||
- Decide and document the support position for PAX headers
|
||||
- Decide and document the support position for sparse files
|
||||
- Decide and document the support position for global extended headers
|
||||
|
||||
### Priority 3
|
||||
|
||||
- Add negative writer tests for unsupported wrapper compressions
|
||||
- Evaluate whether sync and async archive open contracts should match exactly
|
||||
- Improve metadata round-trip behavior only if there is a consumer need
|
||||
|
||||
## Summary
|
||||
|
||||
The SharpCompress Tar implementation is strong on common read scenarios and basic write scenarios, but the current gaps fall into four categories:
|
||||
|
||||
- documentation overstating or under-describing support
|
||||
- incomplete feature coverage for less common tar dialect features
|
||||
- sync/async and file/directory inconsistencies in writer header-format handling
|
||||
- test coverage holes around links and advanced tar metadata features
|
||||
|
||||
`docs/TAR_SPEC.md` should be treated as the implementation baseline. This document identifies where that baseline is incomplete, inconsistent, or incorrectly reflected elsewhere in the repository.
|
||||
430
docs/TAR_SPEC.md
Normal file
430
docs/TAR_SPEC.md
Normal file
@@ -0,0 +1,430 @@
|
||||
# Tar Spec
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the Tar implementation that exists in SharpCompress today.
|
||||
|
||||
It is intentionally SharpCompress-specific. It documents actual behavior in the current codebase, including partial support and limitations. It is not a general tar format reference.
|
||||
|
||||
Primary implementation files:
|
||||
|
||||
- `src/SharpCompress/Factories/TarFactory.cs`
|
||||
- `src/SharpCompress/Factories/TarWrapper.cs`
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.cs`
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.Async.cs`
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.Factory.cs`
|
||||
- `src/SharpCompress/Readers/Tar/TarReader.cs`
|
||||
- `src/SharpCompress/Readers/Tar/TarReader.Async.cs`
|
||||
- `src/SharpCompress/Writers/Tar/TarWriter.cs`
|
||||
- `src/SharpCompress/Writers/Tar/TarWriter.Async.cs`
|
||||
- `src/SharpCompress/Writers/Tar/TarWriterOptions.cs`
|
||||
- `src/SharpCompress/Common/Tar/Headers/TarHeader.cs`
|
||||
- `src/SharpCompress/Common/Tar/Headers/TarHeader.Async.cs`
|
||||
- `src/SharpCompress/Common/Tar/TarHeaderFactory.cs`
|
||||
- `src/SharpCompress/Common/Tar/TarHeaderFactory.Async.cs`
|
||||
|
||||
## API Surface
|
||||
|
||||
SharpCompress exposes Tar support through four main entry points.
|
||||
|
||||
| Type | Role |
|
||||
| ---- | ---- |
|
||||
| `TarFactory` | Format detection and factory entry point for archive, reader, and writer APIs |
|
||||
| `TarArchive` | Archive API for enumerating and rewriting tar archives |
|
||||
| `TarReader` | Forward-only reader API for streaming tar extraction |
|
||||
| `TarWriter` | Forward-only writer API for creating tar archives |
|
||||
|
||||
`TarWriterOptions` controls output compression, stream ownership, archive finalization, encoding, and header write format.
|
||||
|
||||
## Supported Wrapper Formats
|
||||
|
||||
Tar wrapper detection is defined by `TarWrapper.Wrappers` in `src/SharpCompress/Factories/TarWrapper.cs`.
|
||||
|
||||
### Supported Extensions
|
||||
|
||||
| Wrapper | Extensions |
|
||||
| ------- | ---------- |
|
||||
| Plain tar | `tar` |
|
||||
| Tar + BZip2 | `tar.bz2`, `tb2`, `tbz`, `tbz2`, `tz2` |
|
||||
| Tar + GZip | `tar.gz`, `taz`, `tgz` |
|
||||
| Tar + ZStandard | `tar.zst`, `tar.zstd`, `tzst`, `tzstd` |
|
||||
| Tar + LZip | `tar.lz` |
|
||||
| Tar + XZ | `tar.xz`, `txz` |
|
||||
| Tar + LZW compress | `tar.Z`, `tZ`, `taZ` |
|
||||
|
||||
### API Support Matrix
|
||||
|
||||
| Wrapper | Detection | `TarArchive` read | `TarReader` read | `TarWriter` write |
|
||||
| ------- | --------- | ----------------- | ---------------- | ----------------- |
|
||||
| Plain tar | Yes | Yes | Yes | Yes |
|
||||
| Tar + GZip | Yes | Yes | Yes | Yes |
|
||||
| Tar + BZip2 | Yes | Yes | Yes | Yes |
|
||||
| Tar + LZip | Yes | Yes | Yes | Yes |
|
||||
| Tar + XZ | Yes | Yes | Yes | No |
|
||||
| Tar + ZStandard | Yes | Yes | Yes | No |
|
||||
| Tar + LZW compress | Yes | Yes | Yes | No |
|
||||
|
||||
Write support is implemented in `src/SharpCompress/Writers/Tar/TarWriter.cs` and currently accepts only `CompressionType.None`, `CompressionType.GZip`, `CompressionType.BZip2`, and `CompressionType.LZip`.
|
||||
|
||||
## Detection Behavior
|
||||
|
||||
Tar detection is implemented in `TarFactory.IsArchive`, `TarFactory.IsArchiveAsync`, `TarFactory.GetCompressionType`, and `TarFactory.GetCompressionTypeAsync`.
|
||||
|
||||
Detection behavior is:
|
||||
|
||||
1. Wrap the incoming stream in `SharpCompressStream`.
|
||||
2. Start recording with a rewind buffer sized from `TarWrapper.MaximumRewindBufferSize`.
|
||||
3. Probe each registered wrapper in order.
|
||||
4. If a wrapper matches, create a decompression stream for that wrapper.
|
||||
5. Call `TarArchive.IsTarFile` or `TarArchive.IsTarFileAsync` on the decompressed stream.
|
||||
6. If the tar probe succeeds, treat the stream as tar with that wrapper compression.
|
||||
|
||||
Implications:
|
||||
|
||||
- Tar detection is content-based, not extension-based.
|
||||
- Wrapper detection is not sufficient by itself. The decompressed payload must also parse as tar.
|
||||
- Non-seekable detection is supported through the recording and rewind mechanism.
|
||||
- The largest rewind requirement currently comes from BZip2, which declares a larger minimum probe buffer in `TarWrapper`.
|
||||
|
||||
`TarArchive.IsTarFile` and `TarArchive.IsTarFileAsync` attempt to read a single tar header and return `false` on any exception. They also treat an all-zero empty archive block as a valid empty tar archive when the entry type is defined.
|
||||
|
||||
## Reader Behavior
|
||||
|
||||
`TarReader` is the forward-only streaming API.
|
||||
|
||||
Implementation files:
|
||||
|
||||
- `src/SharpCompress/Readers/Tar/TarReader.cs`
|
||||
- `src/SharpCompress/Readers/Tar/TarReader.Async.cs`
|
||||
- `src/SharpCompress/Common/Tar/TarEntry.cs`
|
||||
- `src/SharpCompress/Common/Tar/TarEntry.Async.cs`
|
||||
- `src/SharpCompress/Common/Tar/TarReadOnlySubStream.cs`
|
||||
|
||||
Reader behavior:
|
||||
|
||||
- The reader always enumerates entries in streaming mode.
|
||||
- It works with non-seekable input streams.
|
||||
- It applies decompression based on the detected wrapper compression type before parsing tar headers.
|
||||
- Entry streams are backed by `TarReadOnlySubStream`.
|
||||
|
||||
`TarReadOnlySubStream` has an important behavior: disposing an entry stream consumes any unread entry bytes and any required 512-byte padding so that the next header can be read correctly. This is what makes skipping entries work in streaming mode.
|
||||
|
||||
### Reader Compression Mapping
|
||||
|
||||
`TarReader.RequestInitialStream` and `RequestInitialStreamAsync` map the detected wrapper to the corresponding decompression stream:
|
||||
|
||||
- `None`
|
||||
- `BZip2`
|
||||
- `GZip`
|
||||
- `ZStandard`
|
||||
- `LZip`
|
||||
- `Xz`
|
||||
- `Lzw`
|
||||
|
||||
### Reader Entry Semantics
|
||||
|
||||
For each entry, SharpCompress exposes:
|
||||
|
||||
- `Key` from the parsed tar name
|
||||
- `LinkTarget` for symbolic and hard links
|
||||
- `Size`
|
||||
- `CompressedSize`
|
||||
- `LastModifiedTime`
|
||||
- `IsDirectory`
|
||||
- `Mode`
|
||||
- `UserID`
|
||||
- `GroupId`
|
||||
|
||||
Tar entries are always reported as unencrypted and CRC is always `0`.
|
||||
|
||||
## Archive Behavior
|
||||
|
||||
`TarArchive` is the archive API.
|
||||
|
||||
Implementation files:
|
||||
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.cs`
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.Async.cs`
|
||||
- `src/SharpCompress/Archives/Tar/TarArchive.Factory.cs`
|
||||
|
||||
### Open Behavior
|
||||
|
||||
Synchronous `TarArchive.OpenArchive(Stream)` requires a seekable stream and throws `ArgumentException` when `CanSeek` is `false`.
|
||||
|
||||
`TarArchive.OpenArchive(FileInfo)` and the list-based overloads use `SourceStream` and determine wrapper compression by calling `TarFactory.GetCompressionType`.
|
||||
|
||||
Asynchronous `OpenAsyncArchive` overloads use `TarFactory.GetCompressionTypeAsync` and do not enforce the same explicit seekability check at the public API boundary.
|
||||
|
||||
### Entry Loading
|
||||
|
||||
`TarArchive.LoadEntries` and `LoadEntriesAsync` parse entries differently depending on wrapper compression:
|
||||
|
||||
- Uncompressed tar uses `StreamingMode.Seekable`.
|
||||
- Wrapped tar uses `StreamingMode.Streaming` because the decompressed stream is not treated as random-access.
|
||||
|
||||
When seekable mode is used, the header stores `DataStartPosition`, and entries reopen data through `TarFilePart` by seeking back to the data position.
|
||||
|
||||
When streaming mode is used, the header stores a `PackedStream`, and entry access follows streaming semantics over the decompressed stream.
|
||||
|
||||
### Archive Rewrite Behavior
|
||||
|
||||
`TarArchive` supports creating and modifying archives through `AbstractWritableArchive`:
|
||||
|
||||
- add file entries
|
||||
- add directory entries
|
||||
- remove entries
|
||||
- save to a new stream or path
|
||||
|
||||
Archive rewrite is implemented by enumerating the existing and new entries and writing them back out through `TarWriter`.
|
||||
|
||||
## Writer Behavior
|
||||
|
||||
`TarWriter` is the forward-only tar writer.
|
||||
|
||||
Implementation files:
|
||||
|
||||
- `src/SharpCompress/Writers/Tar/TarWriter.cs`
|
||||
- `src/SharpCompress/Writers/Tar/TarWriter.Async.cs`
|
||||
- `src/SharpCompress/Writers/Tar/TarWriterOptions.cs`
|
||||
|
||||
### Supported Output Compression
|
||||
|
||||
The writer supports these output compression types:
|
||||
|
||||
- `CompressionType.None`
|
||||
- `CompressionType.GZip`
|
||||
- `CompressionType.BZip2`
|
||||
- `CompressionType.LZip`
|
||||
|
||||
Any other compression type causes `InvalidFormatException`.
|
||||
|
||||
### Stream Ownership
|
||||
|
||||
If `LeaveStreamOpen` is `true`, `TarWriter` wraps the destination in a non-disposing stream.
|
||||
|
||||
### File Writing
|
||||
|
||||
`TarWriter.Write` and `WriteAsync` write a tar header followed by file contents, then pad the payload to the next 512-byte boundary.
|
||||
|
||||
If the source stream is non-seekable and the caller does not supply `size`, the writer throws `ArgumentException` because tar requires the file size in the header.
|
||||
|
||||
### Directory Writing
|
||||
|
||||
`WriteDirectory` and `WriteDirectoryAsync` normalize the directory name to use forward slashes and ensure the key ends with `/`.
|
||||
|
||||
Empty or root-equivalent directory names are skipped.
|
||||
|
||||
### Archive Finalization
|
||||
|
||||
If `FinalizeArchiveOnClose` is `true`, disposing the writer writes two 512-byte zero blocks to terminate the archive.
|
||||
|
||||
If the output stream implements `IFinishable`, dispose also calls `Finish()`.
|
||||
|
||||
## Header Write Formats
|
||||
|
||||
`TarHeaderWriteFormat` is defined in `src/SharpCompress/Common/Tar/Headers/TarHeaderWriteFormat.cs`.
|
||||
|
||||
Supported write formats:
|
||||
|
||||
- `GNU_TAR_LONG_LINK`
|
||||
- `USTAR`
|
||||
|
||||
`TarWriterOptions.HeaderFormat` defaults to `GNU_TAR_LONG_LINK`.
|
||||
|
||||
Current implementation behavior is narrower than the option surface suggests:
|
||||
|
||||
- sync file writes use the configured `HeaderFormat`
|
||||
- sync directory writes currently construct the default tar header format
|
||||
- async file writes currently construct the default tar header format
|
||||
- async directory writes currently construct the default tar header format
|
||||
|
||||
In practice, this means the configured `HeaderFormat` is currently honored only by the synchronous file write path.
|
||||
|
||||
### GNU Long Name Write Behavior
|
||||
|
||||
In GNU mode, when a file name exceeds the 100-byte field, `TarHeader.WriteGnuTarLongLink` writes a synthetic long-name header using `././@LongLink` and `EntryType.LongName`, then writes the long name payload, and finally writes the actual file entry.
|
||||
|
||||
GNU mode also writes large file sizes using binary size encoding when the size does not fit the standard octal field.
|
||||
|
||||
### USTAR Write Behavior
|
||||
|
||||
When the synchronous file write path is configured for `USTAR`, `TarHeader.WriteUstar` attempts to split a long path into:
|
||||
|
||||
- the main `name` field
|
||||
- the `prefix` field
|
||||
|
||||
If the name cannot be represented in USTAR field limits, the writer throws `InvalidFormatException` and instructs the caller to use GNU Tar format instead.
|
||||
|
||||
## Header Read Behavior
|
||||
|
||||
Tar header parsing is implemented in `TarHeader.Read` and `TarHeader.ReadAsync`.
|
||||
|
||||
### Implemented Read Features
|
||||
|
||||
| Feature | Read support |
|
||||
| ------- | ------------ |
|
||||
| Regular file entries | Yes |
|
||||
| Directory entries | Yes |
|
||||
| Symbolic link target reading | Yes |
|
||||
| Hard link target reading | Yes |
|
||||
| GNU long name (`L`) | Yes |
|
||||
| GNU long link (`K`) | Yes |
|
||||
| USTAR prefix reconstruction | Yes |
|
||||
| Binary size field parsing | Yes |
|
||||
| oldgnu uid/gid numeric quirk parsing | Yes |
|
||||
| POSIX and signed checksum validation | Yes |
|
||||
|
||||
### Entry Types Recognized by the Code
|
||||
|
||||
`EntryType` currently declares these values in `src/SharpCompress/Common/Tar/Headers/EntryType.cs`:
|
||||
|
||||
- `File`
|
||||
- `OldFile`
|
||||
- `HardLink`
|
||||
- `SymLink`
|
||||
- `CharDevice`
|
||||
- `BlockDevice`
|
||||
- `Directory`
|
||||
- `Fifo`
|
||||
- `LongLink`
|
||||
- `LongName`
|
||||
- `SparseFile`
|
||||
- `VolumeHeader`
|
||||
- `GlobalExtendedHeader`
|
||||
|
||||
SharpCompress currently has explicit handling for only a subset of those values during read and write.
|
||||
|
||||
### Long Name and Long Link Reads
|
||||
|
||||
When `TarHeader.Read` encounters `EntryType.LongName` or `EntryType.LongLink`, it reads the payload and applies it to the next real header.
|
||||
|
||||
Long-name payload reads are capped at `32768` bytes to avoid memory exhaustion from malformed archives.
|
||||
|
||||
### Name Reconstruction
|
||||
|
||||
For USTAR headers, if the magic field is `ustar` and the prefix field is populated, SharpCompress reconstructs the entry name as `prefix + "/" + name`.
|
||||
|
||||
## Name and Metadata Handling
|
||||
|
||||
### Path Normalization
|
||||
|
||||
Writer path normalization is implemented in `TarWriter.NormalizeFilename` and `NormalizeDirectoryName`.
|
||||
|
||||
Behavior:
|
||||
|
||||
- backslashes are converted to `/`
|
||||
- drive prefixes before `:` are removed
|
||||
- leading and trailing `/` are trimmed for file entries
|
||||
- directory entries are normalized to end with `/`
|
||||
|
||||
### Encoding
|
||||
|
||||
Tar name encoding and decoding is controlled by `IArchiveEncoding`.
|
||||
|
||||
- reader APIs decode names with `ReaderOptions.ArchiveEncoding`
|
||||
- writer APIs encode names with `TarWriterOptions.ArchiveEncoding`
|
||||
|
||||
The tests include UTF-8 and code page coverage for tar name handling.
|
||||
|
||||
### Metadata Surface
|
||||
|
||||
Tar metadata currently surfaced through `TarEntry` includes:
|
||||
|
||||
- name
|
||||
- link target
|
||||
- mode
|
||||
- uid
|
||||
- gid
|
||||
- size
|
||||
- last modified time
|
||||
|
||||
Writer metadata is narrower. The writer sets:
|
||||
|
||||
- `LastModifiedTime`
|
||||
- `Name`
|
||||
- `Size`
|
||||
- entry type for file or directory
|
||||
|
||||
The current writer writes fixed mode, owner id, and group id defaults rather than round-tripping full metadata.
|
||||
|
||||
## Async Behavior
|
||||
|
||||
Async tar support is provided by:
|
||||
|
||||
- `TarArchive.OpenAsyncArchive`
|
||||
- `TarReader.OpenAsyncReader`
|
||||
- `TarWriter.WriteAsync`
|
||||
- `TarWriter.WriteDirectoryAsync`
|
||||
- `TarHeader.ReadAsync`
|
||||
- `TarHeader.WriteAsync`
|
||||
|
||||
The async implementations generally mirror the sync implementations while using async header parsing, decompression, and stream copy paths. The most important current exception is `TarWriterOptions.HeaderFormat`, which is not consistently honored outside the synchronous file write path.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
This section documents current implementation limits, not desired future behavior.
|
||||
|
||||
### Write limitations
|
||||
|
||||
- No write support for `tar.xz`
|
||||
- No write support for `tar.zst`
|
||||
- No write support for `tar.Z`
|
||||
- No public API for writing symbolic links or hard links
|
||||
- No PAX write support
|
||||
- No sparse file write support
|
||||
- No device or FIFO write support
|
||||
|
||||
### Read limitations or partial support
|
||||
|
||||
- No explicit PAX local header support
|
||||
- No semantic sparse file handling beyond recognizing the entry type enum value
|
||||
- No semantic global extended header handling beyond recognizing the entry type enum value
|
||||
- No special device or FIFO object model beyond the raw entry type information available internally
|
||||
|
||||
### Archive behavior limitations
|
||||
|
||||
- Sync archive open requires a seekable input stream
|
||||
- Compressed tar archive access is not full random-access in the same sense as uncompressed seekable tar
|
||||
|
||||
## Test Coverage Map
|
||||
|
||||
Tar tests live in `tests/SharpCompress.Test/Tar/`.
|
||||
|
||||
Representative coverage:
|
||||
|
||||
| Area | Tests |
|
||||
| ---- | ----- |
|
||||
| Wrapper detection and reading | `TarReaderTests.cs`, `TarReaderAsyncTests.cs` |
|
||||
| Archive open and rewrite | `TarArchiveTests.cs`, `TarArchiveAsyncTests.cs` |
|
||||
| Writer behavior | `TarWriterTests.cs`, `TarWriterAsyncTests.cs` |
|
||||
| Directory entry behavior | `TarWriterDirectoryTests.cs`, `TarArchiveDirectoryTests.cs` |
|
||||
| Long-name behavior | `TarArchiveTests.cs`, `TarReaderTests.cs` |
|
||||
| Corruption and broken stream handling | `TarReaderTests.cs`, `TarReaderAsyncTests.cs` |
|
||||
|
||||
Representative tar test archives in `tests/TestArchives/Archives/`:
|
||||
|
||||
- `Tar.tar`
|
||||
- `Tar.tar.gz`
|
||||
- `Tar.tar.bz2`
|
||||
- `Tar.tar.lz`
|
||||
- `Tar.tar.xz`
|
||||
- `Tar.tar.zst`
|
||||
- `Tar.tar.Z`
|
||||
- `Tar.oldgnu.tar.gz`
|
||||
- `very long filename.tar`
|
||||
- `ustar with long names.tar`
|
||||
- `Tar.LongPathsWithLongNameExtension.tar`
|
||||
- `Tar.Empty.tar`
|
||||
- `TarCorrupted.tar`
|
||||
- `TarWithSymlink.tar.gz`
|
||||
|
||||
## Summary
|
||||
|
||||
SharpCompress Tar support is centered around:
|
||||
|
||||
- broad read support for common tar wrappers
|
||||
- forward-only reader behavior for streamed extraction
|
||||
- seekable archive support for uncompressed tar and archive rewrite workflows
|
||||
- narrower write support than read support
|
||||
- GNU long-name and USTAR write support
|
||||
- partial coverage for less common tar dialect features
|
||||
Reference in New Issue
Block a user