mirror of
https://github.com/adamhathcock/sharpcompress.git
synced 2026-09-23 07:25:11 +00:00
Add GitHub Actions workflow, baseline results, and documentation for performance benchmarks
Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
103
tests/SharpCompress.Performance/README.md
Normal file
103
tests/SharpCompress.Performance/README.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# SharpCompress Performance Benchmarks
|
||||
|
||||
This project contains performance benchmarks for SharpCompress using [BenchmarkDotNet](https://benchmarkdotnet.org/).
|
||||
|
||||
## Overview
|
||||
|
||||
The benchmarks test all major archive formats supported by SharpCompress:
|
||||
- **Zip**: Read (Archive & Reader API) and Write operations
|
||||
- **Tar**: Read (Archive & Reader API) and Write operations, including Tar.GZip
|
||||
- **Rar**: Read operations (Archive & Reader API)
|
||||
- **7Zip**: Read operations for LZMA and LZMA2 compression
|
||||
- **GZip**: Compression and decompression
|
||||
|
||||
## Running Benchmarks
|
||||
|
||||
### Run all benchmarks
|
||||
```bash
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release
|
||||
```
|
||||
|
||||
### Run specific benchmark class
|
||||
```bash
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --filter "*ZipBenchmarks*"
|
||||
```
|
||||
|
||||
### Run with specific job configuration
|
||||
```bash
|
||||
# Quick run for testing (1 warmup, 1 iteration)
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --job Dry
|
||||
|
||||
# Short run (3 warmup, 3 iterations)
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --job Short
|
||||
|
||||
# Medium run (default)
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --job Medium
|
||||
```
|
||||
|
||||
### Export results
|
||||
```bash
|
||||
# Export to JSON
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --exporters json
|
||||
|
||||
# Export to multiple formats
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --exporters json markdown html
|
||||
```
|
||||
|
||||
### List available benchmarks
|
||||
```bash
|
||||
dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --list flat
|
||||
```
|
||||
|
||||
## Baseline Results
|
||||
|
||||
The baseline results are stored in `baseline-results.md` and represent the expected performance characteristics of the library. These results are used in CI to detect significant performance regressions.
|
||||
|
||||
To update the baseline:
|
||||
1. Run the benchmarks: `dotnet run --project tests/SharpCompress.Performance/SharpCompress.Performance.csproj --configuration Release -- --exporters markdown --artifacts baseline-output`
|
||||
2. Combine the results: `cat baseline-output/results/*-report-github.md > baseline-results.md`
|
||||
3. Review the changes and commit if appropriate
|
||||
|
||||
## CI Integration
|
||||
|
||||
The performance benchmarks run automatically in GitHub Actions on:
|
||||
- Push to `master` or `release` branches
|
||||
- Pull requests to `master` or `release` branches
|
||||
- Manual workflow dispatch
|
||||
|
||||
Results are displayed in the GitHub Actions summary and uploaded as artifacts.
|
||||
|
||||
## Benchmark Configuration
|
||||
|
||||
The benchmarks are configured with minimal iterations for CI efficiency:
|
||||
- **Warmup Count**: 1 iteration
|
||||
- **Iteration Count**: 3 iterations
|
||||
- **Invocation Count**: 1
|
||||
- **Unroll Factor**: 1
|
||||
- **Toolchain**: InProcessEmitToolchain (for fast execution)
|
||||
|
||||
These settings provide a good balance between speed and accuracy for CI purposes. For more accurate results, use the `Short`, `Medium`, or `Long` job configurations.
|
||||
|
||||
## Memory Diagnostics
|
||||
|
||||
All benchmarks include memory diagnostics using `[MemoryDiagnoser]`, which provides:
|
||||
- Total allocated memory per operation
|
||||
- Gen 0/1/2 collection counts
|
||||
|
||||
## Understanding Results
|
||||
|
||||
Key metrics in the benchmark results:
|
||||
- **Mean**: Average execution time
|
||||
- **Error**: Half of 99.9% confidence interval
|
||||
- **StdDev**: Standard deviation
|
||||
- **Allocated**: Total managed memory allocated per operation
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new benchmarks:
|
||||
1. Create a new class in the `Benchmarks/` directory
|
||||
2. Inherit from `ArchiveBenchmarkBase` for archive-related benchmarks
|
||||
3. Add `[MemoryDiagnoser]` attribute to the class
|
||||
4. Use `[Benchmark(Description = "...")]` for each benchmark method
|
||||
5. Add `[GlobalSetup]` for one-time initialization
|
||||
6. Update this README if needed
|
||||
81
tests/SharpCompress.Performance/baseline-results.md
Normal file
81
tests/SharpCompress.Performance/baseline-results.md
Normal file
@@ -0,0 +1,81 @@
|
||||
```
|
||||
|
||||
BenchmarkDotNet v0.14.0, Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
|
||||
.NET SDK 10.0.102
|
||||
[Host] : .NET 10.0.2 (10.0.225.61305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
|
||||
|
||||
Toolchain=InProcessEmitToolchain InvocationCount=1 IterationCount=3
|
||||
UnrollFactor=1 WarmupCount=1
|
||||
|
||||
```
|
||||
| Method | Mean | Error | StdDev | Allocated |
|
||||
|------------------------- |-----------:|-----------:|----------:|----------:|
|
||||
| 'GZip: Compress 100KB' | 6,090.9 μs | 1,940.6 μs | 106.37 μs | 523.37 KB |
|
||||
| 'GZip: Decompress 100KB' | 434.5 μs | 389.3 μs | 21.34 μs | 37.41 KB |
|
||||
```
|
||||
|
||||
BenchmarkDotNet v0.14.0, Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
|
||||
.NET SDK 10.0.102
|
||||
[Host] : .NET 10.0.2 (10.0.225.61305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
|
||||
|
||||
Toolchain=InProcessEmitToolchain InvocationCount=1 IterationCount=3
|
||||
UnrollFactor=1 WarmupCount=1
|
||||
|
||||
```
|
||||
| Method | Mean | Error | StdDev | Allocated |
|
||||
|----------------------------------------- |---------:|----------:|----------:|----------:|
|
||||
| 'Rar: Extract all entries (Archive API)' | 2.070 ms | 2.4938 ms | 0.1367 ms | 95.77 KB |
|
||||
| 'Rar: Extract all entries (Reader API)' | 2.359 ms | 0.9123 ms | 0.0500 ms | 154.18 KB |
|
||||
```
|
||||
|
||||
BenchmarkDotNet v0.14.0, Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
|
||||
.NET SDK 10.0.102
|
||||
[Host] : .NET 10.0.2 (10.0.225.61305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
|
||||
|
||||
Toolchain=InProcessEmitToolchain InvocationCount=1 IterationCount=3
|
||||
UnrollFactor=1 WarmupCount=1
|
||||
|
||||
```
|
||||
| Method | Mean | Error | StdDev | Allocated |
|
||||
|---------------------------------- |----------:|-----------:|----------:|----------:|
|
||||
| '7Zip LZMA: Extract all entries' | 14.042 ms | 98.7800 ms | 5.4145 ms | 277.93 KB |
|
||||
| '7Zip LZMA2: Extract all entries' | 8.654 ms | 0.6124 ms | 0.0336 ms | 277.71 KB |
|
||||
```
|
||||
|
||||
BenchmarkDotNet v0.14.0, Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
|
||||
.NET SDK 10.0.102
|
||||
[Host] : .NET 10.0.2 (10.0.225.61305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
|
||||
|
||||
Toolchain=InProcessEmitToolchain InvocationCount=1 IterationCount=3
|
||||
UnrollFactor=1 WarmupCount=1
|
||||
|
||||
```
|
||||
| Method | Mean | Error | StdDev | Allocated |
|
||||
|----------------------------------------- |---------:|---------:|---------:|----------:|
|
||||
| 'Tar: Extract all entries (Archive API)' | 148.0 μs | 356.4 μs | 19.53 μs | 20.73 KB |
|
||||
| 'Tar: Extract all entries (Reader API)' | 259.5 μs | 347.1 μs | 19.02 μs | 217.47 KB |
|
||||
| 'Tar.GZip: Extract all entries' | NA | NA | NA | NA |
|
||||
| 'Tar: Create archive with small files' | 139.1 μs | 728.8 μs | 39.95 μs | 72.55 KB |
|
||||
|
||||
Benchmarks with issues:
|
||||
TarBenchmarks.'Tar.GZip: Extract all entries': Job-NHXEIE(Toolchain=InProcessEmitToolchain, InvocationCount=1, IterationCount=3, UnrollFactor=1, WarmupCount=1)
|
||||
```
|
||||
|
||||
BenchmarkDotNet v0.14.0, Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
|
||||
.NET SDK 10.0.102
|
||||
[Host] : .NET 10.0.2 (10.0.225.61305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
|
||||
|
||||
Toolchain=InProcessEmitToolchain InvocationCount=1 IterationCount=3
|
||||
UnrollFactor=1 WarmupCount=1
|
||||
|
||||
```
|
||||
| Method | Mean | Error | StdDev | Median | Allocated |
|
||||
|----------------------------------------- |---------:|-----------:|----------:|----------:|-----------:|
|
||||
| 'Zip: Extract all entries (Archive API)' | 5.629 ms | 66.953 ms | 3.6699 ms | 7.2353 ms | 186.55 KB |
|
||||
| 'Zip: Extract all entries (Reader API)' | 4.935 ms | 114.613 ms | 6.2823 ms | 1.3354 ms | 128.05 KB |
|
||||
| 'Zip: Create archive with small files' | 1.250 ms | 10.341 ms | 0.5668 ms | 0.9229 ms | 2812.43 KB |
|
||||
Reference in New Issue
Block a user