The read caches sized themselves as MAX_CACHE_SIZE / (SectorSize << shift),
but ctx->shift is only ever assigned on the DDT v1 path, and the context is
zeroed at open. On every DDT v2 image the divisor therefore collapsed to the
sector size and the limit came out as 262144 entries, while each block_cache
entry holds a whole decompressed data block of SectorSize << dataShift bytes,
8 MiB with the default data_shift of 12. That is a ceiling of roughly 2 TiB
rather than the intended 512 MiB, so the count limit was never reached: a
sequential sector-by-sector pass cached the entire decompressed image and
freed none of it until aaruf_close(). Comparing two 40 GB images this way was
killed by the OOM reaper.
An entry count cannot express a memory budget when the values vary in size,
and here they vary by five orders of magnitude: block_header_cache holds
BlockHeader structs while block_cache holds megabyte payloads, yet both were
given the same limit. Track the byte size of each value instead and evict
least-recently-used entries until the cache is back under budget, and drop the
block geometry from cache initialization entirely. The block cache gets the
512 MiB MAX_CACHE_SIZE it always intended; block headers get a separate 8 MiB
budget, which covers far more blocks than the payload cache can hold.
The entry just inserted is never evicted, so callers may keep using the
pointer they handed over for the rest of the call. That also removes the
use-after-free that a zero limit used to cause: the old eviction loop started
at the hash head and could free the entry just added, after which the caller
read from freed memory.
Separate writer finalization from resource cleanup in close.c to
enable a future reader-only library target (aaruformatread).
close_write.c contains all write_* helpers and aaruf_finalize_write().
close.c retains aaruf_close() with cleanup only, dispatching writer
finalization via a function pointer set by aaruf_create()/aaruf_open().
Zero behavioral change — same tests pass, same error semantics.
The LZMA encoder was called with numThreads=8 at all 19 call sites,
but LZMA only supports 1 or 2 (and _7ZIP_ST forces it to 1 anyway).
Replace with LZMA_THREADS(ctx) macro that clamps ctx->num_threads
to [1, 2], making the code say what it means and honoring the
consumer's threading preference from the threads=N option.
Switch zstd encoder from simple API (ZSTD_compress) to advanced API
(ZSTD_CCtx + ZSTD_compress2) with configurable worker threads via
ZSTD_c_nbWorkers. Consumer controls threading through threads=N
option string parameter. Default is 1 (single-threaded, bit-identical
output to previous behavior).
- Enable ZSTD_MULTITHREAD compile definition and link pthreads
- Add num_threads field to options struct and context
- Parse threads=N in option string (clamped >= 1)
- Rewrite aaruf_zstd_encode_buffer() with num_threads parameter
- Update all call sites (write.c, close.c)
Replace string-keyed uthash lookups with HASH_FIND/HASH_ADD on native
uint64_t keys. This eliminates per-lookup malloc/snprintf/strlen/free
overhead from the block cache hot path (2 lookups per sector read).
Before: malloc(17) + snprintf hex + HASH_FIND_STR + free per lookup
After: HASH_FIND with 8-byte integer key, zero allocations
Also removes the unused string-key API (find_in_cache/add_to_cache)
and the uint64_to_string helper — all callers use uint64 keys.
Fix off-by-one in eviction: >= caused cache to hold max_items-1
entries instead of max_items. Changed to > so the configured
capacity is honored exactly.
Implements zstd as an alternative to LZMA for data block and
subchannel compression using the allocated compression IDs 4
(kCompressionZstd) and 5 (kCompressionZstdCst).
Adds zstd v1.5.7 as a bundled submodule with a static library
build including x86_64 assembly fast path for decompression.
Write path selects zstd or LZMA based on the zstd option.
Subchannel blocks use zstd+CST instead of LZMA+CST when enabled.
LZMA properties header is only written for LZMA-based compression
types, preventing format corruption in zstd-compressed images.
Activated via options string: "zstd=true;zstd_level=19".
Default remains LZMA for backwards compatibility.
Tested: SHA-256 verified lossless roundtrips across 9 disc systems
(Dreamcast, Saturn, Mega CD, PC Engine CD, Neo Geo CD, PS1, PS2 CD,
PS2 DVD) and PS1 SBI subchannel preservation with zstd+CST.
