aaruf_create() left block_header_cache.free_func and block_cache.free_func at
the NULL the context was zeroed with, so on the write path every value the
caches evicted or dropped at close was leaked. aaruf_open() has always set
them; do the same here.
add_to_cache_uint64() called HASH_ADD without first looking the key up.
uthash keeps both entries in that case, but HASH_FIND only ever reaches the
newer one, so the older entry becomes unreachable and its value stays alive
until free_cache() tears the whole cache down. It also counted twice against
the cache budget.
ec_recover_data_block() reaches this: it caches the recovered BlockHeader for
a block offset that may already be in block_header_cache, because a header
read from the file is cached before the corruption check that sends the read
into erasure recovery.
Look the key up first, and on a hit free the old value and reuse the entry.
The read caches sized themselves as MAX_CACHE_SIZE / (SectorSize << shift),
but ctx->shift is only ever assigned on the DDT v1 path, and the context is
zeroed at open. On every DDT v2 image the divisor therefore collapsed to the
sector size and the limit came out as 262144 entries, while each block_cache
entry holds a whole decompressed data block of SectorSize << dataShift bytes,
8 MiB with the default data_shift of 12. That is a ceiling of roughly 2 TiB
rather than the intended 512 MiB, so the count limit was never reached: a
sequential sector-by-sector pass cached the entire decompressed image and
freed none of it until aaruf_close(). Comparing two 40 GB images this way was
killed by the OOM reaper.
An entry count cannot express a memory budget when the values vary in size,
and here they vary by five orders of magnitude: block_header_cache holds
BlockHeader structs while block_cache holds megabyte payloads, yet both were
given the same limit. Track the byte size of each value instead and evict
least-recently-used entries until the cache is back under budget, and drop the
block geometry from cache initialization entirely. The block cache gets the
512 MiB MAX_CACHE_SIZE it always intended; block headers get a separate 8 MiB
budget, which covers far more blocks than the payload cache can hold.
The entry just inserted is never evicted, so callers may keep using the
pointer they handed over for the rest of the call. That also removes the
use-after-free that a zero limit used to cause: the old eviction loop started
at the hash head and could free the entry just added, after which the caller
read from freed memory.
Recovers CD sector prefix/suffix arenas in a v2 image whose resumed dump
sessions left the final deduplication table referencing slots past the
shrunken data block. Scans the file for every prefix/suffix generation,
rebuilds a fresh compact arena and DDT2 for the final table by sourcing
each custom slot from the generation that held it, and rewrites the index.
Supports --dry-run to report recoverable/lost slots without writing.
Covers custom prefix/suffix bytes surviving a resumed dump session, a
resume that adds no new customs keeping the arena intact, and non-BCD
headers being stored verbatim rather than regenerated.
The Mode 1 user-data DDT entry was always written as Mode1Correct even
when the prefix or ECC/EDC was wrong; report Errored in that case like the
Mode 2 paths. The all-zero paths assigned SectorStatusNotDumped without
the <<60 status shift (harmless only because it is zero) and the Mode 2
all-zero path never marked its DDTs dirty, so an all-blank Mode 2 track
wrote no DDT at all; shift the status and set the dirty flags. Also widen
a truncating (uint32_t) cast on the Mode 2 prefix slot index to uint64_t.
The arenas were malloc'd and grown by doubling with realloc into the same
pointer: doubling could not grow a zero-length buffer, a failed realloc
leaked and lost the only pointer, and the Mode 2 Form 1/Form 2 paths left
part of each 288-byte slot uninitialized, writing heap garbage into the
image. Add a grow_arena() helper that grows before appending, keeps the
pointer on failure, and zeroes new capacity; allocate the arenas with
calloc and zero partial slots before copying.
The write path decoded the MSF header with a naive nibble multiply and
compared against the positive sector address, so a header that was not
valid BCD but aliased the expected address (e.g. 0x1A vs 0x20) was
classified correct and silently rewritten as canonical BCD on read, and
lead-in (negative) sectors never matched. Validate BCD nibbles and range
via cd_header_matches_lba() using the signed LBA, and reconstruct the
prefix from the signed LBA on read.
A DDT2 entry whose slot index points past the stored prefix/suffix block
(e.g. a resume-damaged or corrupt image) caused an out-of-bounds heap
read that was silently returned as sector data. Validate the pointer and
index against the arena length before copying, returning
AARUF_ERROR_CANNOT_READ_BLOCK with an Errored status instead. Also clamp
the two legacy Mode 2 fallback copies to the actual bare-sector length.
On aaruf_open(resume_mode=true) the loaded prefix/suffix arena blocks had
their lengths recorded but the append offsets were left at zero, so a
resumed dump session re-appended custom sync/ECC bytes from offset 0,
overwriting existing slots and leaving the DDT2 pointing past the
rewritten (shrunken) block. Record the block length on load and set the
append offset to it on resume so new customs append after existing slots.
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] New filesystem, test images in [url]
- [ ] New media image, test images in [url]
- [ ] New partition scheme, test images in [url]
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] I have read the **CONTRIBUTING** document.
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.
Add missing media types PS5BD and UHDBD to aaruf_get_xml_mediatype for optical discs.
Maybe the "xml" function should replaced by some generic meaning (e.g. aaruf_get_metadata_mediatype), since export is in .json file.
