Commit Graph

486 Commits

Author SHA1 Message Date
Rebecca Wallander
a6345b7efc Handle flux-only images 2026-05-01 09:14:06 +02:00
2676d8fe0c Merge branch 'devel' of github.com:aaru-dps/libaaruformat into devel 2026-04-12 14:49:24 +01:00
540538426e Merge pull request #39 from FakeShemp/fakeshemp/write-long
Actually write the BD sector tags to file
2026-04-12 14:49:13 +01:00
a5bd3279f5 Do EC recovery on secondary level DDT2 reading. 2026-04-12 14:45:52 +01:00
1f7532b0ee Add EC recovery of non-data blocks. 2026-04-12 14:37:54 +01:00
81c367995f Update recovery footer size and related documentation for AaruRecoveryFooter 2026-04-12 14:17:56 +01:00
Rebecca Wallander
71f1ab5290 Actually write the BD sector tags to file 2026-04-12 15:09:38 +02:00
651ad79a23 Fix bug in calculation of metadata block size when erasure coding. 2026-04-12 11:52:22 +01:00
9b87f90833 Write only needed shard size uncompressed. 2026-04-12 10:39:01 +01:00
60435e1fa4 Use correct sector size. 2026-04-12 10:38:35 +01:00
67fffced6e Add SIMD for GF for ARM32. 2026-04-11 23:27:12 +01:00
0811c4de50 Adjust auto erasure coding setter. 2026-04-11 22:49:41 +01:00
edc65f6b1c Add function to set EC based on recoverability percentage. 2026-04-11 22:11:56 +01:00
c401f40fe0 Compress batch parity. 2026-04-11 21:35:00 +01:00
501405a6ee Recover damaged block headers. 2026-04-11 21:32:25 +01:00
640ec68146 Recover damaged headers. 2026-04-11 21:27:25 +01:00
6f7cfbdf5e Recover EC of other block types. 2026-04-11 21:27:12 +01:00
ad6404df46 Do EC of other block types. 2026-04-11 21:16:47 +01:00
92ca533db8 Use EC to recover damaged data blocks. 2026-04-11 21:02:44 +01:00
af4503f376 Write EC when writing data blocks. 2026-04-11 20:28:48 +01:00
c16120ae5b Add GF and Reed Solomon. 2026-04-11 20:14:43 +01:00
Rebecca Wallander
d6b6accbf5 Add raw blu-ray and aacs tags 2026-04-10 19:47:34 +02:00
112f53dc3d More 64-bit file offset fixes. 2026-04-04 16:32:25 +01:00
509bb6026d Implement reentrant strtok function alias for compiler compatibility. 2026-04-04 15:53:45 +01:00
0ee5a97d23 Implement AARU_PREFETCH macro for improved prefetching across compilers 2026-04-04 15:53:03 +01:00
9c1d6d5e6f refactor: use AARU_ALIGN16 macro for 16-byte alignment in crc64_clmul.c and crc64_vmull.c 2026-04-04 15:11:13 +01:00
2aac1ce89c refactor: update file I/O to support large files with aaru_off_t type 2026-04-04 15:10:46 +01:00
31418a8511 On Win32 use SetLastError instead of setting errno. 2026-04-03 20:19:22 +01:00
3718f88b53 Merge pull request #36 from RomTholos/refactor/split-metadata-reader-writer
refactor: split metadata.c into reader/writer
2026-04-02 22:52:20 +01:00
Kevin Bortis
59c1aebcae refactor: split metadata.c into reader/writer
Separate read-only metadata accessors from write-path mutators to
prepare for the dual-license split (MIT reader / LGPL writer).

metadata.c retains 20 aaruf_get_* functions: geometry, media sequence,
creator, comments, media title, media/drive manufacturer/model/serial/
barcode/part number, firmware revision, CICM metadata, Aaru JSON
metadata, user/negative/overflow sectors, and image info.

metadata_write.c receives 28 functions: 15 aaruf_set_* (geometry,
media sequence, creator, comments, media title, media/drive
manufacturer/model/serial/barcode/part number, firmware revision,
Aaru JSON metadata) and 13 aaruf_clear_* counterparts.

No shared static helpers between the two files. No behavioral change.
All functions are independent public API — no function pointer or
callback mechanism needed (unlike close.c which required a finalization
hook).

Verified:
- Builds cleanly, zero warnings
- 163/165 tests pass (2 pre-existing failures)
- metadata.o exports no set_/clear_ symbols
2026-04-02 09:31:15 +02:00
Kevin Bortis
d8ff7b4a28 refactor: split close.c into reader/writer
Separate writer finalization from resource cleanup in close.c to
enable a future reader-only library target (aaruformatread).

close_write.c contains all write_* helpers and aaruf_finalize_write().
close.c retains aaruf_close() with cleanup only, dispatching writer
finalization via a function pointer set by aaruf_create()/aaruf_open().

Zero behavioral change — same tests pass, same error semantics.
2026-04-02 08:32:10 +02:00
62202a5c29 Merge pull request #32 from RomTholos/fix/verify-v1-lzma-crc
Fix v1 image verification and add aaruf_verify_image tests
2026-04-01 22:41:49 +01:00
Kevin Bortis
83433faaf7 Fix v1 image verification and add aaruf_verify_image tests
V1 images stored cmpCrc64 over the full LZMA payload including the
5-byte properties prefix, but verify.c unconditionally skipped the
prefix before computing the CRC. This caused aaruf_verify_image() to
report AARUF_ERROR_INVALID_BLOCK_CRC on every valid v1 image.

The fix makes the LZMA properties skip conditional on v2+ images:
- V1: CRC computed over full cmpLength (properties included), then byte-swapped
- V2+: CRC computed over cmpLength minus 5 (properties excluded), no byte-swap

Applies to DataBlock, DeDuplicationTable (v1), and DeDuplicationTable2.

Verified against fixtures created by Aaru 5.3 (v1) and aaruformattool (v2).

Tests added (14 total, VerifyImageFixture suite):
- Happy path: floptical/mf2hd/cdmode1 for both v1 and v2 (6 tests)
- Invalid context: NULL and garbage (2 tests)
- Corruption detection: data block v1/v2, TracksBlock (3 tests)
- Index corruption: invalid offset, bad signature (2 tests)
- Programmatic round-trip: create + write + close + verify (1 test)
2026-04-01 20:49:25 +02:00
Kevin Bortis
6fabbefb30 Extend zstd compression to all sections, add featureIncompatible bit 2026-04-01 19:45:41 +02:00
Kevin Bortis
b88e71a865 fix Mode 2 form detection in errored-suffix read path 2026-03-31 19:40:47 +02:00
Kevin Bortis
3c00aed352 Replace hardcoded LZMA thread count with LZMA_THREADS(ctx)
The LZMA encoder was called with numThreads=8 at all 19 call sites,
but LZMA only supports 1 or 2 (and _7ZIP_ST forces it to 1 anyway).
Replace with LZMA_THREADS(ctx) macro that clamps ctx->num_threads
to [1, 2], making the code say what it means and honoring the
consumer's threading preference from the threads=N option.
2026-03-29 19:14:38 +02:00
c58bfc167f Merge remote-tracking branch 'origin/devel' into devel 2026-03-29 18:08:31 +01:00
a81a0fe09a Merge pull request #23 from RomTholos/fix/flac-error-handling
fix: FLAC error handling — decode corruption, encode crash, unchecked…
2026-03-29 18:01:20 +01:00
Kevin Bortis
26e8d92810 fix: FLAC error handling — decode corruption, encode crash, unchecked returns 2026-03-29 15:47:27 +02:00
Kevin Bortis
120174e4cd Add multi-threaded zstd compression support
Switch zstd encoder from simple API (ZSTD_compress) to advanced API
(ZSTD_CCtx + ZSTD_compress2) with configurable worker threads via
ZSTD_c_nbWorkers. Consumer controls threading through threads=N
option string parameter. Default is 1 (single-threaded, bit-identical
output to previous behavior).

- Enable ZSTD_MULTITHREAD compile definition and link pthreads
- Add num_threads field to options struct and context
- Parse threads=N in option string (clamped >= 1)
- Rewrite aaruf_zstd_encode_buffer() with num_threads parameter
- Update all call sites (write.c, close.c)
2026-03-29 12:56:19 +02:00
42c4febd98 Increase maximum number of plaintext regions from 32 to 64 in ps3_encryption_map.h 2026-03-25 16:49:24 +00:00
0b6f10e526 Merge pull request #14 from FakeShemp/fakeshemp/wii-long
Add long sector for Wii/GC
2026-03-19 20:07:18 +00:00
Kevin Bortis
8a8a89450d lru: use native uint64 keys instead of string conversion
Replace string-keyed uthash lookups with HASH_FIND/HASH_ADD on native
uint64_t keys. This eliminates per-lookup malloc/snprintf/strlen/free
overhead from the block cache hot path (2 lookups per sector read).

Before: malloc(17) + snprintf hex + HASH_FIND_STR + free per lookup
After:  HASH_FIND with 8-byte integer key, zero allocations

Also removes the unused string-key API (find_in_cache/add_to_cache)
and the uint64_to_string helper — all callers use uint64 keys.

Fix off-by-one in eviction: >= caused cache to hold max_items-1
entries instead of max_items. Changed to > so the configured
capacity is honored exactly.
2026-03-19 20:35:42 +01:00
6dbd5f1bb2 Merge pull request #12 from RomTholos/fix/cross-compilation-arm-riscv
Fix cross-compilation for ARM FPU variants and add RISC-V support
2026-03-19 11:58:49 +00:00
c9cf1a42c0 Merge pull request #11 from RomTholos/fix/mode2-errored-suffix-read
Fix Mode 2 errored sector suffix not restored on read
2026-03-19 11:57:44 +00:00
Kevin Bortis
bd01e7d8a5 Fix cross-compilation for ARM FPU variants and add RISC-V support 2026-03-19 12:34:00 +01:00
65b00baf1a Merge pull request #10 from RomTholos/fix/calling-convention-pointer-order
Fix calling convention placement for pointer-returning functions
2026-03-19 09:50:02 +00:00
Kevin Bortis
48093ba779 Fix Mode 2 errored sector suffix not restored on read
When reading back Mode 2 sectors with errored ECC/EDC (SectorStatusErrored),
the else branch copied 2328 bytes from bare_data. However, bare_data only
contains the user data portion (2048 bytes for Form 1, 2324 for Form 2).
The EDC/ECC suffix was stored separately in sector_suffix during write but
never read back, causing the suffix area to contain stale data from adjacent
sectors in the decompressed block.

Fix both DDT v2 and v1 code paths to check the subheader form bit and copy
user data + suffix from their respective storage locations.

Affects any Mode 2 disc image (Dreamcast, PS1, PS2 CD, Saturn, Mega CD,
PC Engine CD, Neo Geo CD) where sectors have incorrect or zeroed ECC/EDC.
2026-03-19 10:11:06 +01:00
Kevin Bortis
981b54562e Fix calling convention placement for pointer-returning functions 2026-03-19 08:53:35 +01:00
Kevin Bortis
6be36b6bda Add Zstandard compression support
Implements zstd as an alternative to LZMA for data block and
subchannel compression using the allocated compression IDs 4
(kCompressionZstd) and 5 (kCompressionZstdCst).

Adds zstd v1.5.7 as a bundled submodule with a static library
build including x86_64 assembly fast path for decompression.

Write path selects zstd or LZMA based on the zstd option.
Subchannel blocks use zstd+CST instead of LZMA+CST when enabled.
LZMA properties header is only written for LZMA-based compression
types, preventing format corruption in zstd-compressed images.

Activated via options string: "zstd=true;zstd_level=19".
Default remains LZMA for backwards compatibility.

Tested: SHA-256 verified lossless roundtrips across 9 disc systems
(Dreamcast, Saturn, Mega CD, PC Engine CD, Neo Geo CD, PS1, PS2 CD,
PS2 DVD) and PS1 SBI subchannel preservation with zstd+CST.
2026-03-19 08:45:07 +01:00