Loading...
Searching...
No Matches
internal.h File Reference
`#include "utarray.h"`

Go to the source code of this file.

## Functions

UT_array * | process_index_v1 (aaruformat_context *ctx)
| Processes an index block (version 1) from the image stream.
int32_t | verify_index_v1 (aaruformat_context *ctx)
| Verifies the integrity of an index block (version 1) in the image stream.
UT_array * | process_index_v2 (aaruformat_context *ctx)
| Processes an index block (version 2) from the image stream.
int32_t | verify_index_v2 (aaruformat_context *ctx)
| Verifies the integrity of an index block (version 2) in the image stream.
UT_array * | process_index_v3 (aaruformat_context *ctx)
| Processes an index block (version 3) from the image stream.
int32_t | verify_index_v3 (aaruformat_context *ctx)
| Verifies the integrity of an index block (version 3) in the image stream.
int32_t | process_data_block (aaruformat_context *ctx, IndexEntry *entry)
| Processes a data block from the image stream.
int32_t | process_ddt_v1 (aaruformat_context *ctx, IndexEntry *entry, bool *found_user_data_ddt)
| Processes a DDT v1 block from the image stream.
int32_t | process_ddt_v2 (aaruformat_context *ctx, IndexEntry *entry, bool *found_user_data_ddt)
| Processes a DDT v2 block from the image stream.
void | process_metadata_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a metadata block from the image stream.
void | process_geometry_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a logical geometry block from the image stream.
void | process_tracks_block (aaruformat_context *ctx, const IndexEntry *entry)
| Parse and integrate a Tracks block from the image stream into the context.
void | process_cicm_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a CICM XML metadata block from the image stream.
void | process_aaru_metadata_json_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes an Aaru metadata JSON block from the image stream during image opening.
void | process_dumphw_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a dump hardware block from the image stream.
void | process_checksum_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a checksum block from the image stream.
void | process_tape_files_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a tape file metadata block from the image stream.
void | process_tape_partitions_block (aaruformat_context *ctx, const IndexEntry *entry)
| Processes a tape partition metadata block from the image stream.
void | process_flux_data_block (aaruformat_context *ctx, const IndexEntry *entry)
| Parse and integrate a Flux Data block from the image stream into the context.
int32_t | flux_map_rebuild_from_entries (aaruformat_context *ctx)
| Rebuild the flux capture lookup map from the flux_entries array.
int32_t | decode_ddt_entry_v1 (aaruformat_context *ctx, uint64_t sector_address, uint64_t *offset, uint64_t *block_offset, uint8_t *sector_status)
| Decodes a DDT v1 entry for a given sector address.
int32_t | decode_ddt_entry_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t *offset, uint64_t *block_offset, uint8_t *sector_status)
| Decodes a DDT v2 entry for a given sector address.
int32_t | decode_ddt_single_level_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t *offset, uint64_t *block_offset, uint8_t *sector_status)
| Decodes a single-level DDT v2 entry for a given sector address.
int32_t | decode_ddt_multi_level_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t *offset, uint64_t *block_offset, uint8_t *sector_status)
| Decodes a multi-level DDT v2 entry for a given sector address.
bool | set_ddt_entry_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t offset, uint64_t block_offset, uint8_t sector_status, uint64_t *ddt_entry)
| Sets a DDT v2 entry for a given sector address.
bool | set_ddt_single_level_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t offset, uint64_t block_offset, uint8_t sector_status, uint64_t *ddt_entry)
| Sets a single-level DDT v2 entry for a given sector address.
bool | set_ddt_multi_level_v2 (aaruformat_context *ctx, uint64_t sector_address, bool negative, uint64_t offset, uint64_t block_offset, uint8_t sector_status, uint64_t *ddt_entry)
| Sets a multi-level DDT v2 entry for a given sector address.
bool | set_ddt_tape (aaruformat_context *ctx, uint64_t sector_address, uint64_t offset, uint64_t block_offset, uint8_t sector_status, uint64_t *ddt_entry)
| Sets a DDT entry for tape media using a hash-based lookup table.
aaru_options | parse_options (const char *options, bool *table_shift_found)
| Parses the options string for AaruFormat image creation/opening.
uint64_t | get_filetime_uint64 ()
| Gets the current time as a 64-bit FILETIME value.
int32_t | aaruf_close_current_block (aaruformat_context *ctx)
| Finalizes and writes the current data block to the AaruFormat image file.
int | compare_extents (const void *a, const void *b)
| Comparison function for sorting DumpExtent arrays by start sector.
void | generate_random_bytes (uint8_t *buffer, size_t length)
| Generates cryptographically strong random bytes.

## Function Documentation

## aaruf_close_current_block()

int32_t aaruf_close_current_block | ( | aaruformat_context * | ctx | ) |

Finalizes and writes the current data block to the AaruFormat image file.

This function completes the current writing block by computing checksums, optionally compressing the buffered sector data, writing the block header and data to the image file, updating the index, and cleaning up resources. It is called automatically when a block is full (reaches maximum size determined by dataShift), when sector size changes, or when image finalization begins. The function supports multiple compression algorithms (FLAC for audio, LZMA for data) with automatic fallback to uncompressed storage if compression is ineffective.

Block Finalization Sequence:

1. Context Validation: Verify context is valid and opened for writing
2. Length Calculation: Set block length = currentBlockOffset × sectorSize
3. CRC64 Computation: Calculate checksum of uncompressed block data
4. Compression Processing (if enabled):
   - FLAC: For audio tracks, compress using Red Book audio encoding with configurable block size
   - LZMA: For data tracks, compress using LZMA algorithm with dictionary size optimization
   - Fallback: If compressed size ≥ uncompressed size, use uncompressed storage
5. Compressed CRC64: Calculate checksum of compressed data (if compression was applied)
6. Index Registration: Add IndexEntry to ctx->indexEntries for block lookup
7. File Writing: Write BlockHeader, optional LZMA properties, and block data to image stream
8. Position Update: Calculate next block position with alignment boundary adjustment
9. Resource Cleanup: Free buffers, reset counters, clear block header

Compression Handling:

- None (CompressionType = 0): No compression applied
  - cmpLength = length (uncompressed size)
  - cmpCrc64 = crc64 (same checksum)
  - Direct write of ctx->writingBuffer to file
- FLAC (CompressionType = 2): For CD audio tracks (Red Book format)
  - Allocates 2× length buffer for compressed data
  - Calculates optimal FLAC block size (MIN_FLAKE_BLOCK to MAX_FLAKE_BLOCK range)
  - Pads incomplete blocks with zeros to meet FLAC block size requirements
  - Encoding parameters: mid-side stereo, Hamming apodization, 12 max LPC order
  - Falls back to None if compression ineffective (compressed ≥ uncompressed)
- LZMA (CompressionType = 1): For data tracks and non-audio content
  - Allocates 2× length buffer for compressed data
  - LZMA properties: level 9, dictionary size from ctx->lzma_dict_size
  - Properties stored as 5-byte header: lc=4, lp=0, pb=2, fb=273, threads=8
  - Falls back to None if compression ineffective (compressed ≥ uncompressed)
  - Compressed length includes LZMA_PROPERTIES_LENGTH (5 bytes) overhead

Index Entry Creation:

Each closed block is registered in the index with:

- blockType = DataBlock (0x4B4C4244)
- dataType = UserData (1)
- offset = ctx->nextBlockPosition (file position where block was written)

This enables efficient block lookup during image reading via binary search on index entries.

File Layout:

Written to ctx->imageStream at ctx->nextBlockPosition:

1. BlockHeader (sizeof(BlockHeader) bytes)
2. LZMA properties (5 bytes, only if compression = Lzma)
3. Block data (cmpLength bytes - compressed or uncompressed depending on compression type)

Next Block Position Calculation:

After writing, nextBlockPosition is updated to the next aligned boundary:

- block_total_size = sizeof(BlockHeader) + cmpLength
- alignment_mask = (1 << blockAlignmentShift) - 1
- nextBlockPosition = (currentPosition + block_total_size + alignment_mask) & ~alignment_mask

This ensures all blocks begin on properly aligned file offsets for efficient I/O.

Resource Cleanup:

Before returning, the function:

- Frees ctx->writingBuffer and sets pointer to NULL
- Resets ctx->currentBlockOffset to 0
- Clears ctx->currentBlockHeader (memset to 0)
- Frees ctx->crc64Context
- Resets ctx->writingBufferPosition to 0

This prepares the context for the next block or signals that no block is currently open.

Parameters

  ctx | Pointer to an initialized aaruformatContext in write mode.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully finalized and wrote the block. This is returned when:
  - The context is valid and properly initialized
  - CRC64 computation completed successfully
  - Compression (if applicable) succeeded or fell back appropriately
  - Block header was successfully written to the image file
  - Block data (compressed or uncompressed) was successfully written
  - LZMA properties (if applicable) were successfully written
  - Index entry was added to ctx->indexEntries
  - Next block position was calculated and updated
  - All resources were freed and context reset for next block

  AARUF_ERROR_NOT_AARUFORMAT

  (-1) The context is invalid. This occurs when:
  - The context parameter is NULL
  - The context magic number doesn't match AARU_MAGIC (invalid context type)

  AARUF_READ_ONLY

  (-22) Attempting to finalize block on read-only image. This occurs when:
  - The context's isWriting flag is false
  - The image was opened in read-only mode with aaruf_open()

  AARUF_ERROR_NOT_ENOUGH_MEMORY

  (-9) Memory allocation failed. This occurs when:
  - Cannot allocate compression buffer (2× block length) for FLAC compression
  - Cannot allocate compression buffer (2× block length) for LZMA compression
  - System is out of memory or memory is severely fragmented

  AARUF_ERROR_CANNOT_WRITE_BLOCK_HEADER

  (-23) Failed to write block header. This occurs when:
  - fwrite() for BlockHeader returns != 1 (incomplete write or I/O error)
  - Disk space is insufficient for header
  - File system errors or permissions prevent writing
  - Media errors on the destination storage device

  AARUF_ERROR_CANNOT_WRITE_BLOCK_DATA

  (-24) Failed to write block data. This occurs when:
  - fwrite() for LZMA properties returns != 1 (when compression = Lzma)
  - fwrite() for uncompressed data returns != 1 (when compression = None)
  - fwrite() for compressed data returns != 1 (when compression = Flac/Lzma)
  - Disk space is insufficient for block data
  - File system errors or permissions prevent writing
  - Invalid compression type (not None, Flac, or Lzma)

Note
  Compression Algorithm Selection:
  - Compression type is determined when block is created in aaruf_write_sector()
  - Audio tracks (TrackType = Audio) use FLAC compression if enabled
  - Data tracks use LZMA compression if enabled
  - Special cases (JaguarCD data in audio, VideoNow) force LZMA even for audio tracks
  - Compression can be disabled entirely via ctx->compression_enabled flag
  Compression Fallback Logic:
  - If compressed size ≥ uncompressed size, compression is abandoned
  - Compression buffer is freed and compression type set to None
  - This prevents storage expansion from ineffective compression
  - Fallback is transparent to caller; function still returns success
  FLAC Encoding Parameters:
  - Encoding for Red Book audio (44.1kHz, 16-bit stereo)
  - Block size: auto-selected between MIN_FLAKE_BLOCK and MAX_FLAKE_BLOCK samples
  - Mid-side stereo enabled for better compression on correlated channels
  - Hamming window apodization for LPC analysis
  - Max LPC order: 12, QLP coefficient precision: 15 bits
  - Application ID: "Aaru" with 4-byte signature
  LZMA Encoding Parameters:
  - Compression level: 9 (maximum compression)
  - Dictionary size: from ctx->lzma_dict_size (configurable per context)
  - Literal context bits (lc): 4
  - Literal position bits (lp): 0
  - Position bits (pb): 2
  - Fast bytes (fb): 273
  - Threads: 8 (for multi-threaded compression)
  Index Management:
  - Every closed block gets an index entry for efficient lookup
  - Index entries are stored in ctx->indexEntries (dynamic array)
  - Final index is serialized during aaruf_close() for image finalization
  - Index enables O(log n) block lookup during image reading
  Alignment Requirements:
  - Blocks must start on aligned boundaries per blockAlignmentShift
  - Typical alignment: 512 bytes (shift=9) or 4096 bytes (shift=12)
  - Alignment ensures efficient sector-aligned I/O on modern storage
  - Gap between blocks is implicit; no padding data is written

Warning
  This function assumes ctx->writingBuffer contains valid data for ctx->currentBlockOffset sectors of ctx->currentBlockHeader.sectorSize bytes each.
  Do not call this function when no block is open (ctx->writingBuffer == NULL). This will result in undefined behavior or segmentation fault.
  The function modifies ctx->nextBlockPosition, which affects where subsequent blocks are written. Ensure file positioning is properly managed.
  Memory allocated for compression buffers is freed before returning. Do not retain pointers to compressed data after function completion.
  CRC64 context (ctx->crc64Context) is freed during cleanup. Do not access this pointer after calling this function.

Definition at line 1438 of file write.c.

References AARU_MAGIC, aaruf_crc64_data(), aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_WRITE_BLOCK_DATA, AARUF_ERROR_CANNOT_WRITE_BLOCK_HEADER, AARUF_ERROR_NOT_AARUFORMAT, AARUF_ERROR_NOT_ENOUGH_MEMORY, aaruf_flac_encode_redbook_buffer(), aaruf_lzma_encode_buffer(), AARUF_READ_ONLY, AARUF_STATUS_OK, DdtHeader2::blockAlignmentShift, IndexEntry::blockType, BlockHeader::cmpCrc64, BlockHeader::cmpLength, BlockHeader::compression, BlockHeader::crc64, aaruformat_context::crc64_context, aaruformat_context::current_block_header, aaruformat_context::current_block_offset, DataBlock, IndexEntry::dataType, aaruformat_context::dirty_index_block, FATAL, Flac, aaruformat_context::imageStream, aaruformat_context::index_entries, aaruformat_context::is_writing, BlockHeader::length, Lzma, aaruformat_context::lzma_dict_size, LZMA_PROPERTIES_LENGTH, aaruformat_context::magic, MAX_FLAKE_BLOCK, MIN_FLAKE_BLOCK, aaruformat_context::next_block_position, None, IndexEntry::offset, SAMPLES_PER_SECTOR, BlockHeader::sectorSize, TRACE, aaruformat_context::user_data_ddt_header, UserData, aaruformat_context::writing_buffer, and aaruformat_context::writing_buffer_position.

Referenced by aaruf_close(), aaruf_write_sector(), and set_ddt_multi_level_v2().

## compare_extents()

int compare_extents | ( | const void * | a,
|  | const void * | b )

Comparison function for sorting DumpExtent arrays by start sector.

This function is used by qsort() to order dump extents in ascending order based on their start sector values. Extents with lower start sectors will appear first in the sorted array. This ordering is important for efficient extent lookup and validation during image operations.

Parameters

  a | Pointer to the first DumpExtent to compare.
  b | Pointer to the second DumpExtent to compare.

Returns
  Negative value if a->start < b->start, zero if equal, positive if a->start > b->start.

Definition at line 463 of file helpers.c.

References DumpExtent::start.

Referenced by aaruf_set_dumphw(), and process_dumphw_block().

## decode_ddt_entry_v1()

int32_t decode_ddt_entry_v1 | ( | aaruformat_context * | ctx,
|  | const uint64_t | sector_address,
|  | uint64_t * | offset,
|  | uint64_t * | block_offset,
|  | uint8_t * | sector_status )

Decodes a DDT v1 entry for a given sector address.

Determines the offset and block offset for a sector using the DDT v1 table. This function performs bit manipulation on the DDT entry to extract the sector offset within a block and the block offset, and determines whether the sector was dumped or not based on the DDT entry value.

Parameters

  ctx | Pointer to the aaruformat context containing the loaded DDT table.
  sector_address | Logical sector address to decode (must be within valid range).
  offset | Pointer to store the resulting sector offset within the block.
  block_offset | Pointer to store the resulting block offset in the image.
  sector_status | Pointer to store the sector status (dumped or not dumped).

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully decoded the DDT entry. This is always returned when:
  - The context and image stream are valid
  - The DDT entry is successfully extracted and decoded
  - The offset, block_offset, and sector_status are successfully populated
  - A zero DDT entry is encountered (indicates sector not dumped)
  - A non-zero DDT entry is encountered (indicates sector was dumped)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers). This is the only error condition that can occur in this function.

Note
  This is a lightweight function that performs only basic validation and bit manipulation. It does not perform bounds checking on the sector_address parameter.
  DDT Entry Decoding:
  - Uses a bit mask derived from ctx->shift to extract the offset within block
  - Right-shifts the DDT entry by ctx->shift bits to get the block offset
  - A zero DDT entry indicates the sector was not dumped (SectorStatusNotDumped)
  - A non-zero DDT entry indicates the sector was dumped (SectorStatusDumped)

Warning
  The function assumes:
  - The DDT table (ctx->userDataDdt) has been properly loaded by process_ddt_v1()
  - The sector_address is within the valid range of the DDT table
  - The shift value (ctx->shift) has been properly initialized
  - All output parameters (offset, block_offset, sector_status) are valid pointers
  No bounds checking is performed on sector_address. Accessing beyond the DDT table boundaries will result in undefined behavior.

Definition at line 405 of file ddt_v1.c.

References AARUF_ERROR_INCORRECT_DATA_SIZE, AARUF_ERROR_NOT_AARUFORMAT, AARUF_STATUS_OK, FATAL, aaruformat_context::imageStream, SectorStatusDumped, SectorStatusNotDumped, aaruformat_context::shift, TRACE, and aaruformat_context::user_data_ddt.

Referenced by aaruf_read_sector().

## decode_ddt_entry_v2()

int32_t decode_ddt_entry_v2 | ( | aaruformat_context * | ctx,
|  | const uint64_t | sector_address,
|  | bool | negative,
|  | uint64_t * | offset,
|  | uint64_t * | block_offset,
|  | uint8_t * | sector_status )

Decodes a DDT v2 entry for a given sector address.

Determines the offset and block offset for a sector using the DDT v2 table(s). This function acts as a dispatcher that automatically selects between single-level and multi-level DDT decoding based on the tableShift parameter in the DDT header. It provides a unified interface for DDT v2 entry decoding regardless of the underlying table structure complexity.

Parameters

  ctx | Pointer to the aaruformat context containing the loaded DDT structures.
  sector_address | Logical sector address to decode (will be adjusted for negative sectors).
  negative | Indicates if the sector address is negative.
  offset | Pointer to store the resulting sector offset within the block.
  block_offset | Pointer to store the resulting block offset in the image.
  sector_status | Pointer to store the sector status (dumped, not dumped, etc.).

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully decoded the DDT entry. This is always returned when:
  - The context and image stream are valid
  - The appropriate decoding function (single-level or multi-level) completes successfully
  - All output parameters are properly populated with decoded values

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers). This is the only error condition that can occur at this dispatcher level.

  Other

  error codes may be returned by the underlying decoding functions:
  - From decode_ddt_single_level_v2(): AARUF_ERROR_CANNOT_READ_BLOCK (-7)
  - From decode_ddt_multi_level_v2(): AARUF_ERROR_CANNOT_READ_BLOCK (-7), AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK (-17), AARUF_ERROR_INVALID_BLOCK_CRC (-18)

Note
  Function Selection:
  - If tableShift > 0: Uses multi-level DDT decoding (decode_ddt_multi_level_v2)
  - If tableShift = 0: Uses single-level DDT decoding (decode_ddt_single_level_v2)
  - The tableShift parameter is read from ctx->userDataDdtHeader.tableShift
  This function performs minimal validation and primarily acts as a dispatcher. Most error conditions and complex logic are handled by the underlying functions.

Warning
  The function assumes the DDT has been properly loaded by process_ddt_v2(). Calling this function with an uninitialized or corrupted DDT will result in undefined behavior from the underlying decoding functions.
  All output parameters must be valid pointers. No bounds checking is performed on the sector_address parameter at this level.

Definition at line 511 of file ddt_v2.c.

References AARUF_ERROR_NOT_AARUFORMAT, decode_ddt_multi_level_v2(), decode_ddt_single_level_v2(), FATAL, aaruformat_context::imageStream, DdtHeader2::tableShift, TRACE, and aaruformat_context::user_data_ddt_header.

Referenced by aaruf_read_sector().

## decode_ddt_multi_level_v2()

int32_t decode_ddt_multi_level_v2 | ( | aaruformat_context * | ctx,
|  | uint64_t | sector_address,
|  | bool | negative,
|  | uint64_t * | offset,
|  | uint64_t * | block_offset,
|  | uint8_t * | sector_status )

Decodes a multi-level DDT v2 entry for a given sector address.

Used when the DDT table uses multi-level indirection (tableShift > 0). This function handles the complex process of navigating a hierarchical DDT structure where the primary table points to secondary tables that contain the actual sector mappings. It includes caching mechanisms for secondary tables, supports both compressed and uncompressed secondary tables, and performs comprehensive validation including CRC verification.

Parameters

  ctx | Pointer to the aaruformat context containing the loaded primary DDT table.
  sector_address | Logical sector address to decode (adjusted for negative sectors).
  negative | Indicates if the sector address is negative.
  offset | Pointer to store the resulting sector offset within the block.
  block_offset | Pointer to store the resulting block offset in the image.
  sector_status | Pointer to store the sector status (dumped, not dumped, etc.).

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully decoded the DDT entry. This is returned when:
  - The context and image stream are valid
  - The tableShift validation passes (must be > 0)
  - The DDT size type is recognized (SmallDdtSizeType or BigDdtSizeType)
  - Secondary DDT table is successfully loaded (from cache or file)
  - Secondary DDT decompression succeeds (if needed)
  - Secondary DDT CRC validation passes
  - The DDT entry is successfully extracted and decoded
  - All output parameters are properly populated with decoded values
  - Zero DDT entries are handled (indicates sector not dumped)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers).

  AARUF_ERROR_CANNOT_READ_BLOCK

  (-7) Configuration, validation, or file access errors. This occurs when:
  - The tableShift is zero (should use single-level decoding instead)
  - The DDT size type is unknown/unsupported (not SmallDdtSizeType or BigDdtSizeType)
  - Cannot read the secondary DDT header from the image stream
  - Secondary DDT header validation fails (wrong identifier or type)
  - Cannot read uncompressed secondary DDT data from the image stream
  - CRC64 context initialization fails (internal error)
  - Memory allocation fails for secondary DDT data (critical failure)
  - Unknown compression type encountered in secondary DDT

  AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK

  (-17) LZMA decompression failed for secondary DDT. This occurs when:
  - Memory allocation fails for compressed data or decompression buffer
  - Cannot read LZMA properties from the image stream
  - Cannot read compressed secondary DDT data from the image stream
  - The LZMA decoder returns a non-zero error code during decompression
  - The decompressed data size doesn't match the expected secondary DDT length

  AARUF_ERROR_INVALID_BLOCK_CRC

  (-18) CRC64 validation failed for secondary DDT. This occurs when:
  - Calculated CRC64 doesn't match the expected CRC64 in the secondary DDT header
  - Data corruption is detected in the secondary DDT data
  - This applies to both compressed and uncompressed secondary DDT blocks

Note
  Multi-level DDT Navigation:
  - Uses tableShift to calculate items per DDT entry (2^tableShift)
  - Calculates DDT position by dividing sector address by items per entry
  - Retrieves secondary DDT offset from primary table at calculated position
  - Converts block offset to file offset using blockAlignmentShift
  Secondary DDT Caching:
  - Maintains a single cached secondary DDT in memory (ctx->cachedSecondaryDdtSmall/Big)
  - Compares requested offset with cached offset (ctx->cachedDdtOffset)
  - Only loads from disk if the requested secondary DDT is not currently cached
  - Caching improves performance for sequential sector access patterns
  Secondary DDT Processing:
  - Supports both LZMA compression and uncompressed formats
  - Performs full CRC64 validation of secondary DDT data
  - Same bit manipulation as single-level DDT for final entry decoding
  Error Handling Strategy:
  - Memory allocation failures for secondary DDT loading are treated as critical errors
  - File I/O errors and validation failures cause immediate function termination
  - Unknown compression types are treated as errors (unlike the processing functions)
  - All allocated memory is cleaned up on error conditions

Warning
  This function should only be called when tableShift > 0. Calling it with tableShift = 0 will result in AARUF_ERROR_CANNOT_READ_BLOCK.
  The function assumes the primary DDT table has been properly loaded and is accessible via ctx->userDataDdtMini or ctx->userDataDdtBig depending on size type.
  Secondary DDT caching means that memory usage can increase during operation. The cached secondary DDT is replaced when a different secondary table is needed.
  No bounds checking is performed on sector_address or calculated DDT positions. Accessing beyond table boundaries will result in undefined behavior.

Definition at line 728 of file ddt_v2.c.

References aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK, AARUF_ERROR_CANNOT_READ_BLOCK, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, aaruf_lzma_decode_buffer(), AARUF_STATUS_OK, DdtHeader2::blockAlignmentShift, aaruformat_context::cached_ddt_offset, aaruformat_context::cached_secondary_ddt2, DdtHeader2::cmpLength, DdtHeader2::compression, DdtHeader2::crc64, DdtHeader2::dataShift, DeDuplicationTableSAlpha, DeDuplicationTableSecondary, FATAL, DdtHeader2::identifier, aaruformat_context::imageStream, DdtHeader2::length, Lzma, LZMA_PROPERTIES_LENGTH, DdtHeader2::negative, None, SectorStatusNotDumped, DdtHeader2::tableShift, TRACE, DdtHeader2::type, aaruformat_context::user_data_ddt2, aaruformat_context::user_data_ddt_header, and UserData.

Referenced by decode_ddt_entry_v2().

## decode_ddt_single_level_v2()

int32_t decode_ddt_single_level_v2 | ( | aaruformat_context * | ctx,
|  | uint64_t | sector_address,
|  | bool | negative,
|  | uint64_t * | offset,
|  | uint64_t * | block_offset,
|  | uint8_t * | sector_status )

Decodes a single-level DDT v2 entry for a given sector address.

Used when the DDT table does not use multi-level indirection (tableShift = 0). This function performs direct lookup in the primary DDT table to extract sector offset, block offset, and sector status information. It performs bit manipulation to decode the packed DDT entry values.

Parameters

  ctx | Pointer to the aaruformat context containing the loaded DDT table.
  sector_address | Logical sector address to decode (adjusted for negative sectors).
  negative | Indicates if the sector address is negative.
  offset | Pointer to store the resulting sector offset within the block.
  block_offset | Pointer to store the resulting block offset in the image.
  sector_status | Pointer to store the sector status (dumped, not dumped, etc.).

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully decoded the DDT entry. This is always returned when:
  - The context and image stream are valid
  - The tableShift validation passes (must be 0)
  - The DDT size type is recognized (SmallDdtSizeType or BigDdtSizeType)
  - The DDT entry is successfully extracted and decoded
  - All output parameters are properly populated with decoded values
  - Zero DDT entries are handled (indicates sector not dumped)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers).

  AARUF_ERROR_CANNOT_READ_BLOCK

  (-7) Configuration or validation errors. This occurs when:
  - The tableShift is not zero (should use multi-level decoding instead)
  - The DDT size type is unknown/unsupported (not SmallDdtSizeType or BigDdtSizeType)
  - Internal consistency checks fail

Note
  DDT Entry Decoding
  - Bits 63-61: Sector status (4 bits)
  - Bits 60-0: Combined offset and block index (60 bits)
  - Offset mask: Derived from dataShift parameter
  - Block offset: Calculated using blockAlignmentShift parameter
  Negative Sector Handling:
  - Sector address is automatically adjusted by adding ctx->userDataDdtHeader.negative
  - This allows proper indexing into the DDT table for negative sector addresses
  Zero Entry Handling:
  - A zero DDT entry indicates the sector was not dumped
  - Sets sector_status to SectorStatusNotDumped and zeros offset/block_offset
  - This is a normal condition and not an error

Warning
  The function assumes the DDT table has been properly loaded and is accessible via ctx->userDataDdtMini or ctx->userDataDdtBig depending on size type.
  No bounds checking is performed on sector_address. Accessing beyond the DDT table boundaries will result in undefined behavior.
  This function should only be called when tableShift is 0. Calling it with tableShift > 0 will result in AARUF_ERROR_CANNOT_READ_BLOCK.

Definition at line 585 of file ddt_v2.c.

References AARUF_ERROR_CANNOT_READ_BLOCK, AARUF_ERROR_NOT_AARUFORMAT, AARUF_STATUS_OK, DdtHeader2::blockAlignmentShift, DdtHeader2::dataShift, FATAL, aaruformat_context::imageStream, DdtHeader2::negative, SectorStatusNotDumped, DdtHeader2::tableShift, TRACE, aaruformat_context::user_data_ddt2, and aaruformat_context::user_data_ddt_header.

Referenced by decode_ddt_entry_v2().

## flux_map_rebuild_from_entries()

int32_t flux_map_rebuild_from_entries | ( | aaruformat_context * | ctx | ) |

Rebuild the flux capture lookup map from the flux_entries array.

This function clears any existing flux capture lookup map and rebuilds it from the ctx->flux_entries array. The map provides O(1) lookup of flux captures by their (head, track, subtrack, captureIndex) identifier tuple, which is used by aaruf_read_flux_capture() and other flux access functions.

The function iterates through all entries in ctx->flux_entries and adds each one to the hash table, mapping its identifier to its array index. If any entry fails to be added (due to memory allocation failure), the entire map is cleared and an error is returned.

Parameters

  ctx | Pointer to the aaruformat context containing flux_entries and flux_map. Must not be NULL.

Returns
  AARUF_STATUS_OK on success, or AARUF_ERROR_NOT_ENOUGH_MEMORY if memory allocation fails during map construction.

Return values

  AARUF_STATUS_OK | The map was successfully rebuilt (or no entries to process).
  AARUF_ERROR_NOT_ENOUGH_MEMORY | Memory allocation failed; the map is cleared.

Note
  If ctx->flux_entries is NULL or ctx->flux_data_header.entries is 0, the function clears the map and returns AARUF_STATUS_OK (no-op).
  On failure, the map is cleared to ensure a consistent state.
  This function is called automatically by process_flux_data_block() after successfully reading flux entries from the image.

See also
  flux_map_clear() for clearing the map
  flux_map_add() for adding individual entries
  process_flux_data_block() for when this is called during image opening

Definition at line 171 of file flux.c.

References AARUF_ERROR_NOT_ENOUGH_MEMORY, AARUF_STATUS_OK, FluxEntry::captureIndex, FluxHeader::entries, FATAL, aaruformat_context::flux_data_header, aaruformat_context::flux_entries, flux_map_add(), flux_map_clear(), FluxEntry::head, FluxEntry::subtrack, and FluxEntry::track.

Referenced by process_flux_data_block(), and write_flux_blocks().

## generate_random_bytes()

void generate_random_bytes | ( | uint8_t * | buffer,
|  | size_t | length )

Generates cryptographically strong random bytes.

This function fills the provided buffer with random bytes using platform-appropriate cryptographic random number generators. On Unix-like systems (including macOS and Linux), it reads from /dev/urandom. On Windows, it uses CryptGenRandom. If the platform-specific methods fail, it falls back to a time-seeded pseudo-random generator.

Parameters

  buffer | Pointer to the buffer to fill with random bytes.
  length | Number of random bytes to generate.

Definition at line 484 of file helpers.c.

Referenced by aaruf_create().

## get_filetime_uint64()

uint64_t get_filetime_uint64 | ( |  | ) |

Gets the current time as a 64-bit FILETIME value.

Returns the current system time as a 64-bit value compatible with Windows FILETIME (number of 100-nanosecond intervals since January 1, 1601 UTC).

Returns
  The current time as a 64-bit FILETIME value.

Definition at line 45 of file time.c.

Referenced by aaruf_create(), and aaruf_open().

## parse_options()

aaru_options parse_options | ( | const char * | options,
|  | bool * | table_shift_found )

Parses the options string for AaruFormat image creation/opening.

Parses the options string and returns a struct with the parsed options for image creation or opening.

Parameters

  options | String with options to parse (may be NULL).

Returns
  Parsed options as an aaru_options struct.

Definition at line 38 of file options.c.

References aaru_options::blake3, aaru_options::block_alignment, aaru_options::compress, aaru_options::data_shift, aaru_options::deduplicate, aaru_options::dictionary, aaru_options::md5, aaru_options::sha1, aaru_options::sha256, aaru_options::spamsum, aaru_options::table_shift, and TRACE.

Referenced by aaruf_create(), and aaruf_open().

## process_aaru_metadata_json_block()

void process_aaru_metadata_json_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes an Aaru metadata JSON block from the image stream during image opening.

Reads an Aaru metadata JSON block from the image file and loads its contents into the context for subsequent retrieval. The Aaru metadata JSON format is a structured representation of comprehensive image metadata including media information, imaging session details, hardware configuration, optical disc tracks and sessions, checksums, and preservation metadata. The JSON payload is stored in its original form without parsing or interpretation by this function, allowing higher-level code to process the structured data as needed.

This function is called during the image opening process (aaruf_open) when an index entry indicates the presence of an AaruMetadataJsonBlock. The function is non-critical; if reading fails or memory allocation fails, the error is logged but the image opening continues. This allows images without JSON metadata or with corrupted JSON blocks to still be opened for data access.

Processing sequence:

1. Validate context and image stream
2. Seek to the block offset specified by the index entry
3. Read the AaruMetadataJsonBlockHeader (8 bytes: identifier + length)
4. Validate the block identifier matches AaruMetadataJsonBlock
5. Allocate memory for the JSON payload
6. Read the JSON data from the file stream
7. Store header and data pointer in the context for later retrieval

Memory allocation: The function allocates memory (via malloc) sized to hold the entire JSON payload as specified by ctx->jsonBlockHeader.length. This memory remains allocated for the lifetime of the context and is freed during aaruf_close(). If allocation fails, the function returns gracefully without the JSON metadata, allowing the image to still be opened.

Image size tracking: The function increments ctx->imageInfo.ImageSize by the length of the JSON payload to track the total size of metadata and structural blocks in the image.

Error handling: All errors are non-fatal and handled gracefully:

- Seek failures: logged and function returns early
- Header read failures: header zeroed, function returns
- Identifier mismatches: header zeroed, processing continues but data is not loaded
- Memory allocation failures: header zeroed, function returns
- Data read failures: header zeroed, allocated memory freed, function returns

In all error cases, the ctx->jsonBlockHeader is zeroed (memset to 0) to indicate that no valid JSON metadata is available, and any allocated memory is properly freed.

Parameters

  ctx | Pointer to an initialized aaruformatContext being populated during image opening. Must not be NULL. ctx->imageStream must be open and readable. On success, ctx->jsonBlockHeader will contain the block header and ctx->jsonBlock will point to the allocated JSON data.
  entry | Pointer to the IndexEntry that specifies the file offset where the AaruMetadataJsonBlock begins. Must not be NULL. entry->offset indicates the position of the block header in the file.

Note
  JSON Format and Encoding:
  - The JSON payload is stored in UTF-8 encoding
  - The payload may or may not be null-terminated
  - This function treats the JSON as opaque binary data
  - No JSON parsing, interpretation, or validation is performed during loading
  - JSON schema validation is the responsibility of code that retrieves the metadata
  Aaru Metadata JSON Purpose:
  - Provides machine-readable structured metadata about the image
  - Includes comprehensive information about media, sessions, tracks, and checksums
  - Enables programmatic access to metadata without XML parsing overhead
  - Complements CICM XML with a more modern, structured format
  - Used by Aaru and compatible tools for metadata exchange
  Non-Critical Nature:
  - JSON metadata is optional and supplementary to core image data
  - Failures reading this block do not prevent image opening
  - The image remains fully functional for sector data access without JSON metadata
  - Higher-level code should check if ctx->jsonBlock is non-NULL before use
  Distinction from CICM XML:
  - Both CICM and Aaru JSON blocks can coexist in the same image
  - CICM XML follows the Canary Islands Computer Museum schema
  - Aaru JSON follows the Aaru-specific metadata schema
  - Different tools may prefer one format over the other

Warning
  Memory allocated for ctx->jsonBlock persists for the context lifetime and must be freed during context cleanup (aaruf_close).
  This function does not validate JSON syntax or schema. Corrupted JSON data will be loaded successfully and errors will only be detected when attempting to parse.

See also
  AaruMetadataJsonBlockHeader for the on-disk structure definition.
  process_cicm_block() for processing CICM XML metadata blocks.
  aaruf_open() for the overall image opening sequence.

Definition at line 475 of file metadata.c.

References AaruMetadataJsonBlock, FATAL, AaruMetadataJsonBlockHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, aaruformat_context::json_block, aaruformat_context::json_block_header, AaruMetadataJsonBlockHeader::length, IndexEntry::offset, and TRACE.

Referenced by aaruf_open().

## process_checksum_block()

void process_checksum_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a checksum block from the image stream.

Reads a checksum block, parses its entries, and stores the checksums (MD5, SHA1, SHA256, SpamSum) in the context.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the checksum block.

Definition at line 39 of file checksum.c.

References ChecksumBlock, aaruformat_context::checksums, ChecksumHeader::entries, FATAL, Checksums::hasMd5, Checksums::hasSha1, Checksums::hasSha256, Checksums::hasSpamSum, ChecksumHeader::identifier, aaruformat_context::imageStream, ChecksumEntry::length, ChecksumHeader::length, Md5, Checksums::md5, MD5_DIGEST_LENGTH, IndexEntry::offset, Sha1, Checksums::sha1, SHA1_DIGEST_LENGTH, Sha256, Checksums::sha256, SHA256_DIGEST_LENGTH, SpamSum, Checksums::spamsum, TRACE, and ChecksumEntry::type.

Referenced by aaruf_open().

## process_cicm_block()

void process_cicm_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a CICM XML metadata block from the image stream.

Reads a CICM XML metadata block from the image and updates the context with its contents.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the CICM block.

Definition at line 311 of file metadata.c.

References aaruformat_context::cicm_block, aaruformat_context::cicm_block_header, CicmBlock, FATAL, CicmMetadataBlock::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, CicmMetadataBlock::length, IndexEntry::offset, and TRACE.

Referenced by aaruf_open().

## process_data_block()

int32_t process_data_block | ( | aaruformat_context * | ctx,
|  | IndexEntry * | entry )

Processes a data block from the image stream.

Reads a data block from the image, decompresses if needed, and updates the context with its contents. This function handles various types of data blocks including compressed (LZMA) and uncompressed data, performs CRC validation, and stores the processed data in the appropriate context fields.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the data block.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully processed the data block. This is returned when:
  - The block is processed successfully and all validations pass
  - A NoData block type is encountered (these are skipped)
  - A UserData block type is encountered (these update sector size but are otherwise skipped)
  - Block validation fails but processing continues (non-fatal errors like CRC mismatches)
  - Memory allocation failures occur (processing continues with other blocks)
  - Block reading failures occur (processing continues with other blocks)
  - Unknown compression types are encountered (block is skipped)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers).
  AARUF_ERROR_CANNOT_READ_BLOCK | (-7) Failed to seek to the block position in the image stream. This occurs when fseek() fails or the file position doesn't match the expected offset.

  AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK

  (-17) LZMA decompression failed. This can happen when:
  - The LZMA decoder returns an error code
  - The decompressed data size doesn't match the expected block length

Note
  Most validation and reading errors are treated as non-fatal and result in AARUF_STATUS_OK being returned while the problematic block is skipped. This allows processing to continue with other blocks in the image.
  The function performs the following validations:
  - Block identifier matches the expected block type
  - Block data type matches the expected data type
  - CRC64 checksum validation (with version-specific byte order handling)
  - Proper decompression for LZMA-compressed blocks

Warning
  Memory allocated for block data is stored in the context and should be freed when the context is destroyed. The function may replace existing data in the context.

Definition at line 71 of file data.c.

References aaruf_crc64_data(), aaruf_cst_untransform(), AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK, AARUF_ERROR_CANNOT_READ_BLOCK, AARUF_ERROR_NOT_AARUFORMAT, aaruf_get_media_tag_type_for_datatype(), aaruf_lzma_decode_buffer(), AARUF_STATUS_OK, AARUF_VERSION_V1, AppleProfileTag, AppleProfileTagAaru, AppleSonyTag, AppleSonyTagAaru, AaruHeaderV2::biggestSectorSize, IndexEntry::blockType, bswap_64, CdSectorEcc, CdSectorEccP, CdSectorEccQ, CdSectorEdc, CdSectorHeader, CdSectorPrefix, CdSectorPrefixCorrected, CdSectorSubchannel, CdSectorSubchannelAaru, CdSectorSubHeader, CdSectorSuffix, CdSectorSuffixCorrected, CdSectorSync, BlockHeader::cmpLength, CompactDiscMode2Subheader, BlockHeader::compression, BlockHeader::crc64, mediaTagEntry::data, IndexEntry::dataType, DvdCmi, DvdSectorCprMai, DvdSectorEdc, DvdSectorEdcAaru, DvdSectorId, DvdSectorIed, DvdSectorIedAaru, DvdSectorInformation, DvdSectorNumber, DvdSectorTitleKeyDecrypted, DvdTitleKeyDecrypted, FATAL, aaruformat_context::header, BlockHeader::identifier, aaruformat_context::image_info, AaruHeaderV2::imageMajorVersion, ImageInfo::ImageSize, aaruformat_context::imageStream, BlockHeader::length, mediaTagEntry::length, Lzma, LZMA_PROPERTIES_LENGTH, LzmaClauniaSubchannelTransform, aaruformat_context::mediaTags, aaruformat_context::mode2_subheaders, NoData, None, IndexEntry::offset, PriamDataTowerTag, PriamDataTowerTagAaru, aaruformat_context::readableSectorTags, aaruformat_context::sector_cpr_mai, aaruformat_context::sector_decrypted_title_key, aaruformat_context::sector_edc, aaruformat_context::sector_id, aaruformat_context::sector_ied, aaruformat_context::sector_prefix, aaruformat_context::sector_prefix_corrected, aaruformat_context::sector_subchannel, aaruformat_context::sector_suffix, aaruformat_context::sector_suffix_corrected, ImageInfo::SectorSize, BlockHeader::sectorSize, TRACE, BlockHeader::type, mediaTagEntry::type, and UserData.

Referenced by aaruf_open().

## process_ddt_v1()

int32_t process_ddt_v1 | ( | aaruformat_context * | ctx,
|  | IndexEntry * | entry,
|  | bool * | found_user_data_ddt )

Processes a DDT v1 block from the image stream.

Reads and decompresses (if needed) a DDT v1 block, verifies its integrity, and loads it into memory or maps it. This function handles both user data DDT blocks and CD sector prefix/suffix corrected DDT blocks, supporting both LZMA compression and uncompressed formats. On Linux, uncompressed blocks can be memory-mapped for improved performance.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the DDT block.
  found_user_data_ddt | Pointer to a boolean that will be set to true if a user data DDT was found and loaded.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully processed the DDT block. This is returned when:
  - The DDT block is successfully read, decompressed (if needed), and loaded into memory
  - Memory mapping of uncompressed DDT succeeds on Linux systems
  - CD sector prefix/suffix corrected DDT blocks are processed successfully
  - Memory allocation failures occur for non-critical operations (processing continues)
  - File reading errors occur for compressed data or LZMA properties (processing continues)
  - Unknown compression types are encountered (block is skipped)
  - Memory mapping fails on Linux (sets found_user_data_ddt to false but continues)
  - Uncompressed DDT is encountered on non-Linux systems (not yet implemented, continues)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers).

  AARUF_ERROR_CANNOT_READ_BLOCK

  (-7) Failed to access the DDT block in the image stream. This occurs when:
  - fseek() fails to position at the DDT block offset
  - The file position doesn't match the expected offset after seeking
  - Failed to read the DDT header from the image stream
  - The number of bytes read for the DDT header is insufficient

  AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK

  (-17) LZMA decompression failed. This can happen when:
  - The LZMA decoder returns a non-zero error code during decompression
  - The decompressed data size doesn't match the expected DDT block length
  - This error causes immediate function termination and memory cleanup

Note
  The function exhibits different error handling strategies depending on the operation:
  - Critical errors (seek failures, header read failures, decompression failures) cause immediate return
  - Non-critical errors (memory allocation failures, unknown compression types) allow processing to continue
  - The found_user_data_ddt flag is updated to reflect the success of user data DDT loading
  Memory Management:
  - Allocated DDT data is stored in the context (ctx->userDataDdt, ctx->sectorPrefixDdt, ctx->sectorSuffixDdt)
  - On Linux, uncompressed DDTs may be memory-mapped instead of allocated
  - Memory is automatically cleaned up on decompression errors
  Platform-specific behavior:
  - Linux: Supports memory mapping of uncompressed DDT blocks for better performance
  - Non-Linux: Uncompressed DDT processing is not yet implemented

Warning
  The function modifies context state including sector count, shift value, and DDT version. Ensure proper context cleanup when the function completes.

Definition at line 85 of file ddt_v1.c.

References AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK, AARUF_ERROR_CANNOT_READ_BLOCK, AARUF_ERROR_NOT_AARUFORMAT, aaruf_lzma_decode_buffer(), AARUF_STATUS_OK, CdSectorPrefixCorrected, CdSectorSuffixCorrected, DdtHeader::cmpLength, DdtHeader::compression, IndexEntry::dataType, aaruformat_context::ddt_version, DdtHeader::entries, FATAL, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, aaruformat_context::in_memory_ddt, DdtHeader::length, Lzma, LZMA_PROPERTIES_LENGTH, None, IndexEntry::offset, aaruformat_context::sector_prefix_ddt, aaruformat_context::sector_suffix_ddt, ImageInfo::Sectors, aaruformat_context::shift, DdtHeader::shift, TRACE, aaruformat_context::user_data_ddt, and UserData.

Referenced by aaruf_open().

## process_ddt_v2()

int32_t process_ddt_v2 | ( | aaruformat_context * | ctx,
|  | IndexEntry * | entry,
|  | bool * | found_user_data_ddt )

Processes a DDT v2 block from the image stream.

Reads and decompresses (if needed) a DDT v2 block, verifies its CRC, and loads it into memory. This function handles both user data DDT blocks and CD sector prefix/suffix corrected DDT blocks, supporting both LZMA compression and uncompressed formats. It performs CRC64 validation and stores the processed DDT data in the appropriate context fields based on size type (small/big).

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the DDT block.
  found_user_data_ddt | Pointer to a boolean that will be set to true if a user data DDT was found and loaded.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully processed the DDT block. This is returned when:
  - The DDT block is successfully read, decompressed (if needed), and loaded into memory
  - CRC64 validation passes for the DDT data
  - User data DDT blocks are processed and context is properly updated
  - CD sector prefix/suffix corrected DDT blocks are processed successfully
  - Memory allocation failures occur for non-critical operations (processing continues)
  - File reading errors occur for compressed data or LZMA properties (processing continues)
  - Unknown compression types are encountered (block is skipped)

  AARUF_ERROR_NOT_AARUFORMAT | (-1) The context or image stream is invalid (NULL pointers).

  AARUF_ERROR_CANNOT_READ_BLOCK

  (-7) Failed to access the DDT block in the image stream. This occurs when:
  - fseek() fails to position at the DDT block offset
  - The file position doesn't match the expected offset after seeking
  - Failed to read the DDT header from the image stream
  - The number of bytes read for the DDT header is insufficient
  - CRC64 context initialization fails (internal error)

  AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK

  (-17) LZMA decompression failed. This can happen when:
  - The LZMA decoder returns a non-zero error code during decompression
  - The decompressed data size doesn't match the expected DDT block length
  - This error causes immediate function termination and memory cleanup

  AARUF_ERROR_INVALID_BLOCK_CRC

  (-18) CRC64 validation failed. This occurs when:
  - Calculated CRC64 doesn't match the expected CRC64 in the DDT header
  - Data corruption is detected in the DDT block
  - This applies to both compressed and uncompressed DDT blocks

Note
  Error Handling Strategy:
  - Critical errors (seek failures, header read failures, decompression failures, CRC failures) cause immediate return
  - Non-critical errors (memory allocation failures, unknown compression types) allow processing to continue
  - The found_user_data_ddt flag is updated to reflect the success of user data DDT loading
  DDT v2 Features:
  - Handles multi-level DDT hierarchies with tableShift parameter
  - Updates context with sector counts, DDT version, and primary DDT offset
  - Stores DDT data in size-appropriate context fields (userDataDdtMini/Big, sectorPrefixDdt, etc.)
  Memory Management:
  - Allocated DDT data is stored in the context and becomes part of the context lifecycle
  - Memory is automatically cleaned up on decompression or CRC validation errors
  - Buffer memory is reused for the final DDT data storage (no double allocation)
  CRC Validation:
  - All DDT blocks undergo CRC64 validation regardless of compression type
  - CRC is calculated on the final decompressed data
  - Uses standard CRC64 calculation (no version-specific endianness conversion like v1)

Warning
  The function modifies context state including sector count, DDT version, and primary DDT offset. Ensure proper context cleanup when the function completes.
  Memory allocated for DDT data becomes part of the context and should not be freed separately. The context cleanup functions will handle DDT memory deallocation.

Definition at line 96 of file ddt_v2.c.

References aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_DECOMPRESS_BLOCK, AARUF_ERROR_CANNOT_READ_BLOCK, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, aaruf_lzma_decode_buffer(), AARUF_STATUS_OK, DdtHeader2::blocks, CdSectorPrefix, CdSectorSuffix, DdtHeader2::cmpLength, DdtHeader2::compression, DdtHeader2::crc64, IndexEntry::dataType, aaruformat_context::ddt_version, FATAL, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, aaruformat_context::in_memory_ddt, DdtHeader2::length, Lzma, LZMA_PROPERTIES_LENGTH, DdtHeader2::negative, None, IndexEntry::offset, DdtHeader2::overflow, aaruformat_context::primary_ddt_offset, aaruformat_context::sector_prefix_ddt2, aaruformat_context::sector_suffix_ddt2, ImageInfo::Sectors, TRACE, aaruformat_context::user_data_ddt2, aaruformat_context::user_data_ddt_header, and UserData.

Referenced by aaruf_open().

## process_dumphw_block()

void process_dumphw_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a dump hardware block from the image stream.

Reads a dump hardware block from the image and updates the context with its contents.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the dump hardware block.

Definition at line 107 of file dump.c.

References aaruf_crc64_data(), AARUF_VERSION_V1, IndexEntry::blockType, bswap_64, compare_extents(), DumpHardwareHeader::crc64, aaruformat_context::dump_hardware_entries_with_data, aaruformat_context::dump_hardware_header, DumpHardwareBlock, DumpHardwareHeader::entries, DumpHardwareEntriesWithData::entry, DumpHardwareEntriesWithData::extents, DumpHardwareEntry::extents, FATAL, DumpHardwareEntriesWithData::firmware, DumpHardwareEntry::firmwareLength, free_dump_hardware_entries_array(), aaruformat_context::header, DumpHardwareHeader::identifier, AaruHeaderV2::imageMajorVersion, aaruformat_context::imageStream, DumpHardwareHeader::length, DumpHardwareEntriesWithData::manufacturer, DumpHardwareEntry::manufacturerLength, DumpHardwareEntriesWithData::model, DumpHardwareEntry::modelLength, IndexEntry::offset, read_dump_string(), reset_dump_hardware_context(), DumpHardwareEntriesWithData::revision, DumpHardwareEntry::revisionLength, DumpHardwareEntriesWithData::serial, DumpHardwareEntry::serialLength, DumpHardwareEntriesWithData::softwareName, DumpHardwareEntry::softwareNameLength, DumpHardwareEntriesWithData::softwareOperatingSystem, DumpHardwareEntry::softwareOperatingSystemLength, DumpHardwareEntriesWithData::softwareVersion, DumpHardwareEntry::softwareVersionLength, and TRACE.

Referenced by aaruf_open().

## process_flux_data_block()

void process_flux_data_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Parse and integrate a Flux Data block from the image stream into the context.

This function seeks to the byte offset specified by the supplied `entry`, reads a FluxHeader followed by the declared number of FluxEntry records, validates the block through its CRC64, rebuilds the flux capture lookup map, and populates multiple fields in the provided `ctx:`

- ctx->flux_data_header (identifier, entries, crc64)
- ctx->flux_entries (raw array with ALL flux entries in on-disk order)
- ctx->flux_capture_map (hash table mapping (head, track, subtrack, captureIndex) to entry indices)
- ctx->imageInfo.ImageSize (incremented by sizeof(FluxEntry) * entries)

Before reading new data, the function performs cleanup of any existing flux-related state:

- Frees and clears ctx->flux_captures if it exists (write-mode remnants)
- Frees and clears ctx->flux_entries if it exists (from previous calls or partial reads)
- Clears ctx->flux_capture_map (via flux_map_clear())
- Zeros ctx->flux_data_header

After successfully reading and validating the flux entries, the function rebuilds the in-memory flux capture lookup map by calling flux_map_rebuild_from_entries(). This map enables O(1) lookup of flux captures by their identifiers (head, track, subtrack, captureIndex), which is used by aaruf_read_flux_capture() and other flux access functions.

The function is intended for internal library use and is NOT part of the stable public API (no versioning guarantees). It is called on-demand (lazy loading) when flux data is first accessed, rather than during image opening, to avoid unnecessary I/O if flux data is never used.

Processing Flow:

1. Validation: Check ctx and ctx->imageStream are valid
2. Cleanup: Free existing flux_captures, flux_entries, and clear flux_capture_map
3. Seek: Seek to entry->offset in the image stream
4. Read Header: Read FluxHeader structure
5. Validate Identifier: Verify identifier == FluxDataBlock
6. Allocate Entries: Allocate memory for flux_entries array
7. Read Entries: Read all FluxEntry structures from the stream
8. Validate CRC64: Compute and verify CRC64 checksum over entries
9. Rebuild Map: Call flux_map_rebuild_from_entries() to build lookup map
10. Update ImageSize: Increment ctx->image_info.ImageSize

Error & early-return behavior (no exception mechanism, all via logging + early return):

- NULL `ctx` or NULL ctx->imageStream: Logs FATAL and returns immediately; context left untouched.
- Seek failure: FATAL + return; context left untouched (cleanup already performed).
- FluxHeader read short: flux_data_header zeroed, TRACE logged, return.
- Identifier mismatch: flux_data_header zeroed, TRACE logged, return.
- Allocation failure for flux_entries: flux_data_header zeroed, FATAL logged, return.
- Short read of FluxEntry array: flux_data_header zeroed, allocated flux_entries freed, FATAL logged, return.
- CRC mismatch: TRACE logged and return; (NOTE: at this point flux_entries remain allocated and flux_data_header retains the just-read values, but the lookup map is not built, so entries are not accessible via lookup functions. Caller may wish to discard them or trigger re-read.)
- Map rebuild failure: flux_entries freed, flux_data_header zeroed, FATAL logged, return.

Memory management:

- Frees any pre-existing ctx->flux_captures (utarray) before processing
- Frees any pre-existing ctx->flux_entries before allocating new ones
- Allocates ctx->flux_entries with malloc() sized to entries * sizeof(FluxEntry)
- On certain failure paths (short reads, map rebuild failure) allocated memory is freed
- On CRC mismatch, allocated memory is kept but map is not built (entries not accessible via lookup)
- The function is idempotent in terms of memory: it cleans up before processing, so repeated calls will not leak memory. However, the function is expected to be called exactly once per context lifetime during image opening.

Flux Capture Lookup Map: After successfully reading and validating flux entries, the function rebuilds the in-memory hash table (ctx->flux_capture_map) that maps flux capture identifiers to array indices. This enables efficient lookup of flux captures by their (head, track, subtrack, captureIndex) tuple. The map is built using UTHASH and is used by aaruf_read_flux_capture() and other flux access functions. If map rebuild fails, the function cleans up all allocated resources and returns.

Thread safety:

- Not thread-safe: mutates shared state in `ctx` without synchronization.
- Must not be called concurrently with readers/writers referencing the same context.

Preconditions (

Precondition
  ):
  - `ctx` != NULL
  - `ctx->imageStream` is a valid FILE* opened for reading at least up to the block region.
  - `entry` != NULL and entry->offset points to the start of a well-formed Flux Data block.

Postconditions (

Postcondition
  ) on success (CRC valid and map rebuilt):
  - ctx->flux_data_header.identifier == FluxDataBlock
  - ctx->flux_data_header.entries > 0 implies ctx->flux_entries != NULL
  - ctx->flux_capture_map is populated with entries mapping identifiers to indices
  - ctx->imageInfo.ImageSize incremented by flux data size
  - ctx->flux_captures == NULL (cleared if it existed)

Limitations / Caveats:

- No explicit status code: callers infer success by inspecting ctx->flux_data_header.entries and presence of ctx->flux_entries after invocation.
- In case of CRC mismatch, flux_entries are retained but flux_capture_map is not built, so entries are not accessible via lookup functions. Caller may wish to discard them or trigger re-read.
- The function is idempotent in terms of memory (cleans up before processing), but is expected to be called exactly once per context lifetime during image opening.

Logging strategy:

- FATAL used for unrecoverable structural or resource errors (seek failure, allocation failure, map rebuild failure).
- TRACE used for informational / soft failures (e.g., CRC mismatch, identifier mismatch, short read).

Parameters

  ctx | Mutable pointer to an aaruformatContext receiving parsed flux metadata.
  entry | Pointer to the index entry describing this Flux Data block (offset required; size not strictly used beyond informational logging and sequential reading).

Returns
  void This function does not return a status code; errors are reported via logging side effects.

Warning
  Absence of a returned status requires defensive post-call validation by the caller.
  CRC mismatch leaves possibly invalid data in ctx->flux_entries, and the lookup map is not built, making entries inaccessible via lookup functions.
  Map rebuild failure results in complete cleanup of flux-related state, leaving the context without flux data even if the entries were successfully read and validated.

See also
  flux_map_rebuild_from_entries() for the map rebuilding logic
  flux_map_clear() for the map clearing logic
  aaruf_read_flux_capture() for using the lookup map to access flux captures

Definition at line 307 of file flux.c.

References aaruf_crc64_data(), AARUF_STATUS_OK, FluxHeader::crc64, FluxHeader::entries, FATAL, aaruformat_context::flux_captures, aaruformat_context::flux_data_header, aaruformat_context::flux_entries, flux_map_clear(), flux_map_rebuild_from_entries(), FluxDataBlock, FluxHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, IndexEntry::offset, and TRACE.

Referenced by ensure_flux_entries_loaded().

## process_geometry_block()

void process_geometry_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a logical geometry block from the image stream.

Reads a logical geometry block from the image and updates the context with its contents.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the geometry block.

Definition at line 251 of file metadata.c.

References aaruformat_context::cylinders, GeometryBlockHeader::cylinders, FATAL, aaruformat_context::geometry_block, GeometryBlock, aaruformat_context::heads, GeometryBlockHeader::heads, GeometryBlockHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, IndexEntry::offset, aaruformat_context::sectors_per_track, GeometryBlockHeader::sectorsPerTrack, and TRACE.

Referenced by aaruf_open().

## process_index_v1()

UT_array * process_index_v1 | ( | aaruformat_context * | ctx | ) |

Processes an index block (version 1) from the image stream.

Reads and parses an index block (version 1) from the image, returning an array of index entries. This function handles the legacy index format used in early AaruFormat versions, providing compatibility with older image files. It reads the IndexHeader structure followed by a sequential list of IndexEntry structures, validating the index identifier for format correctness.

Parameters

  ctx | Pointer to the aaruformat context containing the image stream and header information.

Returns
  Returns one of the following values:

Return values

  UT_array*

  Successfully processed the index block. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from the position specified in ctx->header.indexOffset
  - The index identifier matches IndexBlock (legacy format identifier)
  - All index entries are successfully read and stored in the UT_array
  - Memory allocation for the index entries array succeeds
  - The returned array contains all index entries from the version 1 index block

  NULL

  Index processing failed. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid
  - Cannot read the IndexHeader structure from the image stream
  - The index identifier doesn't match IndexBlock (incorrect format or corruption)
  - Memory allocation fails for the UT_array structure
  - File I/O errors occur while reading index entries

Note
  Index Structure (Version 1):
  - IndexHeader: Contains identifier (IndexBlock), entry count, and metadata
  - IndexEntry array: Sequential list of entries describing block locations and types
  - No CRC validation is performed during processing (use verify_index_v1 for validation)
  - No compression support in version 1 index format
  Memory Management:
  - Returns a newly allocated UT_array that must be freed by the caller using utarray_free()
  - On error, any partially allocated memory is cleaned up before returning NULL
  - Each IndexEntry is copied into the array (no reference to original stream data)
  Version Compatibility:
  - Supports only legacy IndexBlock format (not IndexBlock2 or IndexBlock3)
  - Compatible with early AaruFormat and DiscImageChef image files
  - Does not handle subindex or hierarchical index structures

Warning
  The caller is responsible for freeing the returned UT_array using utarray_free(). Failure to free the array will result in memory leaks.
  This function does not validate the CRC integrity of the index data. Use verify_index_v1() to ensure index integrity before processing.
  The function assumes ctx->header.indexOffset points to a valid index block. Invalid offsets may cause file access errors or reading incorrect data.

Definition at line 79 of file index_v1.c.

References IndexHeader::entries, FATAL, aaruformat_context::header, IndexHeader::identifier, aaruformat_context::imageStream, IndexBlock, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_open(), and aaruf_verify_image().

## process_index_v2()

UT_array * process_index_v2 | ( | aaruformat_context * | ctx | ) |

Processes an index block (version 2) from the image stream.

Reads and parses an index block (version 2) from the image, returning an array of index entries. This function handles the intermediate index format used in mid-generation AaruFormat versions, providing compatibility with version 2 image files. It reads the IndexHeader2 structure followed by a sequential list of IndexEntry structures, validating the index identifier for format correctness.

Parameters

  ctx | Pointer to the aaruformat context containing the image stream and header information.

Returns
  Returns one of the following values:

Return values

  UT_array*

  Successfully processed the index block. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from the position specified in ctx->header.indexOffset
  - The index identifier matches IndexBlock2 (version 2 format identifier)
  - All index entries are successfully read and stored in the UT_array
  - Memory allocation for the index entries array succeeds
  - The returned array contains all index entries from the version 2 index block

  NULL

  Index processing failed. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid
  - Cannot read the IndexHeader2 structure from the image stream
  - The index identifier doesn't match IndexBlock2 (incorrect format or corruption)
  - Memory allocation fails for the UT_array structure
  - File I/O errors occur while reading index entries

Note
  Index Structure (Version 2):
  - IndexHeader2: Contains identifier (IndexBlock2), entry count, and enhanced metadata
  - IndexEntry array: Sequential list of entries describing block locations and types
  - No CRC validation is performed during processing (use verify_index_v2 for validation)
  - No compression support in version 2 index format
  - Compatible with mid-generation AaruFormat improvements
  Memory Management:
  - Returns a newly allocated UT_array that must be freed by the caller using utarray_free()
  - On error, any partially allocated memory is cleaned up before returning NULL
  - Each IndexEntry is copied into the array (no reference to original stream data)
  Version Compatibility:
  - Supports only IndexBlock2 format (not IndexBlock or IndexBlock3)
  - Compatible with intermediate AaruFormat image files
  - Does not handle subindex or hierarchical index structures (introduced in v3)

Warning
  The caller is responsible for freeing the returned UT_array using utarray_free(). Failure to free the array will result in memory leaks.
  This function does not validate the CRC integrity of the index data. Use verify_index_v2() to ensure index integrity before processing.
  The function assumes ctx->header.indexOffset points to a valid index block. Invalid offsets may cause file access errors or reading incorrect data.

Definition at line 81 of file index_v2.c.

References IndexHeader2::entries, FATAL, aaruformat_context::header, IndexHeader2::identifier, aaruformat_context::imageStream, IndexBlock2, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_open(), and aaruf_verify_image().

## process_index_v3()

UT_array * process_index_v3 | ( | aaruformat_context * | ctx | ) |

Processes an index block (version 3) from the image stream.

Reads and parses an index block (version 3) from the image, returning an array of index entries. This function handles the advanced index format used in current AaruFormat versions, supporting hierarchical subindex structures for improved scalability. It reads the IndexHeader3 structure followed by index entries, recursively processing any subindex blocks encountered to create a flattened array of all index entries.

Parameters

  ctx | Pointer to the aaruformat context containing the image stream and header information.

Returns
  Returns one of the following values:

Return values

  UT_array*

  Successfully processed the index block and all subindexes. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from the position specified in ctx->header.indexOffset
  - The index identifier matches IndexBlock3 (version 3 format identifier)
  - All index entries are successfully read and stored in the UT_array
  - Any subindex blocks (IndexBlock3 entries) are recursively processed via add_subindex_entries()
  - Memory allocation for the index entries array succeeds
  - The returned array contains all index entries from the main index and all subindexes

  NULL

  Index processing failed. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid
  - Cannot read the IndexHeader3 structure from the image stream
  - The index identifier doesn't match IndexBlock3 (incorrect format or corruption)
  - Memory allocation fails for the UT_array structure
  - File I/O errors occur while reading index entries
  - Recursive subindex processing fails (errors in add_subindex_entries())

Note
  Index Structure (Version 3):
  - IndexHeader3: Contains identifier (IndexBlock3), entry count, and advanced metadata
  - IndexEntry array: May contain regular entries and subindex references (IndexBlock3 type)
  - Hierarchical support: Subindex entries are recursively processed to flatten the structure
  - No CRC validation is performed during processing (use verify_index_v3 for validation)
  - Supports scalable index organization for large image files
  Subindex Processing:
  - When an IndexEntry has blockType == IndexBlock3, it references a subindex
  - Subindexes are recursively processed using add_subindex_entries()
  - All entries from subindexes are flattened into the main index entries array
  - Supports arbitrary nesting depth of subindexes
  Memory Management:
  - Returns a newly allocated UT_array that must be freed by the caller using utarray_free()
  - On error, any partially allocated memory is cleaned up before returning NULL
  - Each IndexEntry is copied into the array (no reference to original stream data)
  - Memory usage scales with total entries across all subindexes
  Version Compatibility:
  - Supports only IndexBlock3 format (not IndexBlock or IndexBlock2)
  - Compatible with current generation AaruFormat image files
  - Backward compatible with images that don't use subindexes

Warning
  The caller is responsible for freeing the returned UT_array using utarray_free(). Failure to free the array will result in memory leaks.
  This function does not validate the CRC integrity of the index data. Use verify_index_v3() to ensure index integrity before processing.
  Recursive subindex processing may cause significant memory usage and processing time for images with deeply nested or numerous subindexes.
  The function assumes ctx->header.indexOffset points to a valid index block. Invalid offsets may cause file access errors or reading incorrect data.

Definition at line 98 of file index_v3.c.

References add_subindex_entries(), IndexEntry::blockType, IndexHeader3::entries, FATAL, aaruformat_context::header, IndexHeader3::identifier, aaruformat_context::imageStream, IndexBlock3, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_open(), and aaruf_verify_image().

## process_metadata_block()

void process_metadata_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a metadata block from the image stream.

Reads a metadata block from the image and updates the context with its contents.

Parameters

  ctx | Pointer to the aaruformat context.
  entry | Pointer to the index entry describing the metadata block.

Definition at line 35 of file metadata.c.

References MetadataBlockHeader::blockSize, IndexEntry::blockType, aaruformat_context::comments, MetadataBlockHeader::commentsLength, MetadataBlockHeader::commentsOffset, aaruformat_context::creator, MetadataBlockHeader::creatorLength, MetadataBlockHeader::creatorOffset, aaruformat_context::drive_firmware_revision, aaruformat_context::drive_manufacturer, aaruformat_context::drive_model, aaruformat_context::drive_serial_number, MetadataBlockHeader::driveFirmwareRevisionLength, MetadataBlockHeader::driveFirmwareRevisionOffset, MetadataBlockHeader::driveManufacturerLength, MetadataBlockHeader::driveManufacturerOffset, MetadataBlockHeader::driveModelLength, MetadataBlockHeader::driveModelOffset, MetadataBlockHeader::driveSerialNumberLength, MetadataBlockHeader::driveSerialNumberOffset, FATAL, MetadataBlockHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, aaruformat_context::last_media_sequence, MetadataBlockHeader::lastMediaSequence, aaruformat_context::media_barcode, aaruformat_context::media_manufacturer, aaruformat_context::media_model, aaruformat_context::media_part_number, aaruformat_context::media_sequence, aaruformat_context::media_serial_number, aaruformat_context::media_title, MetadataBlockHeader::mediaBarcodeLength, MetadataBlockHeader::mediaBarcodeOffset, MetadataBlockHeader::mediaManufacturerLength, MetadataBlockHeader::mediaManufacturerOffset, MetadataBlockHeader::mediaModelLength, MetadataBlockHeader::mediaModelOffset, MetadataBlockHeader::mediaPartNumberLength, MetadataBlockHeader::mediaPartNumberOffset, MetadataBlockHeader::mediaSequence, MetadataBlockHeader::mediaSerialNumberLength, MetadataBlockHeader::mediaSerialNumberOffset, MetadataBlockHeader::mediaTitleLength, MetadataBlockHeader::mediaTitleOffset, aaruformat_context::metadata_block, aaruformat_context::metadata_block_header, IndexEntry::offset, and TRACE.

Referenced by aaruf_open().

## process_tape_files_block()

void process_tape_files_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a tape file metadata block from the image stream.

Reads and parses a TapeFileBlock from the Aaru image, validates its integrity, and populates the context's tape file hash table with file layout information. Each tape file entry defines a logical file on the tape medium by specifying its partition, file number, and block range (FirstBlock to LastBlock inclusive).

The function performs the following operations:

1. Seeks to the block position indicated by the index entry
2. Reads and validates the TapeFileHeader structure
3. Allocates and reads the array of TapeFileEntry structures
4. Validates data integrity using CRC64-ECMA checksum
5. Inserts each file entry into the context's UTHASH table with a composite key
6. Updates image size statistics

Composite Key Construction: Each tape file is uniquely identified by a 64-bit composite key: key = (partition << 32) | file_number This allows files with the same file number in different partitions to coexist in the hash table without conflicts.

Hash Table Management: The function uses HASH_REPLACE to insert entries, which automatically:

- Adds new entries if the key doesn't exist
- Replaces existing entries if the key is found (freeing the old entry) This ensures that duplicate entries (same partition/file combination) are properly handled by keeping only the most recent definition.

Error Handling: The function treats most errors as non-fatal and continues processing:

- Invalid context or stream: Returns immediately (FATAL log)
- Seek failures: Returns immediately (FATAL log)
- Header read failures: Returns early (TRACE log)
- Incorrect block identifier: Logs warning but continues
- Memory allocation failures: Logs error and returns
- Entry read failures: Frees buffer and returns
- CRC64 mismatch: Logs warning, frees buffer, and returns
- Per-entry allocation failures: Logs error and skips that entry

Block Structure: The tape file block consists of:

+-------------------------+
| TapeFileHeader (24 B) | <- identifier, entries, length, crc64
+-------------------------+
| TapeFileEntry 0 (21 B) | <- File, Partition, FirstBlock, LastBlock
| TapeFileEntry 1 (21 B) |
| ... |
| TapeFileEntry (n-1) |
+-------------------------+
TapeFileEntry
Describes a single logical file on a tape medium.
Definition tape.h:134
TapeFileHeader
struct TapeFileHeader TapeFileHeader
TapeFileEntry
struct TapeFileEntry TapeFileEntry

CRC64 Validation: The CRC64 checksum in the header is computed over the entire array of TapeFileEntry structures (excluding the header itself). This provides integrity verification to detect corruption in the file table.

Memory Management:

- Allocates a temporary buffer to read all file entries
- Allocates individual hash table entries for each file
- Frees the temporary buffer before returning
- Frees replaced hash entries automatically
- Hash table entries remain in context until cleanup

Parameters

  ctx | Pointer to the aaruformat context. Must not be NULL. The context must have a valid imageStream open for reading. The ctx->tapeFiles hash table will be populated with file entries. The ctx->imageInfo.ImageSize will be updated with the block size.
  entry | Pointer to the index entry describing the tape file block. Must not be NULL. The entry->offset field indicates the file position where the TapeFileHeader begins.

Note
  This function does not return a status code. All errors are handled internally with appropriate logging and the function returns early on fatal errors.
  The tape file hash table (ctx->tapeFiles) must be initialized to NULL before the first call to this function. UTHASH will manage the table automatically as entries are added.
  Files are ordered in the hash table by their composite key value, not by insertion order. To iterate files in partition/file number order, use HASH_SORT with an appropriate comparison function.
  The function updates ctx->imageInfo.ImageSize by adding the size of all file entries (entries × sizeof(TapeFileEntry)). This contributes to the total reported image size but does not include the header size.

Warning
  The context and imageStream must be valid. Passing NULL pointers will result in immediate return with a FATAL log message.
  If the CRC64 checksum validation fails, all entries in the block are discarded. The function does not attempt partial recovery.
  If memory allocation fails for a hash entry, that specific file entry is skipped but processing continues with remaining entries.

See also
  TapeFileHeader for the block header structure
  TapeFileEntry for individual file entry structure
  tapeFileHashEntry for the hash table entry structure
  process_tape_partition_block() for partition metadata processing

Definition at line 126 of file tape.c.

References aaruf_crc64_data(), TapeFileHeader::crc64, aaruformat_context::dirty_tape_file_block, TapeFileHeader::entries, FATAL, TapeFileHashEntry::fileEntry, TapeFileHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, TapeFileHashEntry::key, IndexEntry::offset, aaruformat_context::tape_files, TapeFileBlock, and TRACE.

Referenced by aaruf_open().

## process_tape_partitions_block()

void process_tape_partitions_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Processes a tape partition metadata block from the image stream.

Reads and parses a TapePartitionBlock from the Aaru image, validates its integrity, and populates the context's tape partition hash table with partition layout information. Each tape partition entry defines a physical division of the tape medium by specifying its partition number and block range (FirstBlock to LastBlock inclusive).

The function performs the following operations:

1. Seeks to the block position indicated by the index entry
2. Reads and validates the TapePartitionHeader structure
3. Allocates and reads the array of TapePartitionEntry structures
4. Validates data integrity using CRC64-ECMA checksum
5. Inserts each partition entry into the context's UTHASH table with partition number as key
6. Updates image size statistics

Partition Identification: Each tape partition is uniquely identified by its partition number (0-255). This number serves as the hash table key for fast lookup operations. Most tapes have a single partition (partition 0), but multi-partition formats like LTO, DLT, and AIT support multiple partitions with independent block address spaces.

Hash Table Management: The function uses HASH_REPLACE to insert entries, which automatically:

- Adds new entries if the partition number doesn't exist
- Replaces existing entries if the partition number is found (freeing the old entry) This ensures that duplicate partition definitions are properly handled by keeping only the most recent definition.

Error Handling: The function treats most errors as non-fatal and continues processing:

- Invalid context or stream: Returns immediately (FATAL log)
- Seek failures: Returns immediately (FATAL log)
- Header read failures: Returns early (TRACE log)
- Incorrect block identifier: Logs warning but continues
- Memory allocation failures: Logs error and returns
- Entry read failures: Frees buffer and returns
- CRC64 mismatch: Logs warning, frees buffer, and returns
- Per-entry allocation failures: Logs error and skips that entry

Block Structure: The tape partition block consists of:

+-----------------------------+
| TapePartitionHeader (24 B) | <- identifier, entries, length, crc64
+-----------------------------+
| TapePartitionEntry 0 (17 B) | <- Number, FirstBlock, LastBlock
| TapePartitionEntry 1 (17 B) |
| ... |
| TapePartitionEntry (n-1) |
+-----------------------------+
TapePartitionEntry
Describes a single physical partition on a tape medium.
Definition tape.h:320
TapePartitionHeader
struct TapePartitionHeader TapePartitionHeader
TapePartitionEntry
struct TapePartitionEntry TapePartitionEntry

CRC64 Validation: The CRC64 checksum in the header is computed over the entire array of TapePartitionEntry structures (excluding the header itself). This provides integrity verification to detect corruption in the partition table.

Partition Block Ranges: Each partition defines a block address space:

- FirstBlock: Starting block address (often 0, but format-dependent)
- LastBlock: Ending block address (inclusive)
- Block count: (LastBlock - FirstBlock + 1)

Block addresses are local to each partition. Different partitions may have overlapping logical block numbers (e.g., both partition 0 and partition 1 can have blocks numbered 0-1000).

Memory Management:

- Allocates a temporary buffer to read all partition entries
- Allocates individual hash table entries for each partition
- Frees the temporary buffer before returning
- Frees replaced hash entries automatically
- Hash table entries remain in context until cleanup

Parameters

  ctx | Pointer to the aaruformat context. Must not be NULL. The context must have a valid imageStream open for reading. The ctx->tapePartitions hash table will be populated with partition entries. The ctx->imageInfo.ImageSize will be updated with the block size.
  entry | Pointer to the index entry describing the tape partition block. Must not be NULL. The entry->offset field indicates the file position where the TapePartitionHeader begins.

Note
  This function does not return a status code. All errors are handled internally with appropriate logging and the function returns early on fatal errors.
  The tape partition hash table (ctx->tapePartitions) must be initialized to NULL before the first call to this function. UTHASH will manage the table automatically as entries are added.
  Partitions are ordered in the hash table by their partition number, not by insertion order. To iterate partitions in numerical order, use HASH_SORT with an appropriate comparison function.
  The function updates ctx->imageInfo.ImageSize by adding the size of all partition entries (entries × sizeof(TapePartitionEntry)). This contributes to the total reported image size but does not include the header size.
  The partition metadata is essential for correctly interpreting tape file locations, as files reference partition numbers in their definitions. Without partition metadata, tape file block ranges may be ambiguous.

Warning
  The context and imageStream must be valid. Passing NULL pointers will result in immediate return with a FATAL log message.
  If the CRC64 checksum validation fails, all entries in the block are discarded. The function does not attempt partial recovery.
  If memory allocation fails for a hash entry, that specific partition entry is skipped but processing continues with remaining entries.
  If multiple partition entries have the same Number field, only the last occurrence is retained. This should not occur in valid images.

See also
  TapePartitionHeader for the block header structure
  TapePartitionEntry for individual partition entry structure
  TapePartitionHashEntry for the hash table entry structure
  process_tape_files_block() for tape file metadata processing

Definition at line 347 of file tape.c.

References aaruf_crc64_data(), TapePartitionHeader::crc64, aaruformat_context::dirty_tape_partition_block, TapePartitionHeader::entries, FATAL, TapePartitionHeader::identifier, aaruformat_context::image_info, ImageInfo::ImageSize, aaruformat_context::imageStream, TapePartitionHashEntry::key, TapePartitionEntry::Number, IndexEntry::offset, TapePartitionHashEntry::partitionEntry, aaruformat_context::tape_partitions, TapePartitionBlock, and TRACE.

Referenced by aaruf_open().

## process_tracks_block()

void process_tracks_block | ( | aaruformat_context * | ctx,
|  | const IndexEntry * | entry )

Parse and integrate a Tracks block from the image stream into the context.

This function seeks to the byte offset specified by the supplied `entry`, reads a TracksHeader followed by the declared number of TrackEntry records, validates the block through its CRC64, and populates multiple fields in the provided `ctx:`

- ctx->tracksHeader (identifier, entries, crc64)
- ctx->trackEntries (raw array with ALL tracks in on-disk order)
- ctx->dataTracks (array filtered to data track sequences in [1..99])
- ctx->numberOfDataTracks
- ctx->imageInfo.ImageSize (incremented by sizeof(TrackEntry) * entries)
- ctx->imageInfo.HasPartitions / HasSessions (both set true unconditionally on success path before filtering)

It also performs a legacy endian correction of the computed CRC for images whose major version is <= AARUF_VERSION_V1 (historical writer quirk).

The function is intended for internal library use during image opening / indexing and is NOT part of the stable public API (no versioning guarantees). Callers outside the library should use the higher-level image open helpers that trigger this parsing implicitly.

Error & early-return behavior (no exception mechanism, all via logging + early return):

- NULL `ctx` or NULL ctx->imageStream: Logs FATAL and returns immediately; context left untouched.
- Seek failure: FATAL + return; context left untouched.
- TracksHeader read short: tracksHeader zeroed, TRACE logged, return.
- Identifier mismatch: tracksHeader zeroed, TRACE logged, return.
- Allocation failure for trackEntries: tracksHeader zeroed, FATAL logged, return.
- Short read of TrackEntry array: tracksHeader zeroed, allocated trackEntries freed, FATAL logged, return.
- CRC mismatch: TRACE logged and return; (NOTE: at this point trackEntries remain allocated and tracksHeader retains the just-read values, but caller receives no explicit status.)

Memory management:

- Allocates ctx->trackEntries with malloc() sized to entries * sizeof(TrackEntry).
- Allocates ctx->dataTracks only if at least one filtered data track is found; otherwise sets it to NULL.
- On certain failure paths (short reads) allocated memory is freed; on CRC mismatch memory is kept.
- Pre-existing ctx->trackEntries or ctx->dataTracks are NOT freed before overwriting pointers, so repeated calls without prior cleanup will leak memory. The function is therefore expected to be called exactly once per context lifetime. This constraint should be observed by library code.

Filtering rule for ctx->dataTracks:

- Any TrackEntry with sequence in inclusive range [1, 99] is considered a data track (historical convention in format) and copied into ctx->dataTracks preserving original order.

Thread safety:

- Not thread-safe: mutates shared state in `ctx` without synchronization.
- Must not be called concurrently with readers/writers referencing the same context.

Preconditions (

Precondition
  ):
  - `ctx` != NULL
  - `ctx->imageStream` is a valid FILE* opened for reading at least up to the block region.
  - `entry` != NULL and entry->offset points to the start of a well-formed Tracks block.

Postconditions (

Postcondition
  ) on success (CRC valid):
  - ctx->tracksHeader.identifier == TracksBlock
  - ctx->tracksHeader.entries > 0 implies ctx->trackEntries != NULL
  - ctx->dataTracks != NULL iff numberOfDataTracks > 0
  - numberOfDataTracks <= tracksHeader.entries
  - imageInfo flags (HasPartitions, HasSessions) set true

Limitations / Caveats:

- No explicit status code: callers infer success by inspecting ctx->tracksHeader.entries and presence of ctx->trackEntries after invocation.
- In case of CRC mismatch trackEntries are retained though not guaranteed trustworthy; caller may wish to discard them or trigger re-read.
- Potential memory leak if invoked multiple times without freeing previous arrays.

Logging strategy:

- FATAL used for unrecoverable structural or resource errors.
- TRACE used for informational / soft failures (e.g., CRC mismatch, identifier mismatch, short read).

Parameters

  ctx | Mutable pointer to an aaruformatContext receiving parsed track metadata.
  entry | Pointer to the index entry describing this Tracks block (offset required; size not strictly used beyond informational logging and sequential reading).

Returns
  void This function does not return a status code; errors are reported via logging side effects.

Warning
  Not idempotent and not leak-safe if called more than once for a context.
  Absence of a returned status requires defensive post-call validation by the caller.
  CRC mismatch leaves possibly invalid data in ctx->trackEntries.

See also
  aaruf_get_tracks()
  aaruf_set_tracks()

Definition at line 111 of file optical.c.

References aaruf_crc64_data(), AARUF_VERSION_V1, bswap_64, TracksHeader::crc64, aaruformat_context::data_tracks, TracksHeader::entries, FATAL, ImageInfo::HasPartitions, ImageInfo::HasSessions, aaruformat_context::header, TracksHeader::identifier, aaruformat_context::image_info, AaruHeaderV2::imageMajorVersion, ImageInfo::ImageSize, aaruformat_context::imageStream, aaruformat_context::number_of_data_tracks, IndexEntry::offset, TrackEntry::sequence, TRACE, aaruformat_context::track_entries, aaruformat_context::tracks_header, and TracksBlock.

Referenced by aaruf_open().

## set_ddt_entry_v2()

bool set_ddt_entry_v2 | ( | aaruformat_context * | ctx,
|  | const uint64_t | sector_address,
|  | const bool | negative,
|  | const uint64_t | offset,
|  | const uint64_t | block_offset,
|  | const uint8_t | sector_status,
|  | uint64_t * | ddt_entry )

Sets a DDT v2 entry for a given sector address.

Updates the DDT v2 table(s) with the specified offset, block offset, and sector status for a sector.

Parameters

  ctx | Pointer to the aaruformat context.
  sector_address | Logical sector address to set.
  negative | Indicates if the sector address is negative.
  offset | Offset to set for the sector.
  block_offset | Block offset to set for the sector.
  sector_status | Status to set for the sector.
  ddt_entry | Existing DDT entry or 0 to create a new one. If 0, a new entry is returned.

Returns
  Returns one of the following status codes:

Return values

  true | if the entry was set successfully, false otherwise.

Definition at line 1002 of file ddt_v2.c.

References FATAL, aaruformat_context::imageStream, set_ddt_multi_level_v2(), set_ddt_single_level_v2(), DdtHeader2::tableShift, TRACE, and aaruformat_context::user_data_ddt_header.

Referenced by aaruf_write_sector().

## set_ddt_multi_level_v2()

bool set_ddt_multi_level_v2 | ( | aaruformat_context * | ctx,
|  | uint64_t | sector_address,
|  | bool | negative,
|  | uint64_t | offset,
|  | uint64_t | block_offset,
|  | uint8_t | sector_status,
|  | uint64_t * | ddt_entry )

Sets a multi-level DDT v2 entry for a given sector address.

Used when the DDT table uses multi-level indirection (tableShift > 0).

Parameters

  ctx | Pointer to the aaruformat context.
  sector_address | Logical sector address to set.
  negative | Indicates if the sector address is negative.
  offset | Offset to set for the sector.
  block_offset | Block offset to set for the sector.
  sector_status | Status to set for the sector.
  ddt_entry | Existing DDT entry or 0 to create a new one. If 0, a new entry is returned.

Returns
  Returns one of the following status codes:

Return values

  true | if the entry was set successfully, false otherwise.

Definition at line 1110 of file ddt_v2.c.

References aaruf_close_current_block(), aaruf_crc64_data(), aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_NOT_ENOUGH_MEMORY, aaruf_lzma_encode_buffer(), DdtHeader2::blockAlignmentShift, DdtHeader2::blocks, IndexEntry::blockType, aaruformat_context::cached_ddt_offset, aaruformat_context::cached_ddt_position, aaruformat_context::cached_secondary_ddt2, DdtHeader2::cmpCrc64, DdtHeader2::cmpLength, DdtHeader2::compression, aaruformat_context::compression_enabled, DdtHeader2::crc64, DdtHeader2::dataShift, IndexEntry::dataType, DeDuplicationTable2, DeDuplicationTableSAlpha, DeDuplicationTableSecondary, aaruformat_context::dirty_index_block, aaruformat_context::dirty_primary_ddt, aaruformat_context::dirty_secondary_ddt, DdtHeader2::entries, FATAL, DdtHeader2::identifier, aaruformat_context::imageStream, aaruformat_context::index_entries, DdtHeader2::length, DdtHeader2::levels, Lzma, aaruformat_context::lzma_dict_size, LZMA_PROPERTIES_LENGTH, DdtHeader2::negative, aaruformat_context::next_block_position, None, IndexEntry::offset, DdtHeader2::overflow, DdtHeader2::previousLevelOffset, aaruformat_context::primary_ddt_offset, DdtHeader2::start, DdtHeader2::tableLevel, DdtHeader2::tableShift, TRACE, DdtHeader2::type, aaruformat_context::user_data_ddt2, aaruformat_context::user_data_ddt_header, UserData, and aaruformat_context::writing_buffer.

Referenced by set_ddt_entry_v2().

## set_ddt_single_level_v2()

bool set_ddt_single_level_v2 | ( | aaruformat_context * | ctx,
|  | uint64_t | sector_address,
|  | const bool | negative,
|  | const uint64_t | offset,
|  | const uint64_t | block_offset,
|  | const uint8_t | sector_status,
|  | uint64_t * | ddt_entry )

Sets a single-level DDT v2 entry for a given sector address.

Used when the DDT table does not use multi-level indirection.

Parameters

  ctx | Pointer to the aaruformat context.
  sector_address | Logical sector address to set.
  negative | Indicates if the sector address is negative.
  offset | Offset to set for the sector.
  block_offset | Block offset to set for the sector.
  sector_status | Status to set for the sector.
  ddt_entry | Existing DDT entry or 0 to create a new one. If 0, a new entry is returned.

Returns
  Returns one of the following status codes:

Return values

  true | if the entry was set successfully, false otherwise.

Definition at line 1038 of file ddt_v2.c.

References DdtHeader2::blockAlignmentShift, DdtHeader2::dataShift, aaruformat_context::dirty_single_level_ddt, FATAL, aaruformat_context::imageStream, DdtHeader2::negative, DdtHeader2::tableShift, TRACE, aaruformat_context::user_data_ddt2, and aaruformat_context::user_data_ddt_header.

Referenced by set_ddt_entry_v2().

## set_ddt_tape()

bool set_ddt_tape | ( | aaruformat_context * | ctx,
|  | uint64_t | sector_address,
|  | const uint64_t | offset,
|  | const uint64_t | block_offset,
|  | const uint8_t | sector_status,
|  | uint64_t * | ddt_entry )

Sets a DDT entry for tape media using a hash-based lookup table.

This function is specifically designed for tape media images where sectors are accessed non-sequentially and the traditional DDT array structure is inefficient. Instead of using a large contiguous array, it uses a hash table (UTHASH) to store only the sectors that have been written, providing sparse storage for tape media.

The function performs the following operations:

1. Validates the context and verifies it's a tape image
2. Constructs a DDT entry encoding offset, block alignment, and sector status
3. Creates a hash table entry with the sector address as the key
4. Inserts or replaces the entry in the tape DDT hash table

DDT Entry Format: The DDT entry is a 64-bit value with the following bit layout:

Bits 0-(dataShift-1): Sector offset within block (masked by dataShift)
Bits dataShift-27: Block index (block_offset >> blockAlignmentShift)
Bits 28-31: Sector status (4 bits for status flags)
Bits 32-63: Unused (reserved for future use)

Hash Table Management: Uses HASH_REPLACE macro from UTHASH library which:

- Adds new entries if the key (sector_address) doesn't exist
- Replaces existing entries if the key is found (automatically frees old entry)
- Maintains O(1) average lookup time for sector address resolution

Overflow Detection: The function checks if the constructed DDT entry exceeds 28 bits (0xFFFFFFF). This limit ensures the sector status can fit in the upper 4 bits while leaving room for future extensions in the upper 32 bits.

Parameters

  ctx | Pointer to the aaruformat context. Must not be NULL. The context must have a valid imageStream and is_tape must be true. The ctx->tapeDdt hash table will be updated with the new entry. The ctx->userDataDdtHeader contains alignment and shift parameters.
  sector_address | Logical sector address on the tape to set. This serves as the unique key in the hash table. Multiple calls with the same sector_address will replace the previous entry.
  offset | Byte offset within the aligned block where the sector data begins. This value is masked by (1 << dataShift) - 1 to extract only the lower bits representing the offset within the block.
  block_offset | Absolute byte offset in the image file where the data block starts. This is right-shifted by blockAlignmentShift to get the block index, which is stored in the DDT entry's middle bits.

  sector_status

  Status flags for the sector (4 bits). Common values include:
  - 0x0 (SectorStatusNotDumped): Sector not yet acquired during image dumping
  - 0x1 (SectorStatusDumped): Sector successfully dumped without error
  - 0x2 (SectorStatusErrored): Error during dumping; data may be incomplete or corrupt
  - 0x3 (SectorStatusMode1Correct): Valid MODE 1 data with regenerable suffix/prefix
  - 0x4 (SectorStatusMode2Form1Ok): Suffix verified/regenerable for MODE 2 Form 1
  - 0x5 (SectorStatusMode2Form2Ok): Suffix matches MODE 2 Form 2 with valid CRC
  - 0x6 (SectorStatusMode2Form2NoCrc): Suffix matches MODE 2 Form 2 but CRC empty/missing
  - 0x7 (SectorStatusTwin): Pointer references a twin sector table
  - 0x8 (SectorStatusUnrecorded): Sector physically unrecorded; repeated reads non-deterministic
  - 0x9 (SectorStatusEncrypted): Content encrypted and stored encrypted in image
  - 0xA (SectorStatusUnencrypted): Content originally encrypted but stored decrypted in image See SectorStatus enum for complete list of defined values

  ddt_entry

  Pointer to a 64-bit value that will receive the constructed DDT entry.
  - If *ddt_entry is 0: A new entry is constructed from the provided parameters
  - If *ddt_entry is non-zero: The existing value is used directly The constructed or provided value is stored in the hash table.

Returns
  Returns one of the following status codes:

Return values

  true

  Successfully created and inserted the DDT entry. This occurs when:
  - The context and image stream are valid
  - The image is confirmed to be a tape image (is_tape == true)
  - The DDT entry fits within the 28-bit limit (< 0xFFFFFFF)
  - Memory allocation for the hash entry succeeds
  - The entry is successfully inserted or replaced in the hash table

  false

  Failed to set the DDT entry. This can happen when:
  - ctx is NULL or ctx->imageStream is NULL (invalid context)
  - ctx->is_tape is false (wrong function called for non-tape media)
  - The DDT entry exceeds 0xFFFFFFF (media too large for big DDT)
  - Memory allocation fails for the new hash table entry (out of memory)

Note
  This function is only for tape images. For disk images, use set_ddt_single_level_v2() or set_ddt_multi_level_v2() instead, which use array-based DDT structures.
  Memory Management:
  - Allocates a new TapeDdtHashEntry for each sector
  - HASH_REPLACE automatically frees replaced entries
  - All hash entries remain in context until cleanup
  - The tapeDdt hash table must be freed during context destruction
  Tape Media Characteristics:
  - Tape sectors are typically accessed sequentially during streaming
  - File marks and partition boundaries create sparse address spaces
  - Hash table provides efficient storage for sparse sector maps
  - Supports variable block sizes common in tape formats
  Error Handling:
  - All errors are logged with FATAL level messages
  - Function returns false immediately on any error condition
  - TRACE logging marks entry/exit points for debugging
  - No partial state changes occur on failure

Warning
  The DDT entry overflow check at 0xFFFFFFF (28 bits) is critical. Exceeding this limit indicates the media is too large to fit in the current DDT format, and continuing would cause data corruption.
  This function modifies the shared tapeDdt hash table. In multi-threaded environments, external synchronization is required to prevent race conditions.

See also
  TapeDdtHashEntry for the hash table entry structure
  set_ddt_entry_v2() for the main DDT entry point that dispatches to this function
  get_ddt_tape() for retrieving tape DDT entries from the hash table

Definition at line 1800 of file ddt_v2.c.

References DdtHeader2::blockAlignmentShift, DdtHeader2::dataShift, aaruformat_context::dirty_tape_ddt, FATAL, aaruformat_context::imageStream, aaruformat_context::is_tape, TapeDdtHashEntry::key, aaruformat_context::tape_ddt, TRACE, aaruformat_context::user_data_ddt_header, and TapeDdtHashEntry::value.

## verify_index_v1()

int32_t verify_index_v1 | ( | aaruformat_context * | ctx | ) |

Verifies the integrity of an index block (version 1) in the image stream.

Checks the CRC64 of the index block without decompressing it. This function performs comprehensive validation of the index structure including header validation, data integrity verification, and version-specific CRC calculation. It ensures the index block is valid and uncorrupted before the image can be safely used for data access.

Parameters

  ctx | Pointer to the aaruformat context containing image stream and header information.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully verified index integrity. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from ctx->header.indexOffset
  - The index identifier matches IndexBlock (version 1 format)
  - Memory allocation for index entries succeeds
  - All index entries are successfully read from the image stream
  - CRC64 calculation completes successfully with version-specific endianness handling
  - The calculated CRC64 matches the expected CRC64 in the index header

  AARUF_ERROR_NOT_AARUFORMAT

  (-1) Invalid context or stream. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid

  AARUF_ERROR_CANNOT_READ_HEADER

  (-6) Index header reading failed. This occurs when:
  - Cannot read the complete IndexHeader structure from the image stream
  - File I/O errors prevent accessing the header at ctx->header.indexOffset
  - Insufficient data available at the specified index offset

  AARUF_ERROR_CANNOT_READ_INDEX

  (-19) Index format or data access errors. This occurs when:
  - The index identifier doesn't match IndexBlock (wrong format or corruption)
  - Cannot read all index entries from the image stream
  - File I/O errors during index entry reading
  - Index structure is corrupted or truncated

  AARUF_ERROR_NOT_ENOUGH_MEMORY

  (-9) Memory allocation failed. This occurs when:
  - Cannot allocate memory for the index entries array
  - System memory exhaustion prevents loading index data for verification

  AARUF_ERROR_INVALID_BLOCK_CRC

  (-18) CRC64 validation failed. This occurs when:
  - The calculated CRC64 doesn't match the expected CRC64 in the index header
  - Index data corruption is detected
  - Data integrity verification fails indicating potential file damage

Note
  CRC64 Validation Process:
  - Reads all index entries into memory for CRC calculation
  - Calculates CRC64 over the complete index entries array
  - Applies version-specific endianness conversion for compatibility
  - For imageMajorVersion <= AARUF_VERSION_V1: Uses bswap_64() for byte order correction
  - Compares calculated CRC64 with the value stored in the index header
  Version Compatibility:
  - Handles legacy byte order differences from original C# implementation
  - Supports IndexBlock format (version 1 only)
  - Does not support IndexBlock2 or IndexBlock3 formats
  Memory Management:
  - Allocates temporary memory for index entries during verification
  - Automatically frees allocated memory on both success and error conditions
  - Memory usage is proportional to the number of index entries
  Verification Scope:
  - Validates index header structure and identifier
  - Verifies data integrity through CRC64 calculation
  - Does not validate individual index entry contents or block references
  - Does not check for logical consistency of referenced blocks

Warning
  This function reads the entire index into memory for CRC calculation. Large indexes may require significant memory allocation.
  The function assumes ctx->header.indexOffset points to a valid index location. Invalid offsets will cause file access errors or incorrect validation.
  CRC validation failure indicates potential data corruption and may suggest the image file is damaged or has been modified outside of library control.

Definition at line 225 of file index_v1.c.

References aaruf_crc64_data(), AARUF_ERROR_CANNOT_READ_HEADER, AARUF_ERROR_CANNOT_READ_INDEX, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, AARUF_ERROR_NOT_ENOUGH_MEMORY, AARUF_STATUS_OK, AARUF_VERSION_V1, bswap_64, IndexHeader::crc64, IndexHeader::entries, FATAL, aaruformat_context::header, IndexHeader::identifier, AaruHeaderV2::imageMajorVersion, aaruformat_context::imageStream, IndexBlock, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_verify_image().

## verify_index_v2()

int32_t verify_index_v2 | ( | aaruformat_context * | ctx | ) |

Verifies the integrity of an index block (version 2) in the image stream.

Checks the CRC64 of the index block without decompressing it. This function performs comprehensive validation of the version 2 index structure including header validation, data integrity verification, and version-specific CRC calculation. It ensures the index block is valid and uncorrupted before the image can be safely used for data access.

Parameters

  ctx | Pointer to the aaruformat context containing image stream and header information.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully verified index integrity. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from ctx->header.indexOffset
  - The index identifier matches IndexBlock2 (version 2 format)
  - Memory allocation for index entries succeeds
  - All index entries are successfully read from the image stream
  - CRC64 calculation completes successfully with version-specific endianness handling
  - The calculated CRC64 matches the expected CRC64 in the index header

  AARUF_ERROR_NOT_AARUFORMAT

  (-1) Invalid context or stream. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid

  AARUF_ERROR_CANNOT_READ_HEADER

  (-6) Index header reading failed. This occurs when:
  - Cannot read the complete IndexHeader2 structure from the image stream
  - File I/O errors prevent accessing the header at ctx->header.indexOffset
  - Insufficient data available at the specified index offset

  AARUF_ERROR_CANNOT_READ_INDEX

  (-19) Index format or data access errors. This occurs when:
  - The index identifier doesn't match IndexBlock2 (wrong format or corruption)
  - Cannot read all index entries from the image stream
  - File I/O errors during index entry reading
  - Index structure is corrupted or truncated

  AARUF_ERROR_NOT_ENOUGH_MEMORY

  (-9) Memory allocation failed. This occurs when:
  - Cannot allocate memory for the index entries array
  - System memory exhaustion prevents loading index data for verification

  AARUF_ERROR_INVALID_BLOCK_CRC

  (-18) CRC64 validation failed. This occurs when:
  - The calculated CRC64 doesn't match the expected CRC64 in the index header
  - Index data corruption is detected
  - Data integrity verification fails indicating potential file damage

Note
  CRC64 Validation Process:
  - Reads all index entries into memory for CRC calculation
  - Calculates CRC64 over the complete index entries array
  - Applies version-specific endianness conversion for compatibility
  - For imageMajorVersion <= AARUF_VERSION_V1: Uses bswap_64() for byte order correction
  - Compares calculated CRC64 with the value stored in the IndexHeader2
  Version 2 Enhancements:
  - Uses IndexHeader2 structure with enhanced metadata support
  - Maintains compatibility with legacy endianness handling
  - Supports improved index entry organization compared to version 1
  Memory Management:
  - Allocates temporary memory for index entries during verification
  - Automatically frees allocated memory on both success and error conditions
  - Memory usage is proportional to the number of index entries
  Verification Scope:
  - Validates index header structure and identifier
  - Verifies data integrity through CRC64 calculation
  - Does not validate individual index entry contents or block references
  - Does not check for logical consistency of referenced blocks

Warning
  This function reads the entire index into memory for CRC calculation. Large indexes may require significant memory allocation.
  The function assumes ctx->header.indexOffset points to a valid index location. Invalid offsets will cause file access errors or incorrect validation.
  CRC validation failure indicates potential data corruption and may suggest the image file is damaged or has been modified outside of library control.

Definition at line 227 of file index_v2.c.

References aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_READ_HEADER, AARUF_ERROR_CANNOT_READ_INDEX, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, AARUF_ERROR_NOT_ENOUGH_MEMORY, AARUF_STATUS_OK, AARUF_VERSION_V1, bswap_64, IndexHeader2::crc64, IndexHeader2::entries, FATAL, aaruformat_context::header, IndexHeader2::identifier, AaruHeaderV2::imageMajorVersion, aaruformat_context::imageStream, IndexBlock2, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_verify_image().

## verify_index_v3()

int32_t verify_index_v3 | ( | aaruformat_context * | ctx | ) |

Verifies the integrity of an index block (version 3) in the image stream.

Checks the CRC64 of the index block and all subindexes without decompressing them. This function performs comprehensive validation of the advanced version 3 index structure including header validation, data integrity verification, and version-specific CRC calculation. Note that this function validates only the main index block's CRC and does not recursively validate subindex CRCs, focusing on the primary index structure integrity.

Parameters

  ctx | Pointer to the aaruformat context containing image stream and header information.

Returns
  Returns one of the following status codes:

Return values

  AARUF_STATUS_OK

  (0) Successfully verified index integrity. This is returned when:
  - The context and image stream are valid
  - The index header is successfully read from ctx->header.indexOffset
  - The index identifier matches IndexBlock3 (version 3 format)
  - Memory allocation for index entries succeeds
  - All index entries are successfully read from the image stream
  - CRC64 calculation completes successfully with version-specific endianness handling
  - The calculated CRC64 matches the expected CRC64 in the index header

  AARUF_ERROR_NOT_AARUFORMAT

  (-1) Invalid context or stream. This occurs when:
  - The context parameter is NULL
  - The image stream (ctx->imageStream) is NULL or invalid

  AARUF_ERROR_CANNOT_READ_HEADER

  (-6) Index header reading failed. This occurs when:
  - Cannot read the complete IndexHeader3 structure from the image stream
  - File I/O errors prevent accessing the header at ctx->header.indexOffset
  - Insufficient data available at the specified index offset

  AARUF_ERROR_CANNOT_READ_INDEX

  (-19) Index format or data access errors. This occurs when:
  - The index identifier doesn't match IndexBlock3 (wrong format or corruption)
  - Cannot read all index entries from the image stream
  - File I/O errors during index entry reading
  - Index structure is corrupted or truncated

  AARUF_ERROR_NOT_ENOUGH_MEMORY

  (-9) Memory allocation failed. This occurs when:
  - Cannot allocate memory for the index entries array
  - System memory exhaustion prevents loading index data for verification

  AARUF_ERROR_INVALID_BLOCK_CRC

  (-18) CRC64 validation failed. This occurs when:
  - The calculated CRC64 doesn't match the expected CRC64 in the index header
  - Index data corruption is detected
  - Data integrity verification fails indicating potential file damage

Note
  CRC64 Validation Process:
  - Reads all main index entries into memory for CRC calculation
  - Calculates CRC64 over the complete main index entries array
  - Applies version-specific endianness conversion for compatibility
  - For imageMajorVersion <= AARUF_VERSION_V1: Uses bswap_64() for byte order correction
  - Compares calculated CRC64 with the value stored in the IndexHeader3
  Version 3 Specific Features:
  - Uses IndexHeader3 structure with hierarchical subindex support
  - Maintains compatibility with legacy endianness handling
  - Supports advanced index entry organization with subindex references
  - Only validates the main index block CRC - subindex CRCs are not recursively checked
  Validation Scope:
  - Validates main index header structure and identifier
  - Verifies data integrity of the primary index entries through CRC64 calculation
  - Does not validate individual IndexEntry contents or block references
  - Does not check for logical consistency of referenced blocks
  - Does not recursively validate subindex blocks (unlike hierarchical processing)
  Memory Management:
  - Allocates temporary memory for index entries during verification
  - Automatically frees allocated memory on both success and error conditions
  - Memory usage is proportional to the number of entries in the main index only
  Subindex Validation Limitation:
  - This function does not recursively validate subindex CRCs
  - Each subindex block contains its own CRC that would need separate validation
  - For complete integrity verification, subindex blocks should be validated individually
  - The main index CRC only covers the primary index entries, not the subindex content

Warning
  This function reads only the main index into memory for CRC calculation. Subindex blocks are not loaded or validated by this function.
  The function assumes ctx->header.indexOffset points to a valid index location. Invalid offsets will cause file access errors or incorrect validation.
  CRC validation failure indicates potential data corruption in the main index and may suggest the image file is damaged or has been modified outside of library control.
  For complete integrity verification of hierarchical indexes, additional validation of subindex blocks may be required beyond this function's scope.

Definition at line 408 of file index_v3.c.

References aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_READ_HEADER, AARUF_ERROR_CANNOT_READ_INDEX, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, AARUF_ERROR_NOT_ENOUGH_MEMORY, AARUF_STATUS_OK, AARUF_VERSION_V1, bswap_64, IndexHeader3::crc64, IndexHeader3::entries, FATAL, aaruformat_context::header, IndexHeader3::identifier, AaruHeaderV2::imageMajorVersion, aaruformat_context::imageStream, IndexBlock3, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_verify_image().