index_v3.c File Reference

#include <limits.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include "aaruformat.h"
#include "internal.h"
#include "log.h"
#include "utarray.h"

Go to the source code of this file.

Functions
static bool	add_subindex_entries (aaruformat_context *ctx, UT_array *index_entries, const IndexEntry *subindex_entry)
	Adds entries from a subindex block (version 3) to the main index entries array.
UT_array *	process_index_v3 (aaruformat_context *ctx)
	Processes an index block (version 3) from the image stream.
int32_t	verify_index_v3 (aaruformat_context *ctx)
	Verifies the integrity of an index block (version 3) in the image stream.

Function Documentation

add_subindex_entries()

bool add_subindex_entries ( aaruformat_context * ctx, UT_array * index_entries, const IndexEntry * subindex_entry )

static

Adds entries from a subindex block (version 3) to the main index entries array.

Recursively reads subindex blocks and appends their entries to the main index entries array. This function is a critical component of the hierarchical index system in AaruFormat version 3, enabling scalable index organization for large image files. It performs recursive traversal of subindex structures, flattening the hierarchy into a single array while maintaining entry order and handling nested subindex references.

Parameters

ctx	Pointer to the aaruformat context containing the image stream.
index_entries	Pointer to the UT_array of main index entries to append to.
subindex_entry	Pointer to the IndexEntry describing the subindex location and metadata.

Returns

Returns true when entries are appended successfully, false when an error occurs.

Note

Function Behavior:

• Successfully appends subindex entries when all parameters are valid and file I/O succeeds
• Recursively processes nested subindexes when entries have blockType == IndexBlock3
• Returns false without modifying the array when validation fails or errors occur
• Does not perform error reporting beyond logging (errors are handled gracefully)

Success Conditions - Entries are added when:

• All input parameters (ctx, index_entries, subindex_entry) are non-NULL
• The image stream is valid and accessible
• File positioning succeeds to subindex_entry->offset
• The subindex header is successfully read from the image stream
• The subindex identifier matches IndexBlock3 (correct format)
• All entries in the subindex are successfully read and processed
• Recursive calls for nested subindexes complete successfully

Failure Conditions - Function returns false when:

• Any input parameter is NULL (ctx, index_entries, or subindex_entry)
• The image stream (ctx->imageStream) is NULL or invalid
• File I/O errors prevent reading the subindex header or entries
• The subindex identifier doesn't match IndexBlock3 (format mismatch or corruption)
• Memory operations fail during UT_array manipulation

Recursive Processing:

• When an entry has blockType == IndexBlock3, it indicates another subindex
• The function recursively calls itself to process nested subindexes
• Supports arbitrary nesting depth limited only by stack space and file structure
• All entries are flattened into the main index_entries array regardless of nesting level

Memory and Performance:

• Memory usage scales with the total number of entries across all processed subindexes
• File I/O is performed for each subindex block accessed
• Processing time increases with the depth and breadth of subindex hierarchies
• No internal memory allocation - uses existing UT_array structure

Error Handling:

• Designed for graceful degradation - errors don't propagate to callers
• Invalid subindexes are skipped without affecting other processing
• Partial success is possible when some subindexes fail but others succeed
• Logging provides visibility into processing failures for debugging

Warning

This function modifies the index_entries array by appending new entries. Ensure the array has sufficient capacity and is properly initialized.

Recursive processing can cause significant stack usage for deeply nested subindex structures. Very deep hierarchies may cause stack overflow.

The function assumes subindex_entry->offset points to a valid subindex block. Invalid offsets will cause file access errors but are handled gracefully.

No validation is performed on individual IndexEntry contents - only structural validation of subindex headers is done.

Definition at line 247 of file index_v3.c.

References add_subindex_entries(), IndexEntry::blockType, IndexHeader3::entries, FATAL, IndexHeader3::identifier, aaruformat_context::imageStream, IndexBlock3, IndexEntry::offset, and TRACE.

Referenced by add_subindex_entries(), and process_index_v3().

process_index_v3()

UT_array * process_index_v3

(

aaruformat_context *

ctx

)

Processes an index block (version 3) from the image stream.

Reads and parses an index block (version 3) from the image, returning an array of index entries. This function handles the advanced index format used in current AaruFormat versions, supporting hierarchical subindex structures for improved scalability. It reads the IndexHeader3 structure followed by index entries, recursively processing any subindex blocks encountered to create a flattened array of all index entries.

Parameters

ctx	Pointer to the aaruformat context containing the image stream and header information.

Returns

Returns one of the following values:

Return values

UT_array*

Successfully processed the index block and all subindexes. This is returned when: The context and image stream are valid The index header is successfully read from the position specified in ctx->header.indexOffset The index identifier matches IndexBlock3 (version 3 format identifier) All index entries are successfully read and stored in the UT_array Any subindex blocks (IndexBlock3 entries) are recursively processed via add_subindex_entries() Memory allocation for the index entries array succeeds The returned array contains all index entries from the main index and all subindexes

NULL

Index processing failed. This occurs when: The context parameter is NULL The image stream (ctx->imageStream) is NULL or invalid Cannot read the IndexHeader3 structure from the image stream The index identifier doesn't match IndexBlock3 (incorrect format or corruption) Memory allocation fails for the UT_array structure File I/O errors occur while reading index entries Recursive subindex processing fails (errors in add_subindex_entries())

Note

Index Structure (Version 3):

• IndexHeader3: Contains identifier (IndexBlock3), entry count, and advanced metadata
• IndexEntry array: May contain regular entries and subindex references (IndexBlock3 type)
• Hierarchical support: Subindex entries are recursively processed to flatten the structure
• No CRC validation is performed during processing (use verify_index_v3 for validation)
• Supports scalable index organization for large image files

Subindex Processing:

• When an IndexEntry has blockType == IndexBlock3, it references a subindex
• Subindexes are recursively processed using add_subindex_entries()
• All entries from subindexes are flattened into the main index entries array
• Supports arbitrary nesting depth of subindexes

Memory Management:

• Returns a newly allocated UT_array that must be freed by the caller using utarray_free()
• On error, any partially allocated memory is cleaned up before returning NULL
• Each IndexEntry is copied into the array (no reference to original stream data)
• Memory usage scales with total entries across all subindexes

Version Compatibility:

• Supports only IndexBlock3 format (not IndexBlock or IndexBlock2)
• Compatible with current generation AaruFormat image files
• Backward compatible with images that don't use subindexes

Warning

The caller is responsible for freeing the returned UT_array using utarray_free(). Failure to free the array will result in memory leaks.

This function does not validate the CRC integrity of the index data. Use verify_index_v3() to ensure index integrity before processing.

Recursive subindex processing may cause significant memory usage and processing time for images with deeply nested or numerous subindexes.

The function assumes ctx->header.indexOffset points to a valid index block. Invalid offsets may cause file access errors or reading incorrect data.

Definition at line 98 of file index_v3.c.

References add_subindex_entries(), IndexEntry::blockType, IndexHeader3::entries, FATAL, aaruformat_context::header, IndexHeader3::identifier, aaruformat_context::imageStream, IndexBlock3, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_open(), and aaruf_verify_image().

verify_index_v3()

int32_t verify_index_v3

(

aaruformat_context *

ctx

)

Verifies the integrity of an index block (version 3) in the image stream.

Checks the CRC64 of the index block and all subindexes without decompressing them. This function performs comprehensive validation of the advanced version 3 index structure including header validation, data integrity verification, and version-specific CRC calculation. Note that this function validates only the main index block's CRC and does not recursively validate subindex CRCs, focusing on the primary index structure integrity.

Parameters

ctx	Pointer to the aaruformat context containing image stream and header information.

Returns

Returns one of the following status codes:

Return values

AARUF_STATUS_OK	(0) Successfully verified index integrity. This is returned when: The context and image stream are valid The index header is successfully read from ctx->header.indexOffset The index identifier matches IndexBlock3 (version 3 format) Memory allocation for index entries succeeds All index entries are successfully read from the image stream CRC64 calculation completes successfully with version-specific endianness handling The calculated CRC64 matches the expected CRC64 in the index header
AARUF_ERROR_NOT_AARUFORMAT	(-1) Invalid context or stream. This occurs when: The context parameter is NULL The image stream (ctx->imageStream) is NULL or invalid
AARUF_ERROR_CANNOT_READ_HEADER	(-6) Index header reading failed. This occurs when: Cannot read the complete IndexHeader3 structure from the image stream File I/O errors prevent accessing the header at ctx->header.indexOffset Insufficient data available at the specified index offset
AARUF_ERROR_CANNOT_READ_INDEX	(-19) Index format or data access errors. This occurs when: The index identifier doesn't match IndexBlock3 (wrong format or corruption) Cannot read all index entries from the image stream File I/O errors during index entry reading Index structure is corrupted or truncated
AARUF_ERROR_NOT_ENOUGH_MEMORY	(-9) Memory allocation failed. This occurs when: Cannot allocate memory for the index entries array System memory exhaustion prevents loading index data for verification
AARUF_ERROR_INVALID_BLOCK_CRC	(-18) CRC64 validation failed. This occurs when: The calculated CRC64 doesn't match the expected CRC64 in the index header Index data corruption is detected Data integrity verification fails indicating potential file damage

Note

CRC64 Validation Process:

• Reads all main index entries into memory for CRC calculation
• Calculates CRC64 over the complete main index entries array
• Applies version-specific endianness conversion for compatibility
• For imageMajorVersion <= AARUF_VERSION_V1: Uses bswap_64() for byte order correction
• Compares calculated CRC64 with the value stored in the IndexHeader3

Version 3 Specific Features:

• Uses IndexHeader3 structure with hierarchical subindex support
• Maintains compatibility with legacy endianness handling
• Supports advanced index entry organization with subindex references
• Only validates the main index block CRC - subindex CRCs are not recursively checked

Validation Scope:

• Validates main index header structure and identifier
• Verifies data integrity of the primary index entries through CRC64 calculation
• Does not validate individual IndexEntry contents or block references
• Does not check for logical consistency of referenced blocks
• Does not recursively validate subindex blocks (unlike hierarchical processing)

Memory Management:

• Allocates temporary memory for index entries during verification
• Automatically frees allocated memory on both success and error conditions
• Memory usage is proportional to the number of entries in the main index only

Subindex Validation Limitation:

• This function does not recursively validate subindex CRCs
• Each subindex block contains its own CRC that would need separate validation
• For complete integrity verification, subindex blocks should be validated individually
• The main index CRC only covers the primary index entries, not the subindex content

Warning

This function reads only the main index into memory for CRC calculation. Subindex blocks are not loaded or validated by this function.

The function assumes ctx->header.indexOffset points to a valid index location. Invalid offsets will cause file access errors or incorrect validation.

CRC validation failure indicates potential data corruption in the main index and may suggest the image file is damaged or has been modified outside of library control.

For complete integrity verification of hierarchical indexes, additional validation of subindex blocks may be required beyond this function's scope.

Definition at line 408 of file index_v3.c.

References aaruf_crc64_final(), aaruf_crc64_free(), aaruf_crc64_init(), aaruf_crc64_update(), AARUF_ERROR_CANNOT_READ_HEADER, AARUF_ERROR_CANNOT_READ_INDEX, AARUF_ERROR_INVALID_BLOCK_CRC, AARUF_ERROR_NOT_AARUFORMAT, AARUF_ERROR_NOT_ENOUGH_MEMORY, AARUF_STATUS_OK, AARUF_VERSION_V1, bswap_64, IndexHeader3::crc64, IndexHeader3::entries, FATAL, aaruformat_context::header, IndexHeader3::identifier, AaruHeaderV2::imageMajorVersion, aaruformat_context::imageStream, IndexBlock3, AaruHeaderV2::indexOffset, and TRACE.

Referenced by aaruf_verify_image().