Introduce index block version 3 with enhanced fault tolerance and previous block referencing

This commit is contained in:
2025-08-02 01:45:27 +01:00
parent d518458a63
commit 5de2fc8f84
6 changed files with 79 additions and 46 deletions

View File

@@ -0,0 +1,59 @@
=== Index Block Version 3 (`IDX3`)
Version 3 of the index structure introduces a new field: `previous`. This field enables any index block to reference another, allowing for non-linear and flexible index placement within the file.
This enhancement permits index blocks to be written at arbitrary offsets. If the current index is not the initial one, the `previous` field stores a pointer to the preceding index block, forming a linked chain of indexes.
In scenarios where the file cannot be finalized correctly—due to crashes, write interruptions, or failure to update the master header—the index chain allows reconstruction of the indexing structure. By scanning backward from the end of the file to locate the last written index and following each `previous` pointer, earlier indexes can be discovered and parsed.
This mechanism significantly improves fault tolerance. Unlike earlier versions where a partially written file could render indexing unusable, version 3 allows for resilient recovery and continued access to indexed data.
The index block in version 3 is immediately followed by index entries that maintain the same format as those defined in version 2, ensuring compatibility and easing transition between versions.
Index entries can contain *at most* a single child index pointer.
==== Structure Definition
[source,c]
/**Header for the index, followed by entries */
typedef struct IndexHeader3
{
/**Identifier, <see cref="BlockType.Index" /> */
uint32_t identifier;
/**How many entries follow this header */
uint64_t entries;
/**CRC64-ECMA of the index */
uint64_t crc64;
/**Pointer to the previous index header */
uint64_t previous;
} IndexHeader3;
==== Field Descriptions
[cols="2,2,2,6",options="header"]
|===
|Type
|Size
|Name
|Description
|uint32
|4 bytes
|identifier
|The index block identifier, always `IDX3`
|uint64
|8 bytes
|entries
|The number of entries following this header
|uint64
|8 bytes
|crc64
|CRC64-ECMA checksum of the entries following this header
|uint64
|8 bytes
|previous
|Pointer in image file to previous index block. `0` if this is the first index block.
|===

View File

@@ -1,45 +0,0 @@
=== Index Block Continuation (`IDXC`)
The index block continuation follows the same structure and semantics as the index block version 2, with the exception that it includes a pointer to the preceding index block or index block continuation.
At most, a single index block continuation may appear within any index block structure.
The purpose of this block is to enable incremental indexing prior to finalizing the image file.
This allows new blocks to be indexed as they are written, facilitating partial recovery in the event of application failure.
The block is immediately followed by index entries formatted identically to those defined in index block version 2.
==== Structure Definition
[source,c]
/* Undefined */
==== Field Descriptions
[cols="2,2,2,6",options="header"]
|===
|Type
|Size
|Name
|Description
|uint32
|4 bytes
|identifier
|The index block identifier, always `IDXC`
|uint64
|8 bytes
|entries
|The number of entries following this header
|uint64
|8 bytes
|crc64
|CRC64-ECMA checksum of the entries following this header
|uint64
|8 bytes
|previous
|Pointer in image file to previous index block
|===

View File

@@ -42,7 +42,7 @@ include::blocks/index2.adoc[]
<<<
include::blocks/index_continuation.adoc[]
include::blocks/index3.adoc[]
<<<

View File

@@ -84,4 +84,8 @@ Add annex explaining the meaning and relationship between user, bitstream and fl
Move specification to Asciidoc.
Deprecate Compact Disc lead-in, first track pregap, lead-out, and floppy disk lead-out data types. They are stored as user data now (with negative and overflow sectors as appropiate).
Clarify and correct deduplication table specification.
Rework index version 3 specification.
|===

View File

@@ -210,6 +210,8 @@ typedef enum
IndexBlock = 0x58444E49,
/** Block containing the index v2 */
IndexBlock2 = 0x32584449,
/** Block containing the index v3 */
IndexBlock3 = 0x33584449,
/** Block containing logical geometry */
GeometryBlock = 0x4D4F4547,
/** Block containing metadata */

View File

@@ -43,6 +43,19 @@ typedef struct IndexHeader2
uint64_t crc64;
} IndexHeader2;
/**Header for the index, followed by entries */
typedef struct IndexHeader3
{
/**Identifier, <see cref="BlockType.Index" /> */
uint32_t identifier;
/**How many entries follow this header */
uint64_t entries;
/**CRC64-ECMA of the index */
uint64_t crc64;
/**Pointer to the previous index header */
uint64_t previous;
} IndexHeader3;
/**Index entry */
typedef struct IndexEntry
{