libaaruformat 1.0 Aaru Data Preservation Suite - Format Library

Loading...
Searching...
No Matches
DdtHeader2 Struct Reference

Header preceding a version 2 hierarchical deduplication table. More...

`#include <aaruformat/structs/ddt.h>`

## Data Fields
uint32_t | identifier
| Block identifier, must be BlockType::DeDuplicationTable2.
uint16_t | type
| Data classification (DataType) for sectors referenced by this table.
uint16_t | compression
| Compression algorithm for this table body (CompressionType).
uint8_t | levels
| Total number of hierarchy levels (root depth); > 0.
uint8_t | tableLevel
| Zero-based level index of this table (0 = root, increases downward).
uint64_t | previousLevelOffset
| Absolute byte offset of the parent (previous) level table; 0 if root.
uint16_t | negative
| Leading negative LBA count; added to external L to build internal index.
uint64_t | blocks
| Total internal span (negative + usable + overflow) in logical sectors.
uint16_t | overflow
| Trailing dumped sectors beyond user area (overflow range), still mapped with entries.
uint64_t | start
| Base internal index covered by this table (used for secondary tables; currently informational).
uint8_t | blockAlignmentShift
| 2^blockAlignmentShift = block alignment boundary in bytes.
uint8_t | dataShift
| 2^dataShift = sectors represented per increment in blockIndex field.
uint8_t | tableShift
| 2^tableShift = number of logical sectors per primary entry (multi-level only; 0 for single-level or secondary tables).
uint64_t | entries
| Number of entries contained in (uncompressed) table payload.
uint64_t | cmpLength
| Compressed payload size in bytes.
uint64_t | length
| Uncompressed payload size in bytes.
uint64_t | cmpCrc64
| CRC64-ECMA of compressed table payload.
uint64_t | crc64
| CRC64-ECMA of uncompressed table payload.

## Detailed Description

Header preceding a version 2 hierarchical deduplication table.

Version 2 introduces multi-level tables to efficiently address very large images by subdividing the logical address space. Tables at higher levels partition regions; leaves contain direct (block, sector) entry mappings. Navigation uses tableLevel (0 = root) and levels (total depth).

Logical sector (LBA) mapping (actual implementation in decode_ddt_{single,multi}_level_v2):

1. Let L be the requested logical sector (can be negative externally). Internal index I = L + negative. Valid range: 0 <= I < blocks. (Total user-data sectors often = blocks - negative - overflow.)
2. If tableShift == 0 (single-level): entryIndex = I. Else (multi-level): itemsPerPrimaryEntry = 1 << tableShift primaryIndex = I / itemsPerPrimaryEntry secondaryIndex = I % itemsPerPrimaryEntry The primary table entry at primaryIndex yields a secondary DDT file offset (scaled by 2^blockAlignmentShift), whose table entries are then indexed by secondaryIndex.
3. Read raw DDT entry value E (64-bit).
4. If E == 0: sector_status = SectorStatusNotDumped; offset=block_offset=0. Otherwise extract: statusBits = E >> 60 baseBits = E & 0xFFFFFFFFFFFFFFF sectorOffsetWithinBlock = baseBits & ((1 << dataShift) - 1) blockIndex = baseBits >> dataShift block_offset (bytes) = blockIndex << blockAlignmentShift offset (sector units inside block) = sectorOffsetWithinBlock
5. The consumer combines block_offset, offset, and the (external) logical sector size to locate data.

Field roles:

- negative: Count of leading negative LBAs supported; added to L to form internal index.
- overflow: Count of trailing LBAs beyond the user area upper bound that are still dumped and have normal DDT entries (e.g. optical disc lead-out). Symmetrical to 'negative' on the high end.
- start: For secondary tables, base internal index covered (written when creating new tables). Current decoding logic does not consult this field (future-proof placeholder).
- blockAlignmentShift: log2 alignment of stored data blocks (byte granularity of block_offset).
- dataShift: log2 of the number of addressable sectors per increment of blockIndex bitfield unit.
- tableShift: log2 of number of logical sectors covered by a single primary-table pointer (multi-level only).

Notes & current limitations:

- User area sector count = blocks - negative - overflow.
- Valid external LBA range exposed by the image = [-negative, (blocks - negative - 1)].
  - Negative range: [-negative, -1]
  - User area range: [0, (blocks - negative - overflow - 1)]
  - Overflow range: [(blocks - negative - overflow), (blocks - negative - 1)]
- Both negative and overflow ranges are stored with normal DDT entries (if present), enabling complete reproduction of lead-in / lead-out or similar padding regions.
- start is presently ignored during decoding; integrity checks against it may be added in future revisions.
- No masking is applied to I besides array bounds; callers must ensure L is within representable range.

Example (Compact Disc): Disc has 360000 user sectors. Lead-in captured as 15000 negative sectors and lead-out as 15000 overflow sectors. negative = 15000 overflow = 15000 user sectors = 360000 blocks (internal span) = negative + user + overflow = 390000 External LBA spans: -15000 .. 374999

- Negative: -15000 .. -1 (15000 sectors)
- User: 0 .. 359999 (360000 sectors)
- Overflow: 360000 .. 374999 (15000 sectors) Internal index I for any external L is I = L + negative. User area sector count reported to callers (ctx->imageInfo.Sectors) = blocks - negative - overflow = 360000.

Definition at line 141 of file ddt.h.

## Field Documentation

## ◆ blockAlignmentShift

uint8_t DdtHeader2::blockAlignmentShift

2^blockAlignmentShift = block alignment boundary in bytes.

Definition at line 154 of file ddt.h.

Referenced by aaruf_close_current_block(), aaruf_create(), aaruf_open(), decode_ddt_multi_level_v2(), decode_ddt_single_level_v2(), set_ddt_multi_level_v2(), set_ddt_single_level_v2(), set_ddt_tape(), write_aaru_json_block(), write_cached_secondary_ddt(), write_checksum_block(), write_cicm_block(), write_dumphw_block(), write_dvd_long_sector_blocks(), write_dvd_title_key_decrypted_block(), write_geometry_block(), write_index_block(), write_media_tags(), write_metadata_block(), write_mode2_subheaders_block(), write_sector_prefix(), write_sector_prefix_ddt(), write_sector_subchannel(), write_sector_suffix(), write_sector_suffix_ddt(), write_single_level_ddt(), write_tape_file_block(), write_tape_partition_block(), and write_tracks_block().

## ◆ blocks

uint64_t DdtHeader2::blocks

Total internal span (negative + usable + overflow) in logical sectors.

Definition at line 150 of file ddt.h.

Referenced by aaruf_create(), aaruf_get_user_sectors(), aaruf_open(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), and write_tape_ddt().

## ◆ cmpCrc64

uint64_t DdtHeader2::cmpCrc64

CRC64-ECMA of compressed table payload.

Definition at line 161 of file ddt.h.

Referenced by aaruf_verify_image(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), and write_single_level_ddt().

## ◆ cmpLength

uint64_t DdtHeader2::cmpLength

Compressed payload size in bytes.

Definition at line 159 of file ddt.h.

Referenced by aaruf_verify_image(), decode_ddt_multi_level_v2(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ compression

uint16_t DdtHeader2::compression

Compression algorithm for this table body (CompressionType).

Definition at line 145 of file ddt.h.

Referenced by aaruf_create(), decode_ddt_multi_level_v2(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ crc64

uint64_t DdtHeader2::crc64

CRC64-ECMA of uncompressed table payload.

Definition at line 162 of file ddt.h.

Referenced by decode_ddt_multi_level_v2(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), and write_single_level_ddt().

## ◆ dataShift

uint8_t DdtHeader2::dataShift

2^dataShift = sectors represented per increment in blockIndex field.

Definition at line 155 of file ddt.h.

Referenced by aaruf_create(), aaruf_write_sector(), decode_ddt_multi_level_v2(), decode_ddt_single_level_v2(), set_ddt_multi_level_v2(), set_ddt_single_level_v2(), set_ddt_tape(), write_cached_secondary_ddt(), write_sector_prefix_ddt(), and write_sector_suffix_ddt().

## ◆ entries

uint64_t DdtHeader2::entries

Number of entries contained in (uncompressed) table payload.

Definition at line 158 of file ddt.h.

Referenced by aaruf_create(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ identifier

uint32_t DdtHeader2::identifier

Block identifier, must be BlockType::DeDuplicationTable2.

Definition at line 143 of file ddt.h.

Referenced by aaruf_create(), decode_ddt_multi_level_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ length

uint64_t DdtHeader2::length

Uncompressed payload size in bytes.

Definition at line 160 of file ddt.h.

Referenced by decode_ddt_multi_level_v2(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ levels

uint8_t DdtHeader2::levels

Total number of hierarchy levels (root depth); > 0.

Definition at line 146 of file ddt.h.

Referenced by aaruf_create(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ negative

uint16_t DdtHeader2::negative

Leading negative LBA count; added to external L to build internal index.

Definition at line 149 of file ddt.h.

Referenced by aaruf_create(), aaruf_get_negative_sectors(), aaruf_read_sector(), aaruf_read_sector_long(), aaruf_read_sector_tag(), aaruf_write_sector(), aaruf_write_sector_long(), aaruf_write_sector_tag(), decode_ddt_multi_level_v2(), decode_ddt_single_level_v2(), process_ddt_v2(), set_ddt_multi_level_v2(), set_ddt_single_level_v2(), write_cached_secondary_ddt(), write_dvd_long_sector_blocks(), write_dvd_title_key_decrypted_block(), write_mode2_subheaders_block(), write_sector_prefix_ddt(), write_sector_subchannel(), write_sector_suffix_ddt(), and write_tape_ddt().

## ◆ overflow

uint16_t DdtHeader2::overflow

Trailing dumped sectors beyond user area (overflow range), still mapped with entries.

Definition at line 151 of file ddt.h.

Referenced by aaruf_create(), aaruf_get_overflow_sectors(), aaruf_read_sector(), aaruf_read_sector_long(), aaruf_read_sector_tag(), aaruf_write_sector(), aaruf_write_sector_long(), aaruf_write_sector_tag(), process_ddt_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_dvd_long_sector_blocks(), write_dvd_title_key_decrypted_block(), write_mode2_subheaders_block(), write_sector_prefix_ddt(), write_sector_subchannel(), write_sector_suffix_ddt(), and write_tape_ddt().

## ◆ previousLevelOffset

uint64_t DdtHeader2::previousLevelOffset

Absolute byte offset of the parent (previous) level table; 0 if root.

Definition at line 148 of file ddt.h.

Referenced by aaruf_create(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ start

uint64_t DdtHeader2::start

Base internal index covered by this table (used for secondary tables; currently informational).

Definition at line 153 of file ddt.h.

Referenced by aaruf_create(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), and write_tape_ddt().

## ◆ tableLevel

uint8_t DdtHeader2::tableLevel

Zero-based level index of this table (0 = root, increases downward).

Definition at line 147 of file ddt.h.

Referenced by aaruf_create(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ tableShift

uint8_t DdtHeader2::tableShift

2^tableShift = number of logical sectors per primary entry (multi-level only; 0 for single-level or secondary tables).

Definition at line 156 of file ddt.h.

Referenced by aaruf_create(), decode_ddt_entry_v2(), decode_ddt_multi_level_v2(), decode_ddt_single_level_v2(), set_ddt_entry_v2(), set_ddt_multi_level_v2(), set_ddt_single_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

## ◆ type

uint16_t DdtHeader2::type

Data classification (DataType) for sectors referenced by this table.

Definition at line 144 of file ddt.h.

Referenced by aaruf_create(), decode_ddt_multi_level_v2(), set_ddt_multi_level_v2(), write_cached_secondary_ddt(), write_primary_ddt(), write_sector_prefix_ddt(), write_sector_suffix_ddt(), write_single_level_ddt(), and write_tape_ddt().

The documentation for this struct was generated from the following file:

- include/aaruformat/structs/ddt.h