From c488a27497199aa520df540ffecf1e159fbc06a1 Mon Sep 17 00:00:00 2001 From: Alexandre Mutel Date: Sat, 21 Feb 2026 14:30:37 +0100 Subject: [PATCH] Document parser authoring contracts and migration risks --- doc/parser-authoring-api-breaking-changes.md | 77 ++++++++++ doc/parser-authoring-api.md | 139 +++++++++++++++++++ src/Markdig/Parsers/BlockState.cs | 16 ++- 3 files changed, 225 insertions(+), 7 deletions(-) create mode 100644 doc/parser-authoring-api-breaking-changes.md create mode 100644 doc/parser-authoring-api.md diff --git a/doc/parser-authoring-api-breaking-changes.md b/doc/parser-authoring-api-breaking-changes.md new file mode 100644 index 00000000..0b811ff4 --- /dev/null +++ b/doc/parser-authoring-api-breaking-changes.md @@ -0,0 +1,77 @@ +# Parser + AST API Migration Notes (Potential Breaking Changes) + +This document tracks compatibility risks for consumers adopting the new parser/AST authoring APIs and for future releases that may tighten contracts. + +## Current Release Impact + +The implemented changes are additive/public-surface expansions: + +- `InlineProcessor.ReplaceParentContainer(...)` is now public. +- `BlockProcessor.TakeLinesBefore()` is public. +- `BlockProcessor.IsOpen(Block)` is public. +- `BlockProcessor.TryDiscard(Block)` is new. +- `InlineProcessor.GetParserState(...)` and `InlineProcessor.Emit(...)` are new. +- `Block.Remove()`, `Block.ReplaceBy(...)`, `ContainerBlock.TransferChildrenTo(...)`, and `ContainerInline.TransferChildrenTo(...)` are new. +- `MarkdownPipelineBuilder.TrackTrivia` setter is public. +- Typed metadata helpers (`DataKey`, `MarkdownObjectDataExtensions`) are new. + +No runtime behavior change is required for existing consumers unless they opt into these APIs. + +## Potential Future Breaking Changes + +These are not implemented in this release, but consumers should avoid relying on ambiguous behavior. + +### 1. `ReplaceParentContainer` replacement limits + +Current contract allows a single replacement request per leaf pass. If this evolves to support multiple requests, error behavior and ordering guarantees may change. + +Guidance: + +- Keep replacements localized and deterministic. +- Avoid depending on repeated replacement attempts in one pass. + +### 2. Mutation-helper metadata behavior + +Mutation helpers currently do not recalculate spans/trivia automatically. A future strict mode might validate or enforce metadata consistency. + +Guidance: + +- Explicitly set/update `Span`, `Line`, `Column`, and trivia fields in transforms where source fidelity matters. + +### 3. Internal helper cleanup + +Internal compatibility shims may be removed in a later major release after migration (for example, internal aliases kept during transition). + +Guidance: + +- Use public methods (`TakeLinesBefore`, `TryDiscard`, etc.) directly. + +### 4. Typed metadata API naming and overload resolution + +Some helper overloads share names with existing instance methods. Advanced consumers should prefer explicit generic invocations to avoid ambiguity. + +Guidance: + +- Prefer explicit generic calls: + - `node.SetData(value)` + - `node.SetData(key, value)` + - `node.GetData(key)` + - `node.TryGetData(key, out var value)` + +## Migration Checklist for Extension Authors + +1. Replace custom open-stack discard patterns with `TryDiscard` where appropriate. +2. Use `TakeLinesBefore()` instead of internal trivia helpers. +3. For inline-driven parent replacement: + - replace the parent container in the AST first, + - then call `ReplaceParentContainer(old, @new)`. +4. Replace manual child-move loops (`RemoveAt(0)` patterns) with transfer helpers. +5. Add regression tests for transformed AST shape (parent/child ownership and ordering). + +## Versioning Guidance + +For a future major version, consider: + +- deprecating internal transition shims, +- optionally adding strict validation mode for parser invariants and mutation metadata, +- documenting any tightened contracts as migration steps in this file. diff --git a/doc/parser-authoring-api.md b/doc/parser-authoring-api.md new file mode 100644 index 00000000..51bcf12a --- /dev/null +++ b/doc/parser-authoring-api.md @@ -0,0 +1,139 @@ +# Parser + AST Authoring API Guide + +This guide documents the parser authoring contracts and the new public APIs added for extension parity and safe AST manipulation. + +## Scope + +The authoring model remains two-phase: + +1. Block parsing (`BlockProcessor` + `BlockParser`). +2. Inline parsing (`InlineProcessor` + `InlineParser` + `IPostInlineProcessor`). + +Performance characteristics are unchanged: parser dispatch still relies on opening-character maps and pooled processors. + +## Public API Additions + +### `InlineProcessor.ReplaceParentContainer(...)` + +Use this when an inline parser replaces a parent container block during inline processing. + +Contract: + +- Call only while processing the current leaf (`ProcessInlineLeaf` pass). +- Replace the container in the AST first, then call `ReplaceParentContainer(old, @new)` to synchronize traversal state. +- Only one replacement request is allowed per leaf processing pass. + +### `BlockProcessor.TakeLinesBefore()` + +Use this in `TryOpen` when `TrackTrivia` is enabled and the new block should own pending leading blank/trivia lines. + +Contract: + +- Returns current `LinesBefore` and clears it. +- Returns `null` when there is no pending list. + +### `BlockProcessor.IsOpen(Block)` and `BlockProcessor.TryDiscard(Block)` + +Use these to safely reason about/modify the open block stack. + +Contract: + +- `IsOpen` is a read-only stack membership check. +- `TryDiscard` removes an open non-root block from both parent container and open stack, returning `true` only when a discard happened. + +### `MarkdownPipelineBuilder.TrackTrivia` + +`TrackTrivia` is now publicly settable and flows into the built pipeline/processors. + +## Block Parser Contract + +### `BlockState` semantics + +- `None`: no match for this parser/block. +- `Skip`: skip parser for this line and continue with others. +- `Continue`: block stays open; leaf blocks append line content. +- `ContinueDiscard`: block stays open; line consumed but not appended. +- `Break`: block closes; current line remains available for further parsing. +- `BreakDiscard`: block closes; current line is consumed. + +### `NewBlocks` invariants + +- Every pushed block must have `Parser` set to the creating parser. +- Leaf blocks must be pushed last. +- Push order is outer container to inner container to leaf (LIFO pop in processor). + +### Trivia rules + +When `TrackTrivia` is enabled: + +- Use `TakeLinesBefore()` in `TryOpen` to assign pending leading blank/trivia lines. +- Use `UseTrivia(end)` when a parser needs exact trivia slices around syntax markers. + +## Inline Parser Contract + +### `Match` behavior + +- On success: advance `slice` and set `processor.Inline` when emitting a node. +- On failure: return `false` without mutating parser output state. + +### Emission behavior + +- If a matched parser sets a parentless `processor.Inline`, the processor appends it to the deepest open inline container. +- For explicit emission, use `processor.Emit(inline)`. + +### Parser state + +Use `processor.GetParserState(this)` or `processor.GetParserState(this, factory)` for per-leaf parser state. +Parser states are cleared at the beginning of each `ProcessInlineLeaf`. + +### Block transforms from inline parsing + +- `processor.BlockNew` replaces the current leaf block after the current leaf pass returns. +- `processor.ReplaceParentContainer(old, @new)` keeps traversal coherent when a parent container has already been replaced in the AST. + +## Hook Selection + +| Goal | Preferred hook | +|---|---| +| Add block syntax | `BlockParser` in `MarkdownPipelineBuilder.BlockParsers` | +| Add inline syntax | `InlineParser` in `MarkdownPipelineBuilder.InlineParsers` | +| Deferred inline resolution | `IPostInlineProcessor` | +| Replace current leaf block | `InlineProcessor.BlockNew` | +| Replace parent container from inline | `InlineProcessor.ReplaceParentContainer` | +| Literal post-processing | `LiteralInlineParser.PostMatch` | +| Post-close block transform | `BlockParser.Closed` | +| Per-block inline begin/end behavior | `Block.ProcessInlinesBegin` / `Block.ProcessInlinesEnd` | +| Post-document transform | `MarkdownPipelineBuilder.DocumentProcessed` | + +## AST Mutation Helpers + +### Block-level + +- `Block.Remove()`: remove from parent container. +- `Block.ReplaceBy(replacement, moveChildren: true)`: replace in parent container and optionally move children. +- `ContainerBlock.TransferChildrenTo(destination)`: move child blocks in order. + +### Inline-level + +- `ContainerInline.TransferChildrenTo(destination)`: move child inlines in order. + +Mutation helpers do not auto-recompute spans/trivia; callsites remain responsible for source metadata correctness when required. + +## Typed Metadata Helpers + +For extension state attached to AST nodes: + +- `SetData(value)` / `GetData()`. +- `TryGetData(key, out value)` / `GetData(key)` for explicit object keys. +- `DataKey` for collision-resistant typed keys. + +Example: + +```csharp +var key = new DataKey(); +block.SetData(key, state); +if (block.TryGetData(key, out var existing)) +{ + // use existing +} +``` diff --git a/src/Markdig/Parsers/BlockState.cs b/src/Markdig/Parsers/BlockState.cs index bd1fbc7f..1172f629 100644 --- a/src/Markdig/Parsers/BlockState.cs +++ b/src/Markdig/Parsers/BlockState.cs @@ -10,32 +10,34 @@ namespace Markdig.Parsers; public enum BlockState { /// - /// A line is not accepted by this parser. + /// The parser does not accept the line for this block. + /// No line content is consumed by this result. /// None, /// - /// The parser is skipped. + /// Skips this parser for the current line and continues with the next parser/candidate block. /// Skip, /// - /// The parser accepts a line and instruct to continue. + /// The parser accepts the line and keeps the block open. + /// For leaf blocks, the current line is appended to the block. /// Continue, /// - /// The parser accepts a line, instruct to continue but discard the line (not stored on the block) + /// The parser accepts the line and keeps the block open, but the line is consumed and not appended. /// ContinueDiscard, /// - /// The parser is ending a block, instruct to stop and keep the line being processed. + /// The parser ends the block and keeps the current line available for further parsing. /// Break, /// - /// The parser is ending a block, instruct to stop and discard the line being processed. + /// The parser ends the block and consumes the current line. /// BreakDiscard -} \ No newline at end of file +}