Any way of accessing (external) contextual data in a parser? #262

New Issue

claunia · 2026-01-29T14:31:57Z

claunia commented

2026-01-29 14:31:57 +00:00

Originally created by @patriksvensson on GitHub (Jan 10, 2019).

I've written a MarkDig parser and in the parser I would want to get access to some contextual information about the document being parsed (that's not available in the document itself).

Is there any way of passing contextual data to a parser on a parse basis (just like I can in Scriban when rendering)?

// Something like this?
var html = Markdown.ToHtml(markdown, _pipeline, context);

If there is no work around for my problem, I would be happy to provide a pull request for this functionality if you think it would be a good fit for MarkDig.

Originally created by @patriksvensson on GitHub (Jan 10, 2019). I've written a MarkDig parser and in the parser I would want to get access to some contextual information about the document being parsed (that's not available in the document itself). Is there any way of passing contextual data to a parser on a parse basis (just like I can in Scriban when rendering)? ```csharp // Something like this? var html = Markdown.ToHtml(markdown, _pipeline, context); ``` If there is no work around for my problem, I would be happy to provide a pull request for this functionality if you think it would be a good fit for MarkDig.

claunia added the question label 2026-01-29 14:31:57 +00:00

claunia closed this issue

2026-01-29 14:31:58 +00:00

claunia commented

2026-01-29 14:31:59 +00:00

@xoofx commented on GitHub (Jan 10, 2019):

I've written a MarkDig parser

You mean a Block processor or inline processor?

@xoofx commented on GitHub (Jan 10, 2019): > I've written a MarkDig parser You mean a Block processor or inline processor?

claunia commented

2026-01-29 14:32:00 +00:00

@MihaZupan commented on GitHub (Jan 10, 2019):

You can add context by including it when you create the pipeline - for every parsing, pass the proper context to the parser instance you include in the pipeline.

@MihaZupan commented on GitHub (Jan 10, 2019): You can add context by including it when you create the pipeline - for every parsing, pass the proper context to the parser instance you include in the pipeline.

claunia commented

2026-01-29 14:32:00 +00:00

@patriksvensson commented on GitHub (Jan 10, 2019):

@xoofx An inline processor. Would prefer not to recreate the pipeline on each render, that's why I ask.

@patriksvensson commented on GitHub (Jan 10, 2019): @xoofx An inline processor. Would prefer not to recreate the pipeline on each render, that's why I ask.

claunia commented

2026-01-29 14:32:02 +00:00

@MihaZupan commented on GitHub (Jan 11, 2019):

Essentially something like this then c1ad27670e?

Accessible from parsers via processor.Document.CustomState

@MihaZupan commented on GitHub (Jan 11, 2019): Essentially something like this then https://github.com/MihaZupan/markdig/commit/c1ad27670e2485a5c7de53c25c1a79b3039213ef? Accessible from parsers via `processor.Document.CustomState`

claunia commented

2026-01-29 14:32:02 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

Hm... if it is only a parser information, this could be made available via a MarkdownParserContext, accessible via a MarkdownParser.Context property. By default, this property would be null (and readonly, only passed at construction time)

A MarkdownParserContext would be a class with only a dictionary<object, object> Properties in it... passed down from Markdown.Parse/ToHtml methods as a last argument (null by default)

Not sure it would cover your needs?

@xoofx commented on GitHub (Jan 11, 2019): Hm... if it is only a parser information, this could be made available via a `MarkdownParserContext`, accessible via a `MarkdownParser.Context` property. By default, this property would be null (and readonly, only passed at construction time) A `MarkdownParserContext` would be a class with only a `dictionary<object, object> Properties` in it... passed down from Markdown.Parse/ToHtml methods as a last argument (null by default) Not sure it would cover your needs?

claunia commented

2026-01-29 14:32:03 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

@xoofx That would be perfect! Do you want me to send a PR for this?

@patriksvensson commented on GitHub (Jan 11, 2019): @xoofx That would be perfect! Do you want me to send a PR for this?

claunia commented

2026-01-29 14:32:03 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

@xoofx That would be perfect! Do you want me to send a PR for this?

Yes, please

@xoofx commented on GitHub (Jan 11, 2019): > @xoofx That would be perfect! Do you want me to send a PR for this? Yes, please

claunia commented

2026-01-29 14:32:04 +00:00

@MihaZupan commented on GitHub (Jan 11, 2019):

Is a dictionary necessary if it's essentially only one object?

@MihaZupan commented on GitHub (Jan 11, 2019): Is a dictionary necessary if it's essentially only one object?

claunia commented

2026-01-29 14:32:04 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

Is a dictionary necessary if it's essentially only one object?

The problem is that if all extensions are coming with all their own object, it will not be possible to push for a common object, so it is easier to say the context is dynamic with a single dictionary (and the class is sealed)

@xoofx commented on GitHub (Jan 11, 2019): > Is a dictionary necessary if it's essentially only one object? The problem is that if all extensions are coming with all their own object, it will not be possible to push for a common object, so it is easier to say the context is dynamic with a single dictionary (and the class is sealed)

claunia commented

2026-01-29 14:32:05 +00:00

@MihaZupan commented on GitHub (Jan 11, 2019):

How about adding the passed objects into the MarkdownObject, since a key/value collection for different parsers is already implemented there

@MihaZupan commented on GitHub (Jan 11, 2019): How about adding the passed objects into the MarkdownObject, since a key/value collection for different parsers is already implemented there

claunia commented

2026-01-29 14:32:05 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

How about adding the passed objects into the MarkdownObject, since a key/value collection for different parsers is already implemented there

Indeed, forgot about the codebase actually... 😅

@patriksvensson could you use directly the MarkdownDocument (accessible from both BlockProcessor and InlineProcessor) as an indirect context (as you can add your own SetData/GetData on it directly)?

@xoofx commented on GitHub (Jan 11, 2019): > How about adding the passed objects into the MarkdownObject, since a key/value collection for different parsers is already implemented there Indeed, forgot about the codebase actually... 😅 @patriksvensson could you use directly the `MarkdownDocument` (accessible from both BlockProcessor and InlineProcessor) as an indirect context (as you can add your own SetData/GetData on it directly)?

claunia commented

2026-01-29 14:32:05 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

@xoofx I can try and do that.

From an API perspective I think it would be more discoverable though by explicitly having a context (just my two cents).

@patriksvensson commented on GitHub (Jan 11, 2019): @xoofx I can try and do that. From an API perspective I think it would be more discoverable though by explicitly having a context (just my two cents).

claunia commented

2026-01-29 14:32:06 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

From an API perspective I think it would be more discoverable though by explicitly having a context (just my two cents).

Yeah, but the document object is already used by parsers as a context, so it is unnecessary to introduce something new here. The more I wait to write a documentation + the fact that I don't use the project, the more I forget about all these details... at least the GitHub issues can help folks out there... 😅

@xoofx commented on GitHub (Jan 11, 2019): > From an API perspective I think it would be more discoverable though by explicitly having a context (just my two cents). Yeah, but the document object is already used by parsers as a context, so it is unnecessary to introduce something new here. The more I wait to write a documentation + the fact that I don't use the project, the more I forget about all these details... at least the GitHub issues can help folks out there... 😅

claunia commented

2026-01-29 14:32:06 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

@xoofx Since I'm setting the data directly on the MarkdownDocument I assume I need to implement locking around the parsing (I can't process more than one document at a time since this the document is tied to the pipeline)? Or is the data stored in thread local storage or similar?

@patriksvensson commented on GitHub (Jan 11, 2019): @xoofx Since I'm setting the data directly on the `MarkdownDocument` I assume I need to implement locking around the parsing (I can't process more than one document at a time since this the document is tied to the pipeline)? Or is the data stored in thread local storage or similar?

claunia commented

2026-01-29 14:32:06 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

No, the parser assume that the document is not being modified by another thread while parsing, so you don't need any locking. The processors on the other hand should not store data because they are shared across a pipeline, which is supposed to be thread safe (there are cases where the processors could be modified externally but it is not recommended to do that)

@xoofx commented on GitHub (Jan 11, 2019): No, the parser assume that the document is not being modified by another thread while parsing, so you don't need any locking. The processors on the other hand should not store data because they are shared across a pipeline, which is supposed to be thread safe (there are cases where the processors could be modified externally but it is not recommended to do that)

claunia commented

2026-01-29 14:32:07 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

@xoofx I'm sorry and excuse me for taking up so much of your time with this.
I'm not sure how to do this to be frankly. Is there any source code I can look at to understand what I'm supposed to do? I'm super confused.

@patriksvensson commented on GitHub (Jan 11, 2019): @xoofx I'm sorry and excuse me for taking up so much of your time with this. I'm not sure how to do this to be frankly. Is there any source code I can look at to understand what I'm supposed to do? I'm super confused.

claunia commented

2026-01-29 14:32:07 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

I'm not sure how to do this to be frankly

Precisely, you have nothing to do ☺️

Just use MarkdownDocument that you can access from the inline and block processor the Document instance (check for example LinkInlineParser)

@xoofx commented on GitHub (Jan 11, 2019): > I'm not sure how to do this to be frankly Precisely, you have nothing to do ☺️ Just use MarkdownDocument that you can access from the inline and block processor the Document instance (check for example [LinkInlineParser](https://github.com/lunet-io/markdig/blob/832f86cf5f217db502b00b0b20dca04b7800313c/src/Markdig/Parsers/Inlines/LinkInlineParser.cs#L58))

claunia commented

2026-01-29 14:32:08 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

Yes, but how do I set that from outside the parser in my code?

@patriksvensson commented on GitHub (Jan 11, 2019): Yes, but how do I set that from outside the parser in my code?

claunia commented

2026-01-29 14:32:08 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

Yes, but how do I set that from outside the parser in my code?

You should give an example. I'm now confused what you want to do exactly. 😉

@xoofx commented on GitHub (Jan 11, 2019): > Yes, but how do I set that from outside the parser in my code? You should give an example. I'm now confused what you want to do exactly. 😉

claunia commented

2026-01-29 14:32:09 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

Ah, sorry.

In my case I'm renderering markdown for a custom format.
Currently I'm writing a parser that transforms identifiers like @1234 to a link. So before rendering the template, I want to make sure that my InlineParser have access to contextual data for the markdown data being rendered.

// I would want to do something like this
var rendered1 = MarkDown.Render(first, _pipeline, new Dictionary {
    { "LinkBaseForDocument", "https://foo.com"  
});

var rendered2 = MarkDown.Render(second, _pipeline, new Dictionary {
    { "LinkBaseForDocument", "https://bar.com"  
});

Which given something like:

Hello World @284xyz

would render to:

Hello World [@284xyz](https://foo.com/file/284xyz)

Given that the document was rendered with LinkBaseForDocument set to https://foo.com

@patriksvensson commented on GitHub (Jan 11, 2019): Ah, sorry. In my case I'm renderering markdown for a custom format. Currently I'm writing a parser that transforms identifiers like `@1234` to a link. So before rendering the template, I want to make sure that my InlineParser have access to contextual data for the markdown data being rendered. ``` // I would want to do something like this var rendered1 = MarkDown.Render(first, _pipeline, new Dictionary { { "LinkBaseForDocument", "https://foo.com" }); var rendered2 = MarkDown.Render(second, _pipeline, new Dictionary { { "LinkBaseForDocument", "https://bar.com" }); ``` Which given something like: ``` Hello World @284xyz ``` would render to: ``` Hello World [@284xyz](https://foo.com/file/284xyz) ``` Given that the document was rendered with `LinkBaseForDocument` set to `https://foo.com`

claunia commented

2026-01-29 14:32:09 +00:00

@xoofx commented on GitHub (Jan 11, 2019):

I see, so let's go back to the MarkdownParserContext then, as a MarkdownDocument is more used for internal parser states, not external, so in that case a context is fine. PR welcome (sorry for the ping pong!)

@xoofx commented on GitHub (Jan 11, 2019): I see, so let's go back to the [MarkdownParserContext](https://github.com/lunet-io/markdig/issues/284#issuecomment-453480973) then, as a MarkdownDocument is more used for internal parser states, not external, so in that case a context is fine. PR welcome (sorry for the ping pong!)

claunia commented

2026-01-29 14:32:10 +00:00

@patriksvensson commented on GitHub (Jan 11, 2019):

@xoofx No problem at all! I know how important it is to discuss an issue properly with the maintainers to make sure that there is proper buy-in 😄

@patriksvensson commented on GitHub (Jan 11, 2019): @xoofx No problem at all! I know how important it is to discuss an issue properly with the maintainers to make sure that there is proper buy-in 😄

claunia commented

2026-01-29 14:32:10 +00:00

@leotsarev commented on GitHub (Jan 24, 2019):

Hi @xoofx !
I'm also do same in my code, but don't have any problems with context — I just rebuilt pipeline for every rendering attempt. I wonder if I'm doing that's wrong and I'm supposed to cache pipeline and use this approach with context...

@leotsarev commented on GitHub (Jan 24, 2019): Hi @xoofx ! I'm also do same in my code, but don't have any problems with context — I just rebuilt pipeline for every rendering attempt. I wonder if I'm doing that's wrong and I'm supposed to cache pipeline and use this approach with context...

claunia commented

2026-01-29 14:32:10 +00:00

@xoofx commented on GitHub (Jan 24, 2019):

Usually pipeline are supposed to be cachable

@xoofx commented on GitHub (Jan 24, 2019): Usually pipeline are supposed to be cachable

claunia commented

2026-01-29 14:32:11 +00:00

@leotsarev commented on GitHub (Jan 25, 2019):

Thanks for answer. Waiting for you to new release!

@leotsarev commented on GitHub (Jan 25, 2019): Thanks for answer. Waiting for you to new release!

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/markdig#262