Support for extracting raw text, and replacing raw text. #205

New Issue

Closed

opened 2026-01-29 14:30:20 +00:00 by claunia · 5 comments

claunia commented 2026-01-29 14:30:20 +00:00

Owner

Copy Link

Originally created by @pauldotknopf on GitHub (May 3, 2018).

I have a requirement to support generating translated markdown documents from POT/PO files.

I need to parse a document for all raw string values. I will use this to generate a POT document that translators will use to create new languages.

I also need to support replacing raw text during rendering to enable replacing text with the translated value.

Any guidance would be greatly appreciated!

Originally created by @pauldotknopf on GitHub (May 3, 2018). I have a requirement to support generating translated markdown documents from POT/PO files. I need to parse a document for all raw string values. I will use this to generate a POT document that translators will use to create new languages. I also need to support replacing raw text during rendering to enable replacing text with the translated value. Any guidance would be greatly appreciated!

claunia closed this issue 2026-01-29 14:30:20 +00:00

claunia commented 2026-01-29 14:30:21 +00:00

Author

Owner

Copy Link

@pauldotknopf commented on GitHub (May 4, 2018):

Ok, so I figured it out. Let me know if I am heading down a wrong/stupid track?

Here are is my test case which illustrates my requirements.

[Fact]
public void Can_replace_text()
{
    var result = _markdownRenderer.TransformMarkdown("test", val => $"{val}-append");
    Assert.Equal("<p>test-append</p>", result);

    result = _markdownRenderer.TransformMarkdown("test*another*", val => $"{val}-append");
    Assert.Equal("<p>test-append<em>another-append</em></p>", result);

    var content = new StringBuilder();
    content.AppendLine("a | b");
    content.AppendLine("- | -");
    content.AppendLine("0 | 0");
    result = _markdownRenderer.TransformMarkdown(content.ToString(), val => $"{val}-append");
    content = new StringBuilder();
    content.AppendLine("<table>");
    content.AppendLine("<thead>");
    content.AppendLine("<tr>");
    content.AppendLine("<th>a-append</th>");
    content.AppendLine("<th>b-append</th>");
    content.AppendLine("</tr>");
    content.AppendLine("</thead>");
    content.AppendLine("<tbody>");
    content.AppendLine("<tr>");
    content.AppendLine("<td>0-append</td>");
    content.AppendLine("<td>0-append</td>");
    content.AppendLine("</tr>");
    content.AppendLine("</tbody>");
    content.Append("</table>");
    Assert.Equal(content.ToString(), result);
}

Here is what my internal implementation looks like.

public string TransformMarkdown(string markdown, Func<string, string> replacement)
{
    var pipeline = new MarkdownPipelineBuilder()
        .UseAdvancedExtensions()
        .Build();
    var document = Markdig.Markdown.Parse(markdown, pipeline);
    
    // Do a fake rendering that replaces content.
    var replacementRenderer = new TextReplacementRenderer(TextWriter.Null, replacement);
    pipeline.Setup(replacementRenderer);
    replacementRenderer.Render(document);
    
    using (var stringWriter = new StringWriter())
    {
        var htmlRenderer = new HtmlRenderer(stringWriter);
        pipeline.Setup(htmlRenderer);
        htmlRenderer.Render(document);
        stringWriter.Flush();
        return stringWriter.ToString().TrimEnd(Environment.NewLine.ToCharArray());
    }
}

The key part in the above code is my custom TextReplacementRenderer.

public class TextReplacementRenderer : TextRendererBase<TextReplacementRenderer>
{
    public TextReplacementRenderer(TextWriter writer, Func<string, string> replacement) : base(writer)
    {
        ObjectRenderers.Add(new CustomParagraphRenderer());
        ObjectRenderers.Add(new CustomLiteralInlineRenderer(replacement));
    }

    class CustomParagraphRenderer : MarkdownObjectRenderer<TextReplacementRenderer, ParagraphBlock>
    {
        protected override void Write(TextReplacementRenderer renderer, ParagraphBlock obj)
        {
            renderer.WriteLeafInline(obj);
        }
    }

    class CustomLiteralInlineRenderer : MarkdownObjectRenderer<TextReplacementRenderer, LiteralInline>
    {
        readonly Func<string, string> _replacement;

        public CustomLiteralInlineRenderer(Func<string, string> replacement)
        {
            _replacement = replacement;
        }
        
        protected override void Write(TextReplacementRenderer renderer, LiteralInline obj)
        {
            obj.Content = new StringSlice(_replacement(obj.Content.ToString()));
        }
    }
}

The main thing I'm worried about is that the start/end/position properties used all over the place would be invalid. Should I be? Is this approach ok?

@pauldotknopf commented on GitHub (May 4, 2018): Ok, so I figured it out. Let me know if I am heading down a wrong/stupid track? Here are is my test case which illustrates my requirements. ```c# [Fact] public void Can_replace_text() { var result = _markdownRenderer.TransformMarkdown("test", val => $"{val}-append"); Assert.Equal("<p>test-append</p>", result); result = _markdownRenderer.TransformMarkdown("test*another*", val => $"{val}-append"); Assert.Equal("<p>test-append<em>another-append</em></p>", result); var content = new StringBuilder(); content.AppendLine("a | b"); content.AppendLine("- | -"); content.AppendLine("0 | 0"); result = _markdownRenderer.TransformMarkdown(content.ToString(), val => $"{val}-append"); content = new StringBuilder(); content.AppendLine("<table>"); content.AppendLine("<thead>"); content.AppendLine("<tr>"); content.AppendLine("<th>a-append</th>"); content.AppendLine("<th>b-append</th>"); content.AppendLine("</tr>"); content.AppendLine("</thead>"); content.AppendLine("<tbody>"); content.AppendLine("<tr>"); content.AppendLine("<td>0-append</td>"); content.AppendLine("<td>0-append</td>"); content.AppendLine("</tr>"); content.AppendLine("</tbody>"); content.Append("</table>"); Assert.Equal(content.ToString(), result); } ``` Here is what my internal implementation looks like. ```c# public string TransformMarkdown(string markdown, Func<string, string> replacement) { var pipeline = new MarkdownPipelineBuilder() .UseAdvancedExtensions() .Build(); var document = Markdig.Markdown.Parse(markdown, pipeline); // Do a fake rendering that replaces content. var replacementRenderer = new TextReplacementRenderer(TextWriter.Null, replacement); pipeline.Setup(replacementRenderer); replacementRenderer.Render(document); using (var stringWriter = new StringWriter()) { var htmlRenderer = new HtmlRenderer(stringWriter); pipeline.Setup(htmlRenderer); htmlRenderer.Render(document); stringWriter.Flush(); return stringWriter.ToString().TrimEnd(Environment.NewLine.ToCharArray()); } } ``` The key part in the above code is my custom ```TextReplacementRenderer```. ```c# public class TextReplacementRenderer : TextRendererBase<TextReplacementRenderer> { public TextReplacementRenderer(TextWriter writer, Func<string, string> replacement) : base(writer) { ObjectRenderers.Add(new CustomParagraphRenderer()); ObjectRenderers.Add(new CustomLiteralInlineRenderer(replacement)); } class CustomParagraphRenderer : MarkdownObjectRenderer<TextReplacementRenderer, ParagraphBlock> { protected override void Write(TextReplacementRenderer renderer, ParagraphBlock obj) { renderer.WriteLeafInline(obj); } } class CustomLiteralInlineRenderer : MarkdownObjectRenderer<TextReplacementRenderer, LiteralInline> { readonly Func<string, string> _replacement; public CustomLiteralInlineRenderer(Func<string, string> replacement) { _replacement = replacement; } protected override void Write(TextReplacementRenderer renderer, LiteralInline obj) { obj.Content = new StringSlice(_replacement(obj.Content.ToString())); } } } ``` The main thing I'm worried about is that the start/end/position properties used all over the place would be invalid. Should I be? Is this approach ok?

claunia commented 2026-01-29 14:30:22 +00:00

Author

Owner

Copy Link

@pauldotknopf commented on GitHub (May 6, 2018):

Ok, so I've done some more research, and here are my thoughts. Forgive me for thinking out-loud on a public forum. Any input would be appreciated!

I've implemented a custom renderer that just outputs the the syntax in an easy-to-read structure.

Markdown:

Test paragraph *test* with some em
testt

a | b");
"- | -");
0 | 0");

<div class=\"test\"><img src=\"test.jpg\" /></div>

And [this](/somewhere) is a ~~link~~.
And a new line without a space

Syntax:

<?xml version="1.0" encoding="UTF-8"?>
<MarkdownDocument>
   <ParagraphBlock>
      <ContainerInline>
         <LiteralInline>Test paragraph</LiteralInline>
         <EmphasisInline>
            <LiteralInline>test</LiteralInline>
         </EmphasisInline>
         <LiteralInline>with some em</LiteralInline>
         <LineBreakInline>LineBreakInline</LineBreakInline>
         <LiteralInline>testt</LiteralInline>
      </ContainerInline>
   </ParagraphBlock>
   <Table>
      <TableRow>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>
                  <LiteralInline>a</LiteralInline>
               </ContainerInline>
            </ParagraphBlock>
         </TableCell>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>
                  <LiteralInline>b</LiteralInline>
               </ContainerInline>
            </ParagraphBlock>
         </TableCell>
      </TableRow>
      <TableRow>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>
                  <LiteralInline>0</LiteralInline>
               </ContainerInline>
            </ParagraphBlock>
         </TableCell>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>
                  <LiteralInline>0</LiteralInline>
               </ContainerInline>
            </ParagraphBlock>
         </TableCell>
      </TableRow>
   </Table>
   <HtmlBlock />
   <ParagraphBlock>
      <ContainerInline>
         <LiteralInline>And</LiteralInline>
         <LinkInline>
            <LiteralInline>this</LiteralInline>
         </LinkInline>
         <LiteralInline>is a</LiteralInline>
         <EmphasisInline>
            <LiteralInline>link</LiteralInline>
         </EmphasisInline>
         <LiteralInline>.</LiteralInline>
         <LineBreakInline>LineBreakInline</LineBreakInline>
         <LiteralInline>And a new line without a space</LiteralInline>
      </ContainerInline>
   </ParagraphBlock>
</MarkdownDocument>

I think i've identified the type that will have it's raw contents translated, which is a ContainerInline. I will have a custom renderer that will simple pass the raw markdown through to the new content, but for ContainerInline, the content will be translated.

@pauldotknopf commented on GitHub (May 6, 2018): Ok, so I've done some more research, and here are my thoughts. Forgive me for thinking out-loud on a public forum. Any input would be appreciated! I've implemented a custom renderer that just outputs the the syntax in an easy-to-read structure. **Markdown**: ``` Test paragraph *test* with some em testt a | b"); "- | -"); 0 | 0"); <div class=\"test\"><img src=\"test.jpg\" /></div> And [this](/somewhere) is a ~~link~~. And a new line without a space ``` **Syntax**: ``` <?xml version="1.0" encoding="UTF-8"?> <MarkdownDocument> <ParagraphBlock> <ContainerInline> <LiteralInline>Test paragraph</LiteralInline> <EmphasisInline> <LiteralInline>test</LiteralInline> </EmphasisInline> <LiteralInline>with some em</LiteralInline> <LineBreakInline>LineBreakInline</LineBreakInline> <LiteralInline>testt</LiteralInline> </ContainerInline> </ParagraphBlock> <Table> <TableRow> <TableCell> <ParagraphBlock> <ContainerInline> <LiteralInline>a</LiteralInline> </ContainerInline> </ParagraphBlock> </TableCell> <TableCell> <ParagraphBlock> <ContainerInline> <LiteralInline>b</LiteralInline> </ContainerInline> </ParagraphBlock> </TableCell> </TableRow> <TableRow> <TableCell> <ParagraphBlock> <ContainerInline> <LiteralInline>0</LiteralInline> </ContainerInline> </ParagraphBlock> </TableCell> <TableCell> <ParagraphBlock> <ContainerInline> <LiteralInline>0</LiteralInline> </ContainerInline> </ParagraphBlock> </TableCell> </TableRow> </Table> <HtmlBlock /> <ParagraphBlock> <ContainerInline> <LiteralInline>And</LiteralInline> <LinkInline> <LiteralInline>this</LiteralInline> </LinkInline> <LiteralInline>is a</LiteralInline> <EmphasisInline> <LiteralInline>link</LiteralInline> </EmphasisInline> <LiteralInline>.</LiteralInline> <LineBreakInline>LineBreakInline</LineBreakInline> <LiteralInline>And a new line without a space</LiteralInline> </ContainerInline> </ParagraphBlock> </MarkdownDocument> ``` I think i've identified the type that will have it's raw contents translated, which is a ```ContainerInline```. I will have a custom renderer that will simple pass the raw markdown through to the new content, but for ```ContainerInline```, the content will be translated.

claunia commented 2026-01-29 14:30:22 +00:00

Author

Owner

Copy Link

@pauldotknopf commented on GitHub (May 6, 2018):

@xoofx, can ContainerInline be nested? I'm wondering if I need to support translations with This is {0} a translation where {0} is replaced with the inner ContainerInline.

@pauldotknopf commented on GitHub (May 6, 2018): @xoofx, can ```ContainerInline``` be nested? I'm wondering if I need to support translations with ```This is {0} a translation``` where ```{0}``` is replaced with the inner ```ContainerInline```.

claunia commented 2026-01-29 14:30:22 +00:00

Author

Owner

Copy Link

@pauldotknopf commented on GitHub (May 6, 2018):

I created a custom ContainerInlineRenderer that simply passes through the raw inner markdown. My translation engine will eventually be translating this value, as is.

public class ContainerInlineRenderer : MarkdownObjectRenderer<CustomRenderer, ContainerInline>
{
    protected override void Write(DebugOutputRenderer renderer, ContainerInline obj)
    {
        var start = obj.FirstChild.Span.Start;
        var length = obj.LastChild.Span.End + 1 - start;
        renderer.Write(renderer.OriginalMarkdown.Substring(start, length));
    }
}

My new markdown syntax:

<?xml version="1.0" encoding="UTF-8"?>
<MarkdownDocument>
   <ParagraphBlock>
      <ContainerInline>Test paragraph *test* with some em
testt</ContainerInline>
   </ParagraphBlock>
   <Table>
      <TableRow>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>a</ContainerInline>
            </ParagraphBlock>
         </TableCell>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>b</ContainerInline>
            </ParagraphBlock>
         </TableCell>
      </TableRow>
      <TableRow>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>0</ContainerInline>
            </ParagraphBlock>
         </TableCell>
         <TableCell>
            <ParagraphBlock>
               <ContainerInline>0</ContainerInline>
            </ParagraphBlock>
         </TableCell>
      </TableRow>
   </Table>
   <HtmlBlock />
   <ParagraphBlock>
      <ContainerInline>And [this](/somewhere) is a ~~link~~.
And a new line without a space</ContainerInline>
   </ParagraphBlock>
</MarkdownDocument>

@pauldotknopf commented on GitHub (May 6, 2018): I created a custom ```ContainerInlineRenderer``` that simply passes through the raw inner markdown. My translation engine will eventually be translating this value, as is. ```c# public class ContainerInlineRenderer : MarkdownObjectRenderer<CustomRenderer, ContainerInline> { protected override void Write(DebugOutputRenderer renderer, ContainerInline obj) { var start = obj.FirstChild.Span.Start; var length = obj.LastChild.Span.End + 1 - start; renderer.Write(renderer.OriginalMarkdown.Substring(start, length)); } } ``` My new markdown syntax: ```xml <?xml version="1.0" encoding="UTF-8"?> <MarkdownDocument> <ParagraphBlock> <ContainerInline>Test paragraph *test* with some em testt</ContainerInline> </ParagraphBlock> <Table> <TableRow> <TableCell> <ParagraphBlock> <ContainerInline>a</ContainerInline> </ParagraphBlock> </TableCell> <TableCell> <ParagraphBlock> <ContainerInline>b</ContainerInline> </ParagraphBlock> </TableCell> </TableRow> <TableRow> <TableCell> <ParagraphBlock> <ContainerInline>0</ContainerInline> </ParagraphBlock> </TableCell> <TableCell> <ParagraphBlock> <ContainerInline>0</ContainerInline> </ParagraphBlock> </TableCell> </TableRow> </Table> <HtmlBlock /> <ParagraphBlock> <ContainerInline>And [this](/somewhere) is a ~~link~~. And a new line without a space</ContainerInline> </ParagraphBlock> </MarkdownDocument> ```

claunia commented 2026-01-29 14:30:24 +00:00

Author

Owner

Copy Link

@pauldotknopf commented on GitHub (May 7, 2018):

I finished the library for translating markdown files here. Looking at the tests can give you a good idea of what it does.

With that said, I have effectively answered the original question.

Thanks for listening to me :)

@pauldotknopf commented on GitHub (May 7, 2018): I finished the library for translating markdown files [here](https://github.com/pauldotknopf/markdown-translator). Looking at the [tests](https://github.com/pauldotknopf/markdown-translator/blob/develop/test/MarkdownTranslator.Tests/MarkdownTransformerTests.cs#L87) can give you a good idea of what it does. With that said, I have effectively answered the original question. Thanks for listening to me :)

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

example-lunet-api-doc

jira-markup

perf-core-vs-mono

0.44.0

0.43.0

0.42.0

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.1

0.39.0

0.38.0

0.37.0

0.36.2

0.36.1

0.36.0

0.35.0

0.34.0

0.33.0

0.32.0

0.31.0

0.30.4

0.30.3

0.30.2

0.30.1

0.30.0

0.29.0

0.28.1

0.28.0

0.27.0

0.26.0

0.25.0

0.24.0

0.23.0

0.22.1

0.22.0

0.21.1

0.21.0

0.20.0

0.18.3

0.18.2

0.18.1

0.18.0

0.17.1

0.17.0

v0.16.0

v0.15.7

v0.15.6

v0.15.5

v0.15.4

v0.15.3

v0.15.2

v0.15.1

v0.15.0

v0.14.9

v0.14.8

v0.14.7

v0.14.6

v0.14.5

v0.14.4

v0.14.3

v0.14.2

v0.14.1

v0.14.0

v0.13.4

v0.13.3

v0.13.2

v0.13.1

v0.13.0

v0.12.4

v0.12.3

v0.12.2

v0.12.1

v0.12.0

v0.11.0

v0.10.7

v0.10.6

v0.10.5

v0.10.4

v0.10.3

v0.10.2

v0.10.1

v0.10.0

v0.9.1

v0.9.0

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.5

v0.7.4

v0.7.3

v0.7.2

v0.7.1

v0.7.0

v0.6.2

v0.6.1

v0.6.0

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.0

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.1

v0.2.0

v0.1.1

v0.1.0

Labels

Clear labels

bug

documentation

enhancement

extension

Hacktoberfest

invalid

PR Welcome!

pull-request

Mirrored from GitHub Pull Request

question

wontfix

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

claunia

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/markdig#205