Disable warning for comments as it slows down build on appveyor

Fixes issue where LiteralInlineParser calls PostMatch while processor.Inline is type other than LiteralInline

readme.md

@@ -46,6 +46,8 @@ You can **try Markdig online** and compare it to other implementations on [babel
   - **Emoji** support (inspired from [Markdown-it](https://markdown-it.github.io/))
   - **SmartyPants** (inspired from [Daring Fireball - SmartyPants](https://daringfireball.net/projects/smartypants/))
   - **Bootstrap** class (to output bootstrap class)
+  - **Diagrams** extension whenever a fenced code block contains a special keyword, it will be converted to a div block with the content as-is (currently, supports only for [`mermaid` diagrams](https://knsv.github.io/mermaid/))
+  - **YAML frontmatter** to parse without evaluating the frontmatter and to discard it from the HTML output (typically used for previewing without the frontmatter in MarkdownEditor)
 - Compatible with .NET 3.5, 4.0+ and .NET Core (`netstandard1.1+`)
 	
 ## Documentation

src/Markdig.Tests/Markdig.Tests.csproj

@@ -89,6 +89,7 @@
     <None Include="Specs\BootstrapSpecs.md" />
     <None Include="Specs\DiagramsSpecs.md" />
     <None Include="Specs\NoHtmlSpecs.md" />
+    <None Include="Specs\YamlSpecs.md" />
     <None Include="Specs\TaskListSpecs.md" />
     <None Include="Specs\SmartyPantsSpecs.md" />
     <None Include="Specs\MediaSpecs.md" />

src/Markdig.Tests/Specs/AbbreviationSpecs.md

@@ -57,3 +57,13 @@ We can abbreviate 1A, 1A1 and 1A2!
 .
 <p>We can abbreviate <abbr title="First">1A</abbr>, <abbr title="Second">1A1</abbr> and <abbr title="Third">1A2</abbr>!</p>
 ````````````````````````````````
+
+Abbreviations should match whole word only:
+
+```````````````````````````````` example
+*[1A]: First
+
+We should not abbreviate 1.1A or 11A!
+.
+<p>We should not abbreviate 1.1A or 11A!</p>
+````````````````````````````````

src/Markdig.Tests/Specs/GridTableSpecs.md

@@ -272,4 +272,15 @@ A grid table may not have irregularly shaped cells:
 +---+---+---+
 | DDDDD | E |
 +---+---+---+</p>
-````````````````````````````````
+````````````````````````````````
+
+An empty `+` on a line should result in a simple empty list output:
+
+
+```````````````````````````````` example
++
+.
+<ul>
+<li></li>
+</ul>
+````````````````````````````````

src/Markdig.Tests/Specs/PipeTableSpecs.md

@@ -346,7 +346,8 @@ The first row is considered as a **header row** if it is separated from the regu
 </table>
 ````````````````````````````````
 
-The text alignment is defined by default to be left.
+The text alignment is defined by default to be center for header and left for cells. If the left alignment is applied, it will force the column heading to be left aligned.
+There is no way to define a different alignment for heading and cells (apart from the default).
 The text alignment can be changed by using the character `:` with the header column separator:
  
 ```````````````````````````````` example
@@ -358,19 +359,19 @@ The text alignment can be changed by using the character `:` with the header col
 <table>
 <thead>
 <tr>
-<th>a</th>
+<th style="text-align: left;">a</th>
 <th style="text-align: center;">b</th>
 <th style="text-align: right;">c</th>
 </tr>
 </thead>
 <tbody>
 <tr>
-<td>0</td>
+<td style="text-align: left;">0</td>
 <td style="text-align: center;">1</td>
 <td style="text-align: right;">2</td>
 </tr>
 <tr>
-<td>3</td>
+<td style="text-align: left;">3</td>
 <td style="text-align: center;">4</td>
 <td style="text-align: right;">5</td>
 </tr>
@@ -514,7 +515,7 @@ Tests trailing spaces after pipes
 
 ```````````````````````````````` example
 | abc | def | 
-|---|:---|
+|---|---|
 | cde| ddd| 
 | eee| fff|
 | fff | fffff   | 

src/Markdig.Tests/Specs/Specs.tt

@@ -39,7 +39,7 @@ SOFTWARE.
 <#@ output extension=".cs" #><#
     var specFiles = new KeyValuePair<string, string>[] {
 		// new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/master/spec.txt", string.Empty),
-		new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/91e045ca370258903ed138450373043a496ec64b/spec.txt", string.Empty), // 0.26 specs
+		new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/cfc84164475d3bec8be9482c21a705adc93a54f5/spec.txt", string.Empty), // 0.26 specs
 		new KeyValuePair<string, string>(Host.ResolvePath("PipeTableSpecs.md"), "pipetables|advanced"),
 		new KeyValuePair<string, string>(Host.ResolvePath("FootnotesSpecs.md"), "footnotes|advanced"),
 		new KeyValuePair<string, string>(Host.ResolvePath("GenericAttributesSpecs.md"), "attributes|advanced"),
@@ -60,6 +60,7 @@ SOFTWARE.
 		new KeyValuePair<string, string>(Host.ResolvePath("TaskListSpecs.md"), "tasklists|advanced"),
 		new KeyValuePair<string, string>(Host.ResolvePath("DiagramsSpecs.md"), "diagrams|advanced"),
 		new KeyValuePair<string, string>(Host.ResolvePath("NoHtmlSpecs.md"), "nohtml"),
+		new KeyValuePair<string, string>(Host.ResolvePath("YamlSpecs.md"), "yaml"),
 	};
     var emptyLines = false;
     var displayEmptyLines = false;

src/Markdig.Tests/Specs/YamlSpecs.md

@@ -0,0 +1,44 @@
+# Extensions
+
+Adds support for YAML frontmatter parsing:
+
+## YAML frontmatter discard
+ 
+If a frontmatter is present, it will not be rendered:
+
+```````````````````````````````` example
+---
+this: is a frontmatter
+---
+This is a text
+.
+<p>This is a text</p>
+````````````````````````````````
+ 
+But if a frontmatter doesn't happen on the first line, it will be parse as regular Markdown content
+
+```````````````````````````````` example
+This is a text1
+---
+this: is a frontmatter
+---
+This is a text2
+.
+<h2>This is a text1</h2>
+<h2>this: is a frontmatter</h2>
+<p>This is a text2</p>
+````````````````````````````````
+
+It expects an exact 3 dashes `---`:
+
+```````````````````````````````` example
+----
+this: is a frontmatter
+----
+This is a text
+.
+<hr />
+<h2>this: is a frontmatter</h2>
+<p>This is a text</p>
+````````````````````````````````
+

src/Markdig.Tests/TestParser.cs

@@ -15,25 +15,30 @@ namespace Markdig.Tests
             foreach (var pipeline in GetPipeline(extensions))
             {
                 Console.WriteLine($"Pipeline configured with extensions: {pipeline.Key}");
-                // Uncomment this line to get more debug information for process inlines.
-                //pipeline.DebugLog = Console.Out;
-                var result = Markdown.ToHtml(inputText, pipeline.Value);
-
-                result = Compact(result);
-                expectedOutputText = Compact(expectedOutputText);
-
-                Console.WriteLine("```````````````````Source");
-                Console.WriteLine(DisplaySpaceAndTabs(inputText));
-                Console.WriteLine("```````````````````Result");
-                Console.WriteLine(DisplaySpaceAndTabs(result));
-                Console.WriteLine("```````````````````Expected");
-                Console.WriteLine(DisplaySpaceAndTabs(expectedOutputText));
-                Console.WriteLine("```````````````````");
-                Console.WriteLine();
-                TextAssert.AreEqual(expectedOutputText, result);
+                TestSpec(inputText, expectedOutputText, pipeline.Value);
             }
         }
 
+        public static void TestSpec(string inputText, string expectedOutputText, MarkdownPipeline pipeline)
+        {
+            // Uncomment this line to get more debug information for process inlines.
+            //pipeline.DebugLog = Console.Out;
+            var result = Markdown.ToHtml(inputText, pipeline);
+
+            result = Compact(result);
+            expectedOutputText = Compact(expectedOutputText);
+
+            Console.WriteLine("```````````````````Source");
+            Console.WriteLine(DisplaySpaceAndTabs(inputText));
+            Console.WriteLine("```````````````````Result");
+            Console.WriteLine(DisplaySpaceAndTabs(result));
+            Console.WriteLine("```````````````````Expected");
+            Console.WriteLine(DisplaySpaceAndTabs(expectedOutputText));
+            Console.WriteLine("```````````````````");
+            Console.WriteLine();
+            TextAssert.AreEqual(expectedOutputText, result);
+        }
+
         private static IEnumerable<KeyValuePair<string, MarkdownPipeline>> GetPipeline(string extensionsGroupText)
         {
             // For the standard case, we make sure that both the CommmonMark core and Extra/Advanced are CommonMark compliant!

src/Markdig.Tests/TestPlayParser.cs

@@ -2,6 +2,9 @@
 // This file is licensed under the BSD-Clause 2 license. 
 // See the license.txt file in the project root for more information.
 using System;
+using System.Linq;
+using Markdig.Syntax;
+using Markdig.Syntax.Inlines;
 using NUnit.Framework;
 
 namespace Markdig.Tests
@@ -27,6 +30,17 @@ Later in a text we are using HTML and it becomes an abbr tag HTML
             Console.WriteLine(result);
         }
 
+        [Test]
+        public void TestEmptyLiteral()
+        {
+            var text = @"> *some text* 
+> some other text";
+            var doc = Markdown.Parse(text);
+
+            Assert.True(doc.Descendants().OfType<LiteralInline>().All(x => !x.Content.IsEmpty),
+                "There should not have any empty literals");
+        }
+
         [Test]
         public void TestSelfPipeline1()
         {
@@ -96,6 +110,12 @@ blabla
         }
 
 
+        [Test]
+        public void TestStandardUriEscape()
+        {
+            TestParser.TestSpec(@"![你好](你好.png)", "<p><img src=\"你好.png\" alt=\"你好\" /></p>", "nonascii-noescape");
+        }
+
 
         [Test]
         public void TestBugAdvancaed()

src/Markdig.Tests/TestSourcePosition.cs

@@ -659,6 +659,20 @@ code         ( 2, 0)  3-13
 ");
         }
 
+        [Test]
+        public void TestIndentedCodeAfterList()
+        {
+            //     0         1           2         3          4         5
+            //     012345678901234567 8 901234567890123456 789012345678901234 56789
+            Check("1) Some list item\n\n        some code\n        more code\n", @"
+list         ( 0, 0)  0-53
+listitem     ( 0, 0)  0-53
+paragraph    ( 0, 3)  3-16
+literal      ( 0, 3)  3-16
+code         ( 2, 0) 19-53
+");
+        }
+
         [Test]
         public void TestIndentedCodeWithTabs()
         {

src/Markdig/Extensions/Abbreviations/AbbreviationParser.cs

@@ -101,20 +101,9 @@ namespace Markdig.Extensions.Abbreviations
                 for (int i = content.Start; i < content.End; i++)
                 {
                     string match;
-                    if (matcher.TryMatch(text, i, content.End - i + 1, out match))
+                    if (matcher.TryMatch(text, i, content.End - i + 1, out match) && IsValidAbbreviation(match, content, i))
                     {
-                        // The word matched must be embraced by punctuation or whitespace or \0.
-                        var c = content.PeekCharAbsolute(i - 1);
-                        if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))
-                        {
-                            continue;
-                        }
                         var indexAfterMatch = i + match.Length;
-                        c = content.PeekCharAbsolute(indexAfterMatch);
-                        if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))
-                        {
-                            continue;
-                        }
 
                         // We should have a match, but in case...
                         Abbreviation abbr;
@@ -196,5 +185,39 @@ namespace Markdig.Extensions.Abbreviations
                 }
             };
         }
+
+        private static bool IsValidAbbreviation(string match, StringSlice content, int matchIndex)
+        {
+            // The word matched must be embraced by punctuation or whitespace or \0.
+            var index = matchIndex - 1;
+            while (index > content.Start)
+            {
+                var c = content.PeekCharAbsolute(index);
+                if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))
+                {
+                    return false;
+                }
+                if (!c.IsAsciiPunctuation())
+                {
+                    break;
+                }
+                index--;
+            }
+            index = matchIndex + match.Length;
+            while (index < content.End)
+            {
+                var c = content.PeekCharAbsolute(index);
+                if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))
+                {
+                    return false;
+                }
+                if (!c.IsAsciiPunctuation())
+                {
+                    break;
+                }
+                index++;
+            }
+            return true;
+        }
     }
 }

src/Markdig/Extensions/DefinitionLists/HtmlDefinitionListRenderer.cs

@@ -49,7 +49,7 @@ namespace Markdig.Extensions.DefinitionLists
                     {
                         if (!hasOpendd)
                         {
-                            renderer.Write("<dd>");
+                            renderer.Write("<dd").WriteAttributes(definitionItem).Write(">");
                             countdd = 0;
                             hasOpendd = true;
                         }

src/Markdig/Extensions/NonAsciiNoEscape/NonAsciiNoEscapeExtension.cs

@@ -0,0 +1,27 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license. 
+// See the license.txt file in the project root for more information.
+
+using Markdig.Renderers;
+
+namespace Markdig.Extensions.NonAsciiNoEscape
+{
+    /// <summary>
+    /// Extension that will disable URI escape with % characters for non-US-ASCII characters in order to workaround a bug under IE/Edge with local file links containing non US-ASCII chars. DO NOT USE OTHERWISE.
+    /// </summary>
+    public class NonAsciiNoEscapeExtension : IMarkdownExtension
+    {
+        public void Setup(MarkdownPipelineBuilder pipeline)
+        {
+        }
+
+        public void Setup(IMarkdownRenderer renderer)
+        {
+            var htmlRenderer = renderer as HtmlRenderer;
+            if (htmlRenderer != null)
+            {
+                htmlRenderer.UseNonAsciiNoEscape = true;
+            }
+        }
+    }
+}

src/Markdig/Extensions/Tables/GridTableParser.cs

@@ -27,32 +27,36 @@ namespace Markdig.Extensions.Tables
             var line = processor.Line;
             GridTableState tableState = null;
 
+            // Match the first row that should be of the minimal form: +---------------
             var c = line.CurrentChar;
             var lineStart = line.Start;
-            int startPosition = -1;
-            bool hasLeft = false,
-                 hasRight = false;
-
-            while (line.Length > 0)
+            while (c == '+')
             {
-                if (c == '+')
+                var columnStart = line.Start;
+                line.NextChar();
+                line.TrimStart();
+
+                // if we have reached the end of the line, exit
+                c = line.CurrentChar;
+                if (c == 0)
                 {
-                    tableState = tableState ?? new GridTableState { Start = processor.Start, ExpectRow = true };
-                    if (startPosition != -1)
-                    {
-                        hasRight = line.PeekCharAbsolute(line.Start - 1) == ':';
-                        tableState.AddColumn(startPosition - lineStart, line.Start - lineStart, GetAlignment(hasLeft, hasRight));
-                    }
-                    hasLeft = line.PeekChar(1) == ':';
-                    startPosition = line.Start;
+                    break;
                 }
-                else if (c != ':' && c != '-')
+
+                // Parse a column alignment
+                TableColumnAlign? columnAlign;
+                if (!TableHelper.ParseColumnHeader(ref line, '-', out columnAlign))
                 {
                     return BlockState.None;
                 }
-                c = line.NextChar();
+
+                tableState = tableState ?? new GridTableState { Start = processor.Start, ExpectRow = true };
+                tableState.AddColumn(columnStart - lineStart, line.Start - lineStart, columnAlign);
+
+                c = line.CurrentChar;
             }
-            if (tableState == null || tableState.ColumnSlices.Count == (processor.Line.Length - 1))
+
+            if (c != 0 || tableState == null)
             {
                 return BlockState.None;
             }
@@ -85,13 +89,6 @@ namespace Markdig.Extensions.Tables
             return BlockState.ContinueDiscard;
         }
 
-        private static TableColumnAlign GetAlignment(bool hasLeft, bool hasRight)
-        {
-            return hasLeft && hasRight
-                ? TableColumnAlign.Center
-                : hasRight ? TableColumnAlign.Right : TableColumnAlign.Left;
-        }
-
         public override BlockState TryContinue(BlockProcessor processor, Block block)
         {
             var gridTable = (Table)block;

src/Markdig/Extensions/Tables/GridTableState.cs

@@ -31,7 +31,7 @@ namespace Markdig.Extensions.Tables
             Lines.Add(line);
         }
 
-        public void AddColumn(int start, int end, TableColumnAlign align)
+        public void AddColumn(int start, int end, TableColumnAlign? align)
         {
             if (ColumnSlices == null)
             {
@@ -60,7 +60,7 @@ namespace Markdig.Extensions.Tables
 
             public int End { get; set; }
 
-            public TableColumnAlign Align { get; set; }
+            public TableColumnAlign? Align { get; set; }
 
             public int CurrentColumnSpan { get; set; }
 

src/Markdig/Extensions/Tables/HtmlTableRenderer.cs

@@ -91,14 +91,21 @@ namespace Markdig.Extensions.Tables
                             ? i
                             : cell.ColumnIndex;
                         columnIndex = columnIndex >= table.ColumnDefinitions.Count ? table.ColumnDefinitions.Count - 1 : columnIndex;
-                        switch (table.ColumnDefinitions[columnIndex].Alignment)
+                        var alignment = table.ColumnDefinitions[columnIndex].Alignment;
+                        if (alignment.HasValue)
                         {
-                            case TableColumnAlign.Center:
-                                renderer.Write(" style=\"text-align: center;\"");
-                                break;
-                            case TableColumnAlign.Right:
-                                renderer.Write(" style=\"text-align: right;\"");
-                                break;
+                            switch (alignment)
+                            {
+                                case TableColumnAlign.Center:
+                                    renderer.Write(" style=\"text-align: center;\"");
+                                    break;
+                                case TableColumnAlign.Right:
+                                    renderer.Write(" style=\"text-align: right;\"");
+                                    break;
+                                case TableColumnAlign.Left:
+                                    renderer.Write(" style=\"text-align: left;\"");
+                                    break;
+                            }
                         }
                     }
                     renderer.WriteAttributes(cell);

src/Markdig/Extensions/Tables/PipeTableParser.cs

@@ -453,7 +453,7 @@ namespace Markdig.Extensions.Tables
             return false;
         }
 
-        private static bool ParseHeaderString(Inline inline, out TableColumnAlign align)
+        private static bool ParseHeaderString(Inline inline, out TableColumnAlign? align)
         {
             align = 0;
             var literal = inline as LiteralInline;
@@ -500,7 +500,7 @@ namespace Markdig.Extensions.Tables
                     }
 
                     // Check the left side of a `|` delimiter
-                    TableColumnAlign align = TableColumnAlign.Left;
+                    TableColumnAlign? align = null;
                     if (delimiter.PreviousSibling != null && !ParseHeaderString(delimiter.PreviousSibling, out align))
                     {
                         break;

src/Markdig/Extensions/Tables/TableColumnDefinition.cs

@@ -17,6 +17,6 @@ namespace Markdig.Extensions.Tables
         /// <summary>
         /// Gets or sets the column alignment.
         /// </summary>
-        public TableColumnAlign Alignment { get; set; }
+        public TableColumnAlign? Alignment { get; set; }
     }
 }

src/Markdig/Extensions/Tables/TableHelper.cs

@@ -20,7 +20,7 @@ namespace Markdig.Extensions.Tables
         /// <returns>
         ///   <c>true</c> if parsing was successfull
         /// </returns>
-        public static bool ParseColumnHeader(ref StringSlice slice, char delimiterChar, out TableColumnAlign align)
+        public static bool ParseColumnHeader(ref StringSlice slice, char delimiterChar, out TableColumnAlign? align)
         {
             return ParseColumnHeaderDetect(ref slice, ref delimiterChar, out align);
         }
@@ -34,7 +34,7 @@ namespace Markdig.Extensions.Tables
         /// <returns>
         ///   <c>true</c> if parsing was successfull
         /// </returns>
-        public static bool ParseColumnHeaderAuto(ref StringSlice slice, out char delimiterChar, out TableColumnAlign align)
+        public static bool ParseColumnHeaderAuto(ref StringSlice slice, out char delimiterChar, out TableColumnAlign? align)
         {
             delimiterChar = '\0';
             return ParseColumnHeaderDetect(ref slice, ref delimiterChar, out align);
@@ -49,11 +49,10 @@ namespace Markdig.Extensions.Tables
         /// <returns>
         ///   <c>true</c> if parsing was successfull
         /// </returns>
-        public static bool ParseColumnHeaderDetect(ref StringSlice slice, ref char delimiterChar, out TableColumnAlign align)
+        public static bool ParseColumnHeaderDetect(ref StringSlice slice, ref char delimiterChar, out TableColumnAlign? align)
         {
-            align = TableColumnAlign.Left;
+            align = null;
 
-            // Work on a copy of the slice
             slice.TrimStart();
             var c = slice.CurrentChar;
             bool hasLeft = false;
@@ -87,6 +86,7 @@ namespace Markdig.Extensions.Tables
                 count++;
             }
 
+            // We expect at least one `-` delimiter char
             if (count == 0)
             {
                 return false;
@@ -104,7 +104,7 @@ namespace Markdig.Extensions.Tables
 
             align = hasLeft && hasRight
                 ? TableColumnAlign.Center
-                : hasRight ? TableColumnAlign.Right : TableColumnAlign.Left;
+                : hasRight ? TableColumnAlign.Right : hasLeft ? TableColumnAlign.Left : (TableColumnAlign?) null;
 
             return true;
         }

src/Markdig/Extensions/Yaml/YamlFrontMatterBlock.cs

@@ -0,0 +1,31 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license. 
+// See the license.txt file in the project root for more information.
+using Markdig.Parsers;
+using Markdig.Syntax;
+
+namespace Markdig.Extensions.Yaml
+{
+    /// <summary>
+    /// A YAML frontmatter block.
+    /// </summary>
+    /// <seealso cref="Markdig.Syntax.CodeBlock" />
+    public class YamlFrontMatterBlock : CodeBlock, IFencedBlock
+    {
+        /// <summary>
+        /// Initializes a new instance of the <see cref="YamlFrontMatterBlock"/> class.
+        /// </summary>
+        /// <param name="parser">The parser.</param>
+        public YamlFrontMatterBlock(BlockParser parser) : base(parser)
+        {
+        }
+
+        public string Info { get; set; }
+
+        public string Arguments { get; set; }
+
+        public int FencedCharCount { get; set; }
+
+        public char FencedChar { get; set; }
+    }
+}

src/Markdig/Extensions/Yaml/YamlFrontMatterExtension.cs

@@ -0,0 +1,33 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license. 
+// See the license.txt file in the project root for more information.
+
+using Markdig.Parsers;
+using Markdig.Renderers;
+using Markdig.Renderers.Html;
+
+namespace Markdig.Extensions.Yaml
+{
+    /// <summary>
+    /// Extension to discard a YAML frontmatter at the beginning of a Markdown document.
+    /// </summary>
+    public class YamlFrontMatterExtension : IMarkdownExtension
+    {
+        public void Setup(MarkdownPipelineBuilder pipeline)
+        {
+            if (!pipeline.BlockParsers.Contains<YamlFrontMatterParser>())
+            {
+                // Insert the YAML parser before the thematic break parser, as it is also triggered on a --- dash
+                pipeline.BlockParsers.InsertBefore<ThematicBreakParser>(new YamlFrontMatterParser());
+            }
+        }
+
+        public void Setup(IMarkdownRenderer renderer)
+        {
+            if (!renderer.ObjectRenderers.Contains<YamlFrontMatterRenderer>())
+            {
+                renderer.ObjectRenderers.InsertBefore<CodeBlockRenderer>(new YamlFrontMatterRenderer());
+            }
+        }
+    }
+}

src/Markdig/Extensions/Yaml/YamlFrontMatterParser.cs

@@ -0,0 +1,44 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license. 
+// See the license.txt file in the project root for more information.
+using Markdig.Parsers;
+
+namespace Markdig.Extensions.Yaml
+{
+    /// <summary>
+    /// Block parser for a YAML frontmatter. 
+    /// </summary>
+    /// <seealso cref="Markdig.Parsers.FencedBlockParserBase{YamlFrontMatterBlock}" />
+    public class YamlFrontMatterParser : FencedBlockParserBase<YamlFrontMatterBlock>
+    {
+        // We reuse a FencedCodeBlock parser to grab a frontmatter, only active if it happens on the first line of the document.
+
+        /// <summary>
+        /// Initializes a new instance of the <see cref="FencedCodeBlockParser"/> class.
+        /// </summary>
+        public YamlFrontMatterParser()
+        {
+            OpeningCharacters = new[] { '-' };
+            InfoPrefix = null;
+            // We expect only 3 --- at the beginning of the file no more, no less
+            MinimumMatchCount = 3;
+            MaximumMatchCount = 3;
+        }
+
+        protected override YamlFrontMatterBlock CreateFencedBlock(BlockProcessor processor)
+        {
+            return new YamlFrontMatterBlock(this);
+        }
+
+        public override BlockState TryOpen(BlockProcessor processor)
+        {
+            // Only accept a frontmatter at the beginning of the file
+            if (processor.LineIndex != 0)
+            {
+                return BlockState.None;
+            }
+
+            return base.TryOpen(processor);
+        }
+    }
+}

src/Markdig/Extensions/Yaml/YamlFrontMatterRenderer.cs

@@ -0,0 +1,19 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license. 
+// See the license.txt file in the project root for more information.
+using Markdig.Renderers;
+using Markdig.Renderers.Html;
+
+namespace Markdig.Extensions.Yaml
+{
+    /// <summary>
+    /// Empty renderer for a <see cref="YamlFrontMatterBlock"/>
+    /// </summary>
+    /// <seealso cref="Markdig.Renderers.Html.HtmlObjectRenderer{YamlFrontMatterBlock}" />
+    public class YamlFrontMatterRenderer : HtmlObjectRenderer<YamlFrontMatterBlock>
+    {
+        protected override void Write(HtmlRenderer renderer, YamlFrontMatterBlock obj)
+        {
+        }
+    }
+}

src/Markdig/MarkdownExtensions.cs

@@ -24,8 +24,10 @@ using Markdig.Extensions.NoRefLinks;
 using Markdig.Extensions.PragmaLines;
 using Markdig.Extensions.SelfPipeline;
 using Markdig.Extensions.SmartyPants;
+using Markdig.Extensions.NonAsciiNoEscape;
 using Markdig.Extensions.Tables;
 using Markdig.Extensions.TaskLists;
+using Markdig.Extensions.Yaml;
 using Markdig.Parsers;
 using Markdig.Parsers.Inlines;
 
@@ -63,6 +65,28 @@ namespace Markdig
                 .UseGenericAttributes(); // Must be last as it is one parser that is modifying other parsers
         }
 
+        /// <summary>
+        /// Uses this extension to disable URI escape with % characters for non-US-ASCII characters in order to workaround a bug under IE/Edge with local file links containing non US-ASCII chars. DO NOT USE OTHERWISE.
+        /// </summary>
+        /// <param name="pipeline">The pipeline.</param>
+        /// <returns>The modified pipeline</returns>
+        public static MarkdownPipelineBuilder UseNonAsciiNoEscape(this MarkdownPipelineBuilder pipeline)
+        {
+            pipeline.Extensions.AddIfNotAlready<NonAsciiNoEscapeExtension>();
+            return pipeline;
+        }
+
+        /// <summary>
+        /// Uses YAML frontmatter extension that will parse a YAML frontmatter into the MarkdownDocument. Note that they are not rendered by any default HTML renderer.
+        /// </summary>
+        /// <param name="pipeline">The pipeline.</param>
+        /// <returns>The modified pipeline</returns>
+        public static MarkdownPipelineBuilder UseYamlFrontMatter(this MarkdownPipelineBuilder pipeline)
+        {
+            pipeline.Extensions.AddIfNotAlready<YamlFrontMatterExtension>();
+            return pipeline;
+        }
+
         /// <summary>
         /// Uses the self pipeline extension that will detect the pipeline to use from the markdown input that contains a special tag. See <see cref="SelfPipelineExtension"/>
         /// </summary>
@@ -412,6 +436,8 @@ namespace Markdig
                 return pipeline;
             }
 
+            // TODO: the extension string should come from the extension itself instead of this hardcoded switch case.
+
             foreach (var extension in extensions.Split(new[] { '+' }, StringSplitOptions.RemoveEmptyEntries))
             {
                 switch (extension.ToLowerInvariant())
@@ -490,6 +516,12 @@ namespace Markdig
                     case "nohtml":
                         pipeline.DisableHtml();
                         break;
+                    case "yaml":
+                        pipeline.UseYamlFrontMatter();
+                        break;
+                    case "nonascii-noescape":
+                        pipeline.UseNonAsciiNoEscape();
+                        break;
                     default:
                         throw new ArgumentException($"Invalid extension `{extension}` from `{extensions}`", nameof(extensions));
                 }

src/Markdig/Parsers/BlockProcessor.cs

@@ -304,6 +304,64 @@ namespace Markdig.Parsers
             }
         }
 
+        /// <summary>
+        /// Unwind any previous indent from the current character back to the first space.
+        /// </summary>
+        public void UnwindAllIndents()
+        {
+            // Find the previous first space on the current line
+            var previousStart = Line.Start;
+            for (; Line.Start > originalLineStart; Line.Start--)
+            {
+                var c = Line.PeekCharAbsolute(Line.Start - 1);
+                if (c == 0)
+                {
+                    break;
+                }
+                if (!c.IsSpaceOrTab())
+                {
+                    break;
+                }
+            }
+            var targetStart = Line.Start;
+            // Nothing changed? Early exit
+            if (previousStart == targetStart)
+            {
+                return;
+            }
+
+            // TODO: factorize the following code with what is done with GoToColumn
+
+            // If we have found the first space, we need to recalculate the correct column
+            Line.Start = originalLineStart;
+            Column = 0;
+            ColumnBeforeIndent = 0;
+            StartBeforeIndent = originalLineStart;
+
+            for (; Line.Start < targetStart; Line.Start++)
+            {
+                var c = Line.Text[Line.Start];
+                if (c == '\t')
+                {
+                    Column = CharHelper.AddTab(Column);
+                }
+                else
+                {
+                    if (!c.IsSpaceOrTab())
+                    {
+                        ColumnBeforeIndent = Column + 1;
+                        StartBeforeIndent = Line.Start + 1;
+                    }
+
+                    Column++;
+                }
+            }
+
+            // Reset the indent
+            ColumnBeforeIndent = Column;
+            StartBeforeIndent = Start;
+        }
+
         /// <summary>
         /// Moves to the position to the code indent (<see cref="ColumnBeforeIndent"/> + 4 spaces).
         /// </summary>

src/Markdig/Parsers/IndentedCodeBlockParser.cs

@@ -18,16 +18,23 @@ namespace Markdig.Parsers
 
         public override BlockState TryOpen(BlockProcessor processor)
         {
-            var startColumn = processor.ColumnBeforeIndent;
-            var startPosition = processor.StartBeforeIndent;
             var result = TryContinue(processor, null);
             if (result == BlockState.Continue)
             {
+                // Save the column where we need to go back
+                var column = processor.Column;
+
+                // Unwind all indents all spaces before in order to calculate correct span
+                processor.UnwindAllIndents();
+
                 processor.NewBlocks.Push(new CodeBlock(this)
                 {
-                    Column = startColumn,
-                    Span = new SourceSpan(startPosition, processor.Line.End)
+                    Column = processor.Column,
+                    Span = new SourceSpan(processor.Start, processor.Line.End)
                 });
+
+                // Go back to the correct column
+                processor.GoToColumn(column);
             }
             return result;
         }

src/Markdig/Parsers/Inlines/LinkInlineParser.cs

@@ -211,6 +211,7 @@ namespace Markdig.Parsers.Inlines
                 // compact reference link/image, 
                 // or shortcut reference link/image
                 var parentDelimiter = openParent.Parent;
+                var savedText = text;
                 switch (text.CurrentChar)
                 {
                     case '(':
@@ -253,9 +254,10 @@ namespace Markdig.Parsers.Inlines
 
                             return true;
                         }
-                        break;
-                    default:
 
+                        text = savedText;
+                        goto default;
+                    default:
                         var labelSpan = SourceSpan.Empty;
                         string label = null;
                         bool isLabelSpanLocal = true;

src/Markdig/Parsers/Inlines/LiteralInlineParser.cs

@@ -75,21 +75,26 @@ namespace Markdig.Parsers.Inlines
             }
             else
             {
-                processor.Inline = new LiteralInline()
+                // Create a new LiteralInline only if it is not empty
+                var newSlice = length > 0 ? new StringSlice(slice.Text, slice.Start, endPosition) : StringSlice.Empty;
+                if (!newSlice.IsEmpty)
                 {
-                    Content = length > 0 ? new StringSlice(slice.Text, slice.Start, endPosition) : StringSlice.Empty,
-                    Span = new SourceSpan(startPosition, processor.GetSourcePosition(endPosition)),
-                    Line = line,
-                    Column = column,
-                };
+                    processor.Inline = new LiteralInline()
+                    {
+                        Content = length > 0 ? newSlice : StringSlice.Empty,
+                        Span = new SourceSpan(startPosition, processor.GetSourcePosition(endPosition)),
+                        Line = line,
+                        Column = column,
+                    };
+                }
             }
 
             slice.Start = nextStart;
 
             // Call only PostMatch if necessary
-            if (PostMatch != null)
+            if (processor.Inline is LiteralInline)
             {
-                PostMatch(processor, ref slice);
+                PostMatch?.Invoke(processor, ref slice);
             }
 
             return true;

src/Markdig/Properties/AssemblyInfo.cs

@@ -25,6 +25,6 @@ namespace Markdig
 {
     public static partial class Markdown
     {
-        public const string Version = "0.7.5";
+        public const string Version = "0.8.5";
     }
 }

src/Markdig/Renderers/HtmlRenderer.cs

@@ -64,6 +64,8 @@ namespace Markdig.Renderers
         /// </summary>
         public bool ImplicitParagraph { get; set; }
 
+        public bool UseNonAsciiNoEscape { get; set; }
+
         /// <summary>
         /// Writes the content escaped for HTML.
         /// </summary>
@@ -203,22 +205,29 @@ namespace Markdig.Renderers
                     Write(content, previousPosition, i - previousPosition);
                     previousPosition = i + 1;
 
-                    byte[] bytes;
-                    if (c >= '\ud800' && c <= '\udfff' && previousPosition < length)
+                    // Special case for Edge/IE workaround for MarkdownEditor, don't escape non-ASCII chars to make image links working
+                    if (UseNonAsciiNoEscape)
                     {
-                        bytes = Encoding.UTF8.GetBytes(new[] { c, content[previousPosition] });
-                        // Skip next char as it is decoded above
-                        i++;
-                        previousPosition = i + 1;
+                        Write(c);
                     }
                     else
                     {
-                        bytes = Encoding.UTF8.GetBytes(new[] { c });
-                    }
-
-                    for (var j = 0; j < bytes.Length; j++)
-                    {
-                        Write($"%{bytes[j]:X2}");
+                        byte[] bytes;
+                        if (c >= '\ud800' && c <= '\udfff' && previousPosition < length)
+                        {
+                            bytes = Encoding.UTF8.GetBytes(new[] { c, content[previousPosition] });
+                            // Skip next char as it is decoded above
+                            i++;
+                            previousPosition = i + 1;
+                        }
+                        else
+                        {
+                            bytes = Encoding.UTF8.GetBytes(new[] { c });
+                        }
+                        for (var j = 0; j < bytes.Length; j++)
+                        {
+                            Write($"%{bytes[j]:X2}");
+                        }
                     }
                 }
             }

src/Markdig/project.json

@@ -1,6 +1,6 @@
 {
   "title": "Markdig",
-  "version": "0.7.5",
+  "version": "0.8.5",
   "authors": [ "Alexandre Mutel" ],
   "description": "A fast, powerfull, CommonMark compliant, extensible Markdown processor for .NET with 20+ builtin extensions (pipetables, footnotes, definition lists... etc.)",
   "copyright": "Alexandre Mutel",
@@ -11,7 +11,7 @@
     "projectUrl": "https://github.com/lunet-io/markdig",
     "iconUrl": "https://raw.githubusercontent.com/lunet-io/markdig/master/img/markdig.png",
     "requireLicenseAcceptance": false,
-    "releaseNotes": "> 0.7.5\n- several bug fixes (pipe tables, disable HTML, special attributes, inline maths, abbreviations...)\n- add support for rowspan in grid tables\n> 0.7.4\n- Fix bug with strong emphasis starting at the beginning of a line\n> 0.7.3\n- Fix threading issue with pipeline\n> 0.7.2\n- Fix rendering of table colspan with non english locale\n- Fix grid table colspan parsing\n- Add nofollow extension for links\n> 0.7.1\n- Fix issue in smarty pants which could lead to an InvalidCastException\n- Update parsers to latest CommonMark specs\n>0.7.0\n- Update to latest NETStandard.Library 1.6.0\n- Fix issue with digits in auto-identifier extension\n- Fix incorrect start of span calculated for code indented blocks",
+    "releaseNotes": "> 0.8.5\n- Allow to force table column alignment to left\n> 0.8.4\n- Fix issue when calculating the span of an indented code block within a list. Make sure to include first whitespace on the line\n> 0.8.3\n- fix NullReferenceException with Gridtables extension when a single `+` is entered on a line\n> 0.8.2\n- fix potential cast exception with Abreviation extension and empty literals\n> 0.8.1\n- new extension to disable URI escaping for non-US-ASCII characters to workaround a bug in Edge/IE\n- Fix an issue with abbreviations with left/right multiple non-punctuation/space characters\n> 0.8.0\n- Update to latest CommonMark specs\n- Fix empty literal\n- Add YAML frontmatter extension\n",
     "tags": [ "Markdown CommonMark md html md2html" ]
   },
   "configurations": {
@@ -26,7 +26,8 @@
         "define": [ "RELEASE", "TRACE" ],
         "optimize": true,
         "allowUnsafe": true,
-        "xmlDoc":  true 
+        "xmlDoc":  true,
+        "nowarn": [ "CS1591" ]
       }
     }
   },