Bump version to 0.1.1

readme.md

@@ -1,4 +1,4 @@
-# Markdig  [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)
+# Markdig [![Build status](https://ci.appveyor.com/api/projects/status/hk391x8jcskxt1u8?svg=true)](https://ci.appveyor.com/project/xoofx/markdig) [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)
 
 Markdig is a fast, powerfull, [CommonMark](http://commonmark.org/) compliant, extensible Markdown processor for .NET.
 
@@ -13,32 +13,33 @@ Markdig is a fast, powerfull, [CommonMark](http://commonmark.org/) compliant, ex
 - Includes all the core elements of CommonMark:
   - including GFM fenced code blocks. 
 - **Extensible** architecture
+  - Even the core Markdown/CommonMark parsing is pluggable, so it allows to disable builtin Markdown/Commonmark parsing (e.g [Disable HTML parsing](https://github.com/lunet-io/markdig/blob/7964bd0160d4c18e4155127a4c863d61ebd8944a/src/Markdig/MarkdownExtensions.cs#L306)) or change behaviour (e.g change matching `#` of a headers with `@`)   
 - Built-in with **18+ extensions**, including:
-  - **Abbreviations** (inspired from [PHP Markdown Extra - Abbreviations](https://michelf.ca/projects/php-markdown/extra/#abbr))
-  - **Auto-identifiers** for headings (similar to [Pandoc](http://pandoc.org/README.html#extension-auto_identifiers)
-  - **Bootstrap** class (to output bootstrap class)
-  - **Citation** text by enclosing `""...""` (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/referencing-creative-works-with-cite/892))
-  - **Custom containers** similar to fenced code block `:::` for generating a proper `<div>...</div>` instead (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/custom-container-for-block-and-inline/2051))
-  - **Definition lists** (inspired from [PHP Markdown Extra - Definitions Lists](https://michelf.ca/projects/php-markdown/extra/#def-list))
-  - **Emoji** support (inspired from [Markdown-it](https://markdown-it.github.io/))
+  - 2 kind of tables:
+    - **Pipe tables** (inspired from Github tables and [PanDoc](http://pandoc.org/README.html#pipe_tables))
+    - **Grid tables** (inspired from [Pandoc](http://pandoc.org/README.html#grid_tables)) 
   - **Extra emphasis** (inspired from [Pandoc](http://pandoc.org/README.html#strikeout) and [Markdown-it](https://markdown-it.github.io/)) 
     - strike through `~~`,
     - Subscript `~`
     - Superscript `^` 
     - Inserted `++`
     - Marked `==`
+  - **Special attributes** or attached HTML attributes (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#spe-attr))
+  - **Definition lists** (inspired from [PHP Markdown Extra - Definitions Lists](https://michelf.ca/projects/php-markdown/extra/#def-list))
+  - **Footnotes** (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes))
+  - **Auto-identifiers** for headings (similar to [Pandoc](http://pandoc.org/README.html#extension-auto_identifiers)
+  - **Extra bullet lists**, supporting alpha bullet `a.` `b.` and roman bullet (`i`, `ii`...etc.)
+  - **Media support** for media url (youtube, vimeo, mp4...etc.) (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/embedded-audio-and-video/441))
+  - **Abbreviations** (inspired from [PHP Markdown Extra - Abbreviations](https://michelf.ca/projects/php-markdown/extra/#abbr))
+  - **Citation** text by enclosing `""...""` (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/referencing-creative-works-with-cite/892))
+  - **Custom containers** similar to fenced code block `:::` for generating a proper `<div>...</div>` instead (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/custom-container-for-block-and-inline/2051))
   - **Figures** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/image-tag-should-expand-to-figure-when-used-with-title/265/5))
   - **Footers** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/syntax-for-footer/2070))
-  - **Footnotes** (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes))
-  - **Special attributes** or attached HTML attributes (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#spe-attr))
-  - **Soft lines as hard lines**
-  - **Extra bullet lists**, supporting alpha bullet `a.` `b.` and roman bullet (`i`, `ii`...etc.)
   - **Mathematics**/Latex extension by enclosing `$$` for block and `$` for inline math (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/mathematics-extension/457/31))
-  - **Embed player** for media url (youtube, vimeo, mp4...etc.) (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/embedded-audio-and-video/441))
+  - **Soft lines as hard lines**
+  - **Emoji** support (inspired from [Markdown-it](https://markdown-it.github.io/))
   - **SmartyPants** (inspired from [Daring Fireball - SmartyPants](https://daringfireball.net/projects/smartypants/))
-  - Tables:
-    - **Pipe tables** (inspired from Kramdown/[PanDoc](http://pandoc.org/README.html#pipe_tables))
-    - **Grid tables** (inspired from [Pandoc](http://pandoc.org/README.html#grid_tables)) 
+  - **Bootstrap** class (to output bootstrap class)
 - Compatible with .NET 3.5, 4.0+ and .NET Core (`netstandard1.1+`)
 	
 ## Download
@@ -47,40 +48,108 @@ Markdig is available as a NuGet package: [![NuGet](https://img.shields.io/nuget/
 
 ## Usage
 
-The main entry point for the API is the `Markdown` class:
+The main entry point for the API is the `Markdig.Markdown` class:
 
 By default, without any options, Markdig is using the plain CommonMark parser:
 
 ```csharp
-var result = Markdown.ToHtml("This is a text with some **emphasis**");
+var result = Markdown.ToHtml("This is a text with some *emphasis*");
 Console.WriteLine(result);   // prints: <p>This is a text with some <em>emphasis</em></p>
 ```
 
 In order to activate all extensions (except Emoji)
 
 ```csharp
-var result = Markdown.ToHtml("This is a text with some **emphasis**", new MarkdownPipeline().UseAllExtensions());
+// Configure the pipeline with all extensions active
+var pipeline = new MarkdownPipeline().UseAllExtensions();
+var result = Markdown.ToHtml("This is a text with some *emphasis*", pipeline);
 ```
 
-You can have a look at the [MarkdownPipeline](https://github.com/lunet-io/markdig/blob/src/Markdig/MarkdownPipeline.cs) file that describes all actionable extensions.
+You can have a look at the [MarkdownExtensions](https://github.com/lunet-io/markdig/blob/master/src/Markdig/MarkdownExtensions.cs) that describes all actionable extensions (by modifying the MarkdownPipeline)
 
 ## License
 
-This software is released under the [BSD-Clause 2 license](http://opensource.org/licenses/BSD-2-Clause).
+This software is released under the [BSD-Clause 2 license](https://github.com/lunet-io/markdig/blob/master/license.txt).
+
+
+## Benchmarking
+
+This is an early preview of the benchmarking against various implementations:
+
+- Markdig: itself
+- CommonMarkCpp: [cmark](https://github.com/jgm/cmark), Reference C implementation of CommonMark, no support for extensions
+- [CommonMark.NET](https://github.com/Knagis/CommonMark.NET): CommonMark implementation for .NET, no support for extensions, port of cmark
+  - [CommonMarkNet (devel)](https://github.com/AMDL/CommonMark.NET/tree/pipe-tables): An evolution of CommonMark.NET, supports extensions, not released yet
+- [MarkdownDeep](https://github.com/toptensoftware/markdowndeep) another .NET implementation
+- [MarkdownSharp](https://github.com/Kiri-rin/markdownsharp): Open source C# implementation of Markdown processor, as featured on Stack Overflow, regexp based.
+- [Moonshine](https://github.com/brandonc/moonshine): popular C Markdown processor
+
+Markdig is roughly x100 times faster than MarkdownSharp and extremelly competitive to other implems (that are not feature wise comparable) 
+
+Performance in x86:
+
+```
+// * Summary *
+
+BenchmarkDotNet-Dev=v0.9.6.0+
+OS=Microsoft Windows NT 6.2.9200.0
+Processor=Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, ProcessorCount=8
+Frequency=3319351 ticks, Resolution=301.2637 ns, Timer=TSC
+HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
+JitModules=clrjit-v4.6.1080.0
+
+Type=Program  Mode=SingleRun  LaunchCount=2
+WarmupCount=2  TargetCount=10
+
+               Method |    Median |    StdDev |  Gen 0 | Gen 1 |  Gen 2 | Bytes Allocated/Op |
+--------------------- |---------- |---------- |------- |------ |------- |------------------- |
+          TestMarkdig | 5.4870 ms | 0.0158 ms | 193.00 | 12.00 |  84.00 |       1,425,192.72 |
+    TestCommonMarkCpp | 4.0134 ms | 0.1008 ms |      - |     - | 180.00 |         454,859.74 |
+    TestCommonMarkNet | 4.6139 ms | 0.0581 ms | 193.00 | 12.00 |  84.00 |       1,406,367.27 |
+ TestCommonMarkNetNew | 5.5327 ms | 0.0461 ms | 193.00 | 96.00 |  84.00 |       1,738,465.42 |
+     TestMarkdownDeep | 7.5910 ms | 0.1006 ms | 205.00 | 96.00 |  84.00 |       1,758,383.79 |
+        TestMoonshine | 5.8843 ms | 0.1758 ms |      - |     - | 215.00 |         565,000.73 |
+
+// * Diagnostic Output - MemoryDiagnoser *
+
+
+// ***** BenchmarkRunner: End *****
+```
+
+Performance for x64:
+
+```
+// * Summary *
+
+BenchmarkDotNet-Dev=v0.9.6.0+
+OS=Microsoft Windows NT 6.2.9200.0
+Processor=Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, ProcessorCount=8
+Frequency=3319351 ticks, Resolution=301.2637 ns, Timer=TSC
+HostCLR=MS.NET 4.0.30319.42000, Arch=64-bit RELEASE [RyuJIT]
+JitModules=clrjit-v4.6.1080.0
+
+Type=Program  Mode=SingleRun  LaunchCount=2
+WarmupCount=2  TargetCount=10
+
+               Method |    Median |    StdDev |  Gen 0 |  Gen 1 | Gen 2 | Bytes Allocated/Op |
+--------------------- |---------- |---------- |------- |------- |------ |------------------- |
+          TestMarkdig | 5.9539 ms | 0.0495 ms | 157.00 |  96.00 | 84.00 |       1,767,834.52 |
+    TestCommonMarkNet | 4.3158 ms | 0.0161 ms | 157.00 |  96.00 | 84.00 |       1,747,432.06 |
+ TestCommonMarkNetNew | 5.3421 ms | 0.0435 ms | 229.00 | 168.00 | 84.00 |       2,323,922.97 |
+     TestMarkdownDeep | 7.4750 ms | 0.0281 ms | 318.00 | 186.00 | 84.00 |       2,576,728.69 |
+
+// * Diagnostic Output - MemoryDiagnoser *
+
+
+// ***** BenchmarkRunner: End *****
+```
+
+## Credits
+
+Thanks to the fantastic work done by [John Mac Farlane](http://johnmacfarlane.net/) for the CommonMark specs and all the people involved in making Markdown a better standard!
+
+This project would not have been possible without this huge foundation.
 
 ## Author
 
 Alexandre MUTEL aka [xoofx](http://xoofx.com)
-
-
-
-
-
-
-
-
-
-
-
-
-

src/Markdig.Tests/TestPlayParser.cs

@@ -27,34 +27,47 @@ Later in a text we are using HTML and it becomes an abbr tag HTML
             Console.WriteLine(result);
         }
 
-// Test for emoji and smileys
-//        var text = @" This is a test with a :) and a :angry: smiley";
+        [Test]
+        public void TestSamePipelineAllExtensions()
+        {
+            var pipeline = new MarkdownPipeline().UseAllExtensions();
+
+            // Reuse the same pipeline 
+            var result1 = Markdown.ToHtml("This is a \"\"citation\"\"", pipeline);
+            var result2 = Markdown.ToHtml("This is a \"\"citation\"\"", pipeline);
+
+            Assert.AreEqual("<p>This is a <cite>citation</cite></p>", result1.Trim());
+            Assert.AreEqual(result1, result2);
+        }
+
+        // Test for emoji and smileys
+        //        var text = @" This is a test with a :) and a :angry: smiley";
 
 
-// Test for definition lists:
-//
-//        var text = @"
-//Term 1
-//:   This is a definition item
-//    With a paragraph
-//    > This is a block quote
+        // Test for definition lists:
+        //
+        //        var text = @"
+        //Term 1
+        //:   This is a definition item
+        //    With a paragraph
+        //    > This is a block quote
 
-//    - This is a list
-//    - item2
+        //    - This is a list
+        //    - item2
 
-//    ```java
-//    Test
+        //    ```java
+        //    Test
 
 
-//    ```
+        //    ```
 
-//    And a lazy line
-//:   This ia another definition item
+        //    And a lazy line
+        //:   This ia another definition item
 
-//Term2
-//Term3 *with some inline*
-//:   This is another definition for term2
-//";
+        //Term2
+        //Term3 *with some inline*
+        //:   This is another definition for term2
+        //";
 
 
         // Test for grid table

src/Markdig/Extensions/Cites/CiteExtension.cs

@@ -18,15 +18,8 @@ namespace Markdig.Extensions.Cites
         public void Setup(MarkdownPipeline pipeline)
         {
             var parser = pipeline.InlineParsers.FindExact<EmphasisInlineParser>();
-            if (parser != null)
+            if (parser != null && !parser.HasEmphasisChar('"'))
             {
-                foreach (var emphasis in parser.EmphasisDescriptors)
-                {
-                    if (emphasis.Character == '"')
-                    {
-                        return;
-                    }
-                }
                 parser.EmphasisDescriptors.Add(new EmphasisDescriptor('"', 2, 2, false));
             }
 
@@ -37,6 +30,7 @@ namespace Markdig.Extensions.Cites
                 var emphasisRenderer = htmlRenderer.ObjectRenderers.FindExact<EmphasisInlineRenderer>();
                 if (emphasisRenderer != null)
                 {
+                    // TODO: Use an ordered list instead as we don't know if this specific GetTag has been already added
                     var previousTag = emphasisRenderer.GetTag;
                     emphasisRenderer.GetTag = inline => GetTag(inline) ?? previousTag(inline);
                 }

src/Markdig/Extensions/CustomContainers/CustomContainerExtension.cs

@@ -23,7 +23,7 @@ namespace Markdig.Extensions.CustomContainers
 
             // Plug the inline parser for CustomContainerInline
             var inlineParser = pipeline.InlineParsers.Find<EmphasisInlineParser>();
-            if (inlineParser != null)
+            if (inlineParser != null && !inlineParser.HasEmphasisChar(':'))
             {
                 inlineParser.EmphasisDescriptors.Add(new EmphasisDescriptor(':', 2, 2, true));
                 var previousCreateEmphasisInline = inlineParser.CreateEmphasisInline;

src/Markdig/MarkdownExtensions.cs

@@ -19,6 +19,8 @@ using Markdig.Extensions.Mathematics;
 using Markdig.Extensions.Medias;
 using Markdig.Extensions.SmartyPants;
 using Markdig.Extensions.Tables;
+using Markdig.Parsers;
+using Markdig.Parsers.Inlines;
 
 namespace Markdig
 {
@@ -295,5 +297,26 @@ namespace Markdig
             pipeline.Extensions.AddIfNotAlready<EmojiExtension>();
             return pipeline;
         }
+
+        /// <summary>
+        /// This will disable the HTML support in the markdown processor (for constraint/safe parsing).
+        /// </summary>
+        /// <param name="pipeline">The pipeline.</param>
+        /// <returns>The modified pipeline</returns>
+        public static MarkdownPipeline DisableHtml(this MarkdownPipeline pipeline)
+        {
+            var parser = pipeline.BlockParsers.Find<HtmlBlockParser>();
+            if (parser != null)
+            {
+                pipeline.BlockParsers.Remove(parser);
+            }
+
+            var inlineParser = pipeline.InlineParsers.Find<AutolineInlineParser>();
+            if (inlineParser != null)
+            {
+                inlineParser.EnableHtmlParsing = false;
+            }
+            return pipeline;
+        }
     }
 }

src/Markdig/MarkdownPipeline.cs

@@ -13,6 +13,7 @@ namespace Markdig
     /// <summary>
     /// This class allows to modify the pipeline to parse and render a Markdown document.
     /// </summary>
+    /// <remarks>NOTE: A pipeline is not thread-safe.</remarks>
     public class MarkdownPipeline
     {
         /// <summary>
@@ -95,6 +96,11 @@ namespace Markdig
         /// <exception cref="System.InvalidOperationException">An extension cannot be null</exception>
         public void Initialize()
         {
+            // TODO: Review the whole initialization process for extensions
+            // - It does not prevent a user to modify the pipeline after it has been used
+            // - a pipeline is not thread safe.
+            // We should find a proper way to make the pipeline safely modifiable/freezable (PipelineBuilder -> Pipeline)
+
             // Allow extensions to modify existing BlockParsers, InlineParsers and Renderer
             foreach (var extension in Extensions)
             {

src/Markdig/Parsers/Inlines/AutolineInlineParser.cs

@@ -18,8 +18,14 @@ namespace Markdig.Parsers.Inlines
         public AutolineInlineParser()
         {
             OpeningCharacters = new[] {'<'};
+            EnableHtmlParsing = true;
         }
 
+        /// <summary>
+        /// Gets or sets a value indicating whether to enable HTML parsing. Default is <c>true</c>
+        /// </summary>
+        public bool EnableHtmlParsing { get; set; }
+
         public override bool Match(InlineProcessor processor, ref StringSlice slice)
         {
             string link;
@@ -29,7 +35,7 @@ namespace Markdig.Parsers.Inlines
             {
                 processor.Inline = new AutolinkInline() {IsEmail = isEmail, Url = link};
             }
-            else
+            else if (EnableHtmlParsing)
             {
                 slice = saved;
                 string htmlTag;

src/Markdig/Parsers/Inlines/EmphasisInlineParser.cs

@@ -41,6 +41,23 @@ namespace Markdig.Parsers.Inlines
         /// </summary>
         public List<EmphasisDescriptor> EmphasisDescriptors { get; }
 
+        /// <summary>
+        /// Determines whether this parser is using the specified character as an emphasis delimiter.
+        /// </summary>
+        /// <param name="c">The character to look for.</param>
+        /// <returns><c>true</c> if this parser is using the specified character as an emphasis delimiter; otherwise <c>false</c></returns>
+        public bool HasEmphasisChar(char c)
+        {
+            foreach (var emphasis in EmphasisDescriptors)
+            {
+                if (emphasis.Character == c)
+                {
+                    return true;
+                }
+            }
+            return false;
+        }
+
         /// <summary>
         /// Gets or sets the create emphasis inline delegate (allowing to create a different emphasis inline class)
         /// </summary>

src/Markdig/Properties/AssemblyInfo.cs

@@ -1,7 +1,5 @@
 using System.Resources;
 using System.Reflection;
-using System.Runtime.CompilerServices;
-using System.Runtime.InteropServices;
 
 // General Information about an assembly is controlled through the following 
 // set of attributes. Change these attribute values to modify the information
@@ -20,8 +18,5 @@ using System.Runtime.InteropServices;
 [assembly: AssemblyCulture("")]
 [assembly: NeutralResourcesLanguage("en")]
 
-[assembly: AssemblyVersion("0.1.0.0")]
-[assembly: AssemblyFileVersion("0.1.0.0")]
-
-[assembly: InternalsVisibleTo("Scriban.Tests")]
-
+[assembly: AssemblyVersion("0.1.1.0")]
+[assembly: AssemblyFileVersion("0.1.1.0")]

src/Markdig/project.json

@@ -1,6 +1,6 @@
 {
   "title": "Markdig",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "authors": [ "Alexandre Mutel" ],
   "description": "A fast, powerfull, CommonMark compliant, extensible Markdown processor for .NET",
   "copyright": "Alexandre Mutel",
@@ -10,7 +10,7 @@
     "licenseUrl": "https://github.com/lunet-io/markdig/blob/master/license.txt",
     "projectUrl": "https://github.com/lunet-io/markdig",
     "requireLicenseAcceptance": false,
-    "tags": [ "Markdown CommonMark md html" ]
+    "tags": [ "Markdown CommonMark md html md2html" ]
   },
   "configurations": {
     "Debug": {
@@ -28,9 +28,6 @@
       }
     }
   },
-  "dependencies": {
-    "NETStandard.Library": "1.5.0-rc2-24027"
-  },
   "frameworks": {
     "net35": {
       "buildOptions": {
@@ -61,6 +58,7 @@
     },
     "netstandard1.1": {
       "dependencies": {
+        "NETStandard.Library": "1.5.0-rc2-24027",
         "System.Threading": "4.0.11-rc2-24027",
         "System.Threading.Tasks": "4.0.11-rc2-24027",
         "System.Threading.Tasks.Parallel": "4.0.1-rc2-24027"