Compare commits

...

20 Commits

Author SHA1 Message Date
Alexandre Mutel
5844ccc395 Bump to 0.5.4 2016-06-20 06:58:55 +09:00
Alexandre Mutel
be9c6fa54b Fix bug for html block parsing in StringSlice.Search (issue #12) 2016-06-20 06:56:13 +09:00
Alexandre Mutel
d14f277c7b Bump version to 0.5.3 2016-06-19 11:37:26 +09:00
Alexandre Mutel
593bf08b92 Fix bug in pipetables with a trailing | on the header row separator (issue #10) 2016-06-19 11:37:12 +09:00
Alexandre Mutel
85f631f868 Bump version to 0.5.2 2016-06-17 13:58:31 +09:00
Alexandre Mutel
5ad964bcb6 Rename MarkdownObject.SourceSpan to Span 2016-06-17 13:56:09 +09:00
Alexandre Mutel
137a404bdc Add property handling of precise position for links 2016-06-17 13:45:43 +09:00
Alexandre Mutel
5204ec758a Add version used for benchmarks. Add docfx Microsoft.DocAsCode.MarkdownLite for comparison 2016-06-17 13:15:32 +09:00
Alexandre Mutel
6f4fb69c62 Add precise source location for title, span for inlines/footnote/abbreviations (#8) 2016-06-17 09:34:46 +09:00
Alexandre Mutel
0a1b37c965 Breaking change: Add SourceSpan and replace SourceStartPosition and SourceEndPosition 2016-06-16 23:45:18 +09:00
Alexandre Mutel
90bdafb05a Update link to babelmark3 2016-06-16 13:35:45 +09:00
Alexandre Mutel
8cc668ae6d Update link to babelmark3 https 2016-06-16 13:22:23 +09:00
Alexandre Mutel
1787dc4590 Update readme.md with link to article 2016-06-16 12:43:32 +09:00
Alexandre Mutel
0a9cc8fcd7 Fix typo 2016-06-16 12:31:31 +09:00
Alexandre Mutel
10c06daf5d Make benchmark results shorter 2016-06-16 12:26:28 +09:00
Alexandre Mutel
a262e42980 Update readme and benchmarks 2016-06-16 12:24:01 +09:00
Alexandre Mutel
4cd3d045d1 Add tests precise position with text containing tabs 2016-06-16 10:03:04 +09:00
Alexandre Mutel
92357576b1 Add test precise position for indented code 2016-06-16 09:54:07 +09:00
Alexandre Mutel
d45f67f8c2 Add protection against infinite loops if an extension was messing the parsing of blocks or inlines 2016-06-16 09:22:45 +09:00
Alexandre Mutel
2571cdffee Add tasklists to readme.md 2016-06-16 06:43:19 +09:00
51 changed files with 922 additions and 258 deletions

BIN
img/BenchmarkCPU.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.1 KiB

BIN
img/BenchmarkMemory.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.6 KiB

View File

@@ -6,7 +6,7 @@ Markdig is a fast, powerfull, [CommonMark](http://commonmark.org/) compliant, ex
> NOTE: The repository is under construction. There will be a dedicated website and proper documentation at some point!
You can **try Markdig online** and compare it to other implementations on [babelmark3](http://babelmark.github.io/)
You can **try Markdig online** and compare it to other implementations on [babelmark3](https://babelmark.github.io/?text=Hello+**Markdig**!)
## Features
@@ -18,7 +18,7 @@ You can **try Markdig online** and compare it to other implementations on [babel
- including GFM fenced code blocks.
- **Extensible** architecture
- Even the core Markdown/CommonMark parsing is pluggable, so it allows to disable builtin Markdown/Commonmark parsing (e.g [Disable HTML parsing](https://github.com/lunet-io/markdig/blob/7964bd0160d4c18e4155127a4c863d61ebd8944a/src/Markdig/MarkdownExtensions.cs#L306)) or change behaviour (e.g change matching `#` of a headers with `@`)
- Built-in with **18+ extensions**, including:
- Built-in with **20+ extensions**, including:
- 2 kind of tables:
- **Pipe tables** (inspired from Github tables and [PanDoc - Pipe Tables](http://pandoc.org/README.html#extension-pipe_tables))
- **Grid tables** (inspired from [Pandoc - Grid Tables](http://pandoc.org/README.html#extension-grid_tables))
@@ -32,6 +32,7 @@ You can **try Markdig online** and compare it to other implementations on [babel
- **Definition lists** (inspired from [PHP Markdown Extra - Definitions Lists](https://michelf.ca/projects/php-markdown/extra/#def-list))
- **Footnotes** (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes))
- **Auto-identifiers** for headings (similar to [Pandoc - Auto Identifiers](http://pandoc.org/README.html#extension-auto_identifiers))
- **Task Lists** inspired from [Github Task lists](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments).
- **Extra bullet lists**, supporting alpha bullet `a.` `b.` and roman bullet (`i`, `ii`...etc.)
- **Media support** for media url (youtube, vimeo, mp4...etc.) (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/embedded-audio-and-video/441))
- **Abbreviations** (inspired from [PHP Markdown Extra - Abbreviations](https://michelf.ca/projects/php-markdown/extra/#abbr))
@@ -46,6 +47,13 @@ You can **try Markdig online** and compare it to other implementations on [babel
- **Bootstrap** class (to output bootstrap class)
- Compatible with .NET 3.5, 4.0+ and .NET Core (`netstandard1.1+`)
## Documentation
> The repository is under construction. There will be a dedicated website and proper documentation at some point!
In the meantime, you can have a "behind the scene" article about Markdig in my blog post ["Implementing a Markdown Engine for .NET"](http://xoofx.com/blog/2016/06/13/implementing-a-markdown-processor-for-dotnet/)
## Download
Markdig is available as a NuGet package: [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)
@@ -80,27 +88,44 @@ This software is released under the [BSD-Clause 2 license](https://github.com/lu
This is an early preview of the benchmarking against various implementations:
- Markdig: itself
- CommonMarkCpp: [cmark](https://github.com/jgm/cmark), Reference C implementation of CommonMark, no support for extensions
- [CommonMark.NET](https://github.com/Knagis/CommonMark.NET): CommonMark implementation for .NET, no support for extensions, port of cmark
- [CommonMarkNet (devel)](https://github.com/AMDL/CommonMark.NET/tree/pipe-tables): An evolution of CommonMark.NET, supports extensions, not released yet
- [MarkdownDeep](https://github.com/toptensoftware/markdowndeep) another .NET implementation
- [MarkdownSharp](https://github.com/Kiri-rin/markdownsharp): Open source C# implementation of Markdown processor, as featured on Stack Overflow, regexp based.
- [Moonshine](https://github.com/brandonc/moonshine): popular C Markdown processor
**C implementations**:
- [cmark](https://github.com/jgm/cmark) (version: 0.25.0): Reference C implementation of CommonMark, no support for extensions
- [Moonshine](https://github.com/brandonc/moonshine) (version: : popular C Markdown processor
**.NET implementations**:
- [Markdig](https://github.com/lunet-io/markdig) (version: 0.5.x): itself
- [CommonMark.NET(master)](https://github.com/Knagis/CommonMark.NET) (version: 0.11.0): CommonMark implementation for .NET, no support for extensions, port of cmark
- [CommonMark.NET(pipe_tables)](https://github.com/AMDL/CommonMark.NET/tree/pipe-tables): An evolution of CommonMark.NET, supports extensions, not released yet
- [MarkdownDeep](https://github.com/toptensoftware/markdowndeep) (version: 1.5.0): another .NET implementation
- [MarkdownSharp](https://github.com/Kiri-rin/markdownsharp) (version: 1.13.0): Open source C# implementation of Markdown processor, as featured on Stack Overflow, regexp based.
- [Marked.NET](https://github.com/T-Alex/MarkedNet) (version: 1.0.5) port of original [marked.js](https://github.com/chjj/marked) project
- [Microsoft.DocAsCode.MarkdownLite](https://github.com/dotnet/docfx/tree/dev/src/Microsoft.DocAsCode.MarkdownLite) (version: 2.0.1) used by the [docfx](https://github.com/dotnet/docfx) project
**JavaScript/V8 implementations**:
- [Strike.V8](https://github.com/SimonCropp/Strike) (version: 1.5.0) [marked.js](https://github.com/chjj/marked) running in Google V8 (not .NET based)
### Analysis of the results:
- Markdig is roughly **x100 times faster than MarkdownSharp**
- Markdig is roughly **x100 times faster than MarkdownSharp**, **30x times faster than docfx**
- **Among the best in CPU**, Extremelly competitive and often faster than other implementations (not feature wise equivalent)
- **15% to 30% less allocations** and GC pressure
Because Marked.NET, MarkdownSharp and DocAsCode.MarkdownLite are way too slow, they are not included in the following charts:
### Performance in x86:
![BenchMark CPU Time](img/BenchmarkCPU.png)
![BenchMark Memory](img/BenchmarkMemory.png)
### Performance for x86:
```
BenchmarkDotNet-Dev=v0.9.6.0+
BenchmarkDotNet-Dev=v0.9.7.0+
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, ProcessorCount=8
Processor=Intel(R) Core(TM) i7-4770 CPU 3.40GHz, ProcessorCount=8
Frequency=3319351 ticks, Resolution=301.2637 ns, Timer=TSC
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
JitModules=clrjit-v4.6.1080.0
@@ -108,14 +133,18 @@ JitModules=clrjit-v4.6.1080.0
Type=Program Mode=SingleRun LaunchCount=2
WarmupCount=2 TargetCount=10
Method | Median | StdDev | Gen 0 | Gen 1 | Gen 2 | Bytes Allocated/Op |
--------------------- |---------- |---------- |------- |------ |------- |------------------- |
TestMarkdig | 5.4332 ms | 0.0809 ms | 96.00 | 36.00 | 84.00 | 1,218,643.59 |
TestCommonMarkCpp | 4.0150 ms | 0.1208 ms | - | - | 180.00 | 454,859.74 |
TestCommonMarkNet | 4.5448 ms | 0.0323 ms | 193.00 | 12.00 | 84.00 | 1,406,367.27 |
TestCommonMarkNetNew | 5.4811 ms | 0.0327 ms | 193.00 | 96.00 | 84.00 | 1,738,465.42 |
TestMarkdownDeep | 7.5881 ms | 0.0554 ms | 205.00 | 96.00 | 84.00 | 1,758,383.79 |
TestMoonshine | 5.8586 ms | 0.0764 ms | - | - | 215.00 | 565,000.72 |
Method | Median | StdDev |Scaled | Gen 0 | Gen 1| Gen 2|Bytes Allocated/Op |
--------------------------- |------------ |---------- |------ | ------ |------|---------|------------------ |
Markdig | 5.5316 ms | 0.0372 ms | 0.71 | 56.00| 21.00| 49.00| 1,285,917.31 |
CommonMark.NET(master) | 4.7035 ms | 0.0422 ms | 0.60 | 113.00| 7.00| 49.00| 1,502,404.60 |
CommonMark.NET(pipe_tables) | 5.6164 ms | 0.0298 ms | 0.72 | 111.00| 56.00| 49.00| 1,863,128.13 |
MarkdownDeep | 7.8193 ms | 0.0334 ms | 1.00 | 120.00| 56.00| 49.00| 1,884,854.85 |
cmark | 4.2698 ms | 0.1526 ms | 0.55 | -| -| -| NA |
Moonshine | 6.0929 ms | 0.1053 ms | 1.28 | -| -| -| NA |
Strike.V8 | 10.5895 ms | 0.0492 ms | 1.35 | -| -| -| NA |
Marked.NET | 207.3169 ms | 5.2628 ms | 26.51 | 0.00| 0.00| 0.00| 303,125,228.65 |
MarkdownSharp | 675.0185 ms | 2.8447 ms | 86.32 | 40.00| 27.00| 41.00| 2,413,394.17 |
Microsoft DocfxMarkdownLite | 166.3357 ms | 0.4529 ms | 21.27 |4,452.00|948.00|11,167.00| 180,218,359.60 |
```
### Performance for x64:

View File

@@ -214,6 +214,76 @@ Column delimiters `|` at the very beginning of a line or just before a line endi
</tbody>
</table>
````````````````````````````````
A pipe may be present at both the beginning/ending of each line:
```````````````````````````````` example
|a|b|
|-|-|
|0|1|
.
<table>
<thead>
<tr>
<th>a</th>
<th>b</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
</tr>
</tbody>
</table>
````````````````````````````````
Or may be ommitted on one side:
```````````````````````````````` example
a|b|
-|-|
0|1|
.
<table>
<thead>
<tr>
<th>a</th>
<th>b</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
</tr>
</tbody>
</table>
````````````````````````````````
```````````````````````````````` example
|a|b
|-|-
|0|1
.
<table>
<thead>
<tr>
<th>a</th>
<th>b</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
</tr>
</tbody>
</table>
````````````````````````````````
Single column table can be declared with lines starting only by a column delimiter:
```````````````````````````````` example

View File

@@ -16631,7 +16631,7 @@ namespace Markdig.Tests
TestParser.TestSpec(" a | b |\n-- | --\n| 0 | 1\n| 2 | 3 |\n 4 | 5 ", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n<tr>\n<td>2</td>\n<td>3</td>\n</tr>\n<tr>\n<td>4</td>\n<td>5</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
// Single column table can be declared with lines starting only by a column delimiter:
// A pipe may be present at both the beginning/ending of each line:
[TestFixture]
public partial class TestExtensionsPipeTable
{
@@ -16642,6 +16642,110 @@ namespace Markdig.Tests
// Section: Extensions Pipe Table
//
// The following CommonMark:
// |a|b|
// |-|-|
// |0|1|
//
// Should be rendered as:
// <table>
// <thead>
// <tr>
// <th>a</th>
// <th>b</th>
// </tr>
// </thead>
// <tbody>
// <tr>
// <td>0</td>
// <td>1</td>
// </tr>
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 10, "Extensions Pipe Table");
TestParser.TestSpec("|a|b|\n|-|-|\n|0|1|", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
// Or may be ommitted on one side:
[TestFixture]
public partial class TestExtensionsPipeTable
{
[Test]
public void Example011()
{
// Example 11
// Section: Extensions Pipe Table
//
// The following CommonMark:
// a|b|
// -|-|
// 0|1|
//
// Should be rendered as:
// <table>
// <thead>
// <tr>
// <th>a</th>
// <th>b</th>
// </tr>
// </thead>
// <tbody>
// <tr>
// <td>0</td>
// <td>1</td>
// </tr>
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 11, "Extensions Pipe Table");
TestParser.TestSpec("a|b|\n-|-|\n0|1|", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
[TestFixture]
public partial class TestExtensionsPipeTable
{
[Test]
public void Example012()
{
// Example 12
// Section: Extensions Pipe Table
//
// The following CommonMark:
// |a|b
// |-|-
// |0|1
//
// Should be rendered as:
// <table>
// <thead>
// <tr>
// <th>a</th>
// <th>b</th>
// </tr>
// </thead>
// <tbody>
// <tr>
// <td>0</td>
// <td>1</td>
// </tr>
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 12, "Extensions Pipe Table");
TestParser.TestSpec("|a|b\n|-|-\n|0|1", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
// Single column table can be declared with lines starting only by a column delimiter:
[TestFixture]
public partial class TestExtensionsPipeTable
{
[Test]
public void Example013()
{
// Example 13
// Section: Extensions Pipe Table
//
// The following CommonMark:
// | a
// | --
// | b
@@ -16664,7 +16768,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 10, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 13, "Extensions Pipe Table");
TestParser.TestSpec("| a\n| --\n| b\n| c ", "<table>\n<thead>\n<tr>\n<th>a</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>b</td>\n</tr>\n<tr>\n<td>c</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
@@ -16681,9 +16785,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example011()
public void Example014()
{
// Example 11
// Example 14
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16712,7 +16816,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 11, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 14, "Extensions Pipe Table");
TestParser.TestSpec(" a | b \n-------|-------\n 0 | 1 \n 2 | 3 ", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n<tr>\n<td>2</td>\n<td>3</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
@@ -16722,9 +16826,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example012()
public void Example015()
{
// Example 12
// Example 15
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16756,7 +16860,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 12, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 15, "Extensions Pipe Table");
TestParser.TestSpec(" a | b | c \n:------|:-------:| ----:\n 0 | 1 | 2 \n 3 | 4 | 5 ", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th style=\"text-align: center;\">b</th>\n<th style=\"text-align: right;\">c</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td style=\"text-align: center;\">1</td>\n<td style=\"text-align: right;\">2</td>\n</tr>\n<tr>\n<td>3</td>\n<td style=\"text-align: center;\">4</td>\n<td style=\"text-align: right;\">5</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
@@ -16765,9 +16869,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example013()
public void Example016()
{
// Example 13
// Example 16
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16782,7 +16886,7 @@ namespace Markdig.Tests
// 0 | 1
// 2 | 3</p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 13, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 16, "Extensions Pipe Table");
TestParser.TestSpec(" a | b\n-------|---x---\n 0 | 1\n 2 | 3 ", "<p>a | b\n-------|---x---\n0 | 1\n2 | 3</p> ", "pipetables");
}
}
@@ -16793,9 +16897,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example014()
public void Example017()
{
// Example 14
// Example 17
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16824,7 +16928,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 14, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 17, "Extensions Pipe Table");
TestParser.TestSpec(" *a* | b\n----- |-----\n 0 | _1_\n _2 | 3* ", "<table>\n<thead>\n<tr>\n<th><em>a</em></th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td><em>1</em></td>\n</tr>\n<tr>\n<td>_2</td>\n<td>3*</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
@@ -16835,9 +16939,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example015()
public void Example018()
{
// Example 15
// Example 18
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16847,7 +16951,7 @@ namespace Markdig.Tests
// Should be rendered as:
// <p>a | b <code>0 |</code></p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 15, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 18, "Extensions Pipe Table");
TestParser.TestSpec("a | b `\n0 | ` ", "<p>a | b <code>0 |</code></p> ", "pipetables");
}
}
@@ -16858,9 +16962,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example016()
public void Example019()
{
// Example 16
// Example 19
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16884,7 +16988,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 16, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 19, "Extensions Pipe Table");
TestParser.TestSpec("a <a href=\"\" title=\"|\"></a> | b\n-- | --\n0 | 1", "<table>\n<thead>\n<tr>\n<th>a <a href=\"\" title=\"|\"></a></th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}
@@ -16895,9 +16999,9 @@ namespace Markdig.Tests
public partial class TestExtensionsPipeTable
{
[Test]
public void Example017()
public void Example020()
{
// Example 17
// Example 20
// Section: Extensions Pipe Table
//
// The following CommonMark:
@@ -16921,7 +17025,7 @@ namespace Markdig.Tests
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 17, "Extensions Pipe Table");
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 20, "Extensions Pipe Table");
TestParser.TestSpec("a | b\n-- | --\n[This is a link with a | inside the label](http://google.com) | 1", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><a href=\"http://google.com\">This is a link with a | inside the label</a></td>\n<td>1</td>\n</tr>\n</tbody>\n</table>", "pipetables");
}
}

View File

@@ -80,36 +80,52 @@ namespace Markdig.Tests
[Test]
public void TestUrlAndTitle()
{
// 0 1 2 3
// 0123456789012345678901234567890123456789
var text = new StringSlice(@"(http://google.com 'this is a title')ABC");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual("http://google.com", link);
Assert.AreEqual("this is a title", title);
Assert.AreEqual(new SourceSpan(1, 17), linkSpan);
Assert.AreEqual(new SourceSpan(19, 35), titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
[Test]
public void TestUrlAndTitleEmpty()
{
// 01234
var text = new StringSlice(@"(<>)A");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual(string.Empty, link);
Assert.AreEqual(string.Empty, title);
Assert.AreEqual(new SourceSpan(1, 2), linkSpan);
Assert.AreEqual(SourceSpan.Empty, titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
[Test]
public void TestUrlAndTitleEmpty2()
{
// 012345
var text = new StringSlice(@"( <> )A");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual(string.Empty, link);
Assert.AreEqual(string.Empty, title);
Assert.AreEqual(new SourceSpan(2, 3), linkSpan);
Assert.AreEqual(SourceSpan.Empty, titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
@@ -117,12 +133,18 @@ namespace Markdig.Tests
[Test]
public void TestUrlEmptyWithTitleWithMultipleSpaces()
{
// 0 1 2
// 0123456789012345678901234567
var text = new StringSlice(@"( <> 'toto' )A");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual(string.Empty, link);
Assert.AreEqual("toto", title);
Assert.AreEqual(new SourceSpan(4, 5), linkSpan);
Assert.AreEqual(new SourceSpan(12, 17), titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
@@ -132,50 +154,67 @@ namespace Markdig.Tests
var text = new StringSlice(@"()A");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual(string.Empty, link);
Assert.AreEqual(string.Empty, title);
Assert.AreEqual(SourceSpan.Empty, linkSpan);
Assert.AreEqual(SourceSpan.Empty, titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
[Test]
public void TestMultipleLines()
{
var text = new StringSlice(@"(
<http://google.com>
'toto' )A");
// 0 1 2 3
// 01 2345678901234567890 1234567890123456789
var text = new StringSlice("(\n<http://google.com>\n 'toto' )A");
string link;
string title;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title));
SourceSpan linkSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan));
Assert.AreEqual("http://google.com", link);
Assert.AreEqual("toto", title);
Assert.AreEqual(new SourceSpan(2, 20), linkSpan);
Assert.AreEqual(new SourceSpan(26, 31), titleSpan);
Assert.AreEqual('A', text.CurrentChar);
}
[Test]
public void TestLabelSimple()
{
// 01234
var text = new StringSlice("[foo]");
string label;
Assert.True(LinkHelper.TryParseLabel(ref text, out label));
SourceSpan labelSpan;
Assert.True(LinkHelper.TryParseLabel(ref text, out label, out labelSpan));
Assert.AreEqual(new SourceSpan(1, 3), labelSpan);
Assert.AreEqual("foo", label);
}
[Test]
public void TestLabelEscape()
{
// 012345678
var text = new StringSlice(@"[fo\[\]o]");
string label;
Assert.True(LinkHelper.TryParseLabel(ref text, out label));
SourceSpan labelSpan;
Assert.True(LinkHelper.TryParseLabel(ref text, out label, out labelSpan));
Assert.AreEqual(new SourceSpan(1, 7), labelSpan);
Assert.AreEqual(@"fo[]o", label);
}
[Test]
public void TestLabelEscape2()
{
// 0123
var text = new StringSlice(@"[\]]");
string label;
Assert.True(LinkHelper.TryParseLabel(ref text, out label));
SourceSpan labelSpan;
Assert.True(LinkHelper.TryParseLabel(ref text, out label, out labelSpan));
Assert.AreEqual(new SourceSpan(1, 2), labelSpan);
Assert.AreEqual(@"]", label);
}
@@ -194,23 +233,36 @@ namespace Markdig.Tests
[Test]
public void TestLabelWhitespaceCollapsedAndTrim()
{
// 0 1 2 3
// 0123456789012345678901234567890123456789
var text = new StringSlice(@"[ fo o z ]");
string label;
Assert.True(LinkHelper.TryParseLabel(ref text, out label));
SourceSpan labelSpan;
Assert.True(LinkHelper.TryParseLabel(ref text, out label, out labelSpan));
Assert.AreEqual(new SourceSpan(6, 17), labelSpan);
Assert.AreEqual(@"fo o z", label);
}
[Test]
public void TestlLinkReferenceDefinitionSimple()
{
// 0 1 2 3
// 0123456789012345678901234567890123456789
var text = new StringSlice(@"[foo]: /toto 'title'");
string label;
string url;
string title;
Assert.True(LinkHelper.TryParseLinkReferenceDefinition(ref text, out label, out url, out title));
SourceSpan labelSpan;
SourceSpan urlSpan;
SourceSpan titleSpan;
Assert.True(LinkHelper.TryParseLinkReferenceDefinition(ref text, out label, out url, out title, out labelSpan, out urlSpan, out titleSpan));
Assert.AreEqual(@"foo", label);
Assert.AreEqual(@"/toto", url);
Assert.AreEqual(@"title", title);
Assert.AreEqual(new SourceSpan(1, 3), labelSpan);
Assert.AreEqual(new SourceSpan(7, 11), urlSpan);
Assert.AreEqual(new SourceSpan(13, 19), titleSpan);
}
[Test]

View File

@@ -3,10 +3,13 @@
// See the license.txt file in the project root for more information.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Markdig.Helpers;
using Markdig.Renderers.Html;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
using NUnit.Framework;
namespace Markdig.Tests
@@ -163,16 +166,44 @@ literal ( 0, 4) 4-5
}
[Test]
public void TestHtmlInline()
public void TestLinkParts1()
{
// 0123456789
Check("01<b>4</b>", @"
paragraph ( 0, 0) 0-9
literal ( 0, 0) 0-1
html ( 0, 2) 2-4
literal ( 0, 5) 5-5
html ( 0, 6) 6-9
");
// 0 1
// 01 2 3456789012345
var link = Markdown.Parse("0\n\n01 [234](/56)", new MarkdownPipelineBuilder().UsePreciseSourceLocation().Build()).Descendants().OfType<LinkInline>().FirstOrDefault();
Assert.NotNull(link);
Assert.AreEqual(new SourceSpan(7, 9), link.LabelSpan);
Assert.AreEqual(new SourceSpan(12, 14), link.UrlSpan);
Assert.AreEqual(SourceSpan.Empty, link.TitleSpan);
}
[Test]
public void TestLinkParts2()
{
// 0 1
// 01 2 34567890123456789
var link = Markdown.Parse("0\n\n01 [234](/56 'yo')", new MarkdownPipelineBuilder().UsePreciseSourceLocation().Build()).Descendants().OfType<LinkInline>().FirstOrDefault();
Assert.NotNull(link);
Assert.AreEqual(new SourceSpan(7, 9), link.LabelSpan);
Assert.AreEqual(new SourceSpan(12, 14), link.UrlSpan);
Assert.AreEqual(new SourceSpan(16, 19), link.TitleSpan);
}
[Test]
public void TestLinkParts3()
{
// 0 1
// 01 2 3456789012345
var link = Markdown.Parse("0\n\n01![234](/56)", new MarkdownPipelineBuilder().UsePreciseSourceLocation().Build()).Descendants().OfType<LinkInline>().FirstOrDefault();
Assert.NotNull(link);
Assert.AreEqual(new SourceSpan(5, 15), link.Span);
Assert.AreEqual(new SourceSpan(7, 9), link.LabelSpan);
Assert.AreEqual(new SourceSpan(12, 14), link.UrlSpan);
Assert.AreEqual(SourceSpan.Empty, link.TitleSpan);
}
[Test]
@@ -208,6 +239,64 @@ literal ( 4, 0) 16-16
");
}
[Test]
public void TestHtmlBlock1()
{
// 0 1
// 01 2 345678901 23
Check("0\n\n<!--A-->\n1\n", @"
paragraph ( 0, 0) 0-0
literal ( 0, 0) 0-0
html ( 2, 0) 3-10
paragraph ( 3, 0) 12-12
literal ( 3, 0) 12-12
");
}
[Test]
public void TestHtmlComment()
{
// 0 1 2
// 012345678901 234567890 1234
Check("# 012345678\n<!--0-->\n123\n", @"
heading ( 0, 0) 0-10
literal ( 0, 2) 2-10
html ( 1, 0) 12-19
paragraph ( 2, 0) 21-23
literal ( 2, 0) 21-23
");
}
[Test]
public void TestHtmlInline()
{
// 0123456789
Check("01<b>4</b>", @"
paragraph ( 0, 0) 0-9
literal ( 0, 0) 0-1
html ( 0, 2) 2-4
literal ( 0, 5) 5-5
html ( 0, 6) 6-9
");
}
[Test]
public void TestHtmlInline1()
{
// 0
// 0123456789
Check("0<!--A-->1", @"
paragraph ( 0, 0) 0-9
literal ( 0, 0) 0-0
html ( 0, 1) 1-8
literal ( 0, 9) 9-9
");
}
[Test]
public void TestThematicBreak()
{
@@ -479,7 +568,7 @@ literal ( 0, 0) 0-0
linebreak ( 0, 1) 1-1
literal ( 1, 0) 2-5
math ( 1, 4) 6-11
attributes ( 0, 0) 0-0
attributes ( 0, 0) 0--1
", "mathematics");
}
@@ -562,6 +651,54 @@ literal ( 4, 2) 13-13
", "pipetables");
}
[Test]
public void TestIndentedCode()
{
// 01 2 345678 9ABCDE
Check("0\n\n 0\n 1\n", @"
paragraph ( 0, 0) 0-0
literal ( 0, 0) 0-0
code ( 2, 4) 7-13
");
}
[Test]
public void TestIndentedCodeWithTabs()
{
// 01 2 3 45 6 78
Check("0\n\n\t0\n\t1\n", @"
paragraph ( 0, 0) 0-0
literal ( 0, 0) 0-0
code ( 2, 4) 4-7
");
}
[Test]
public void TestIndentedCodeWithMixedTabs()
{
// 01 2 34 56 78 9
Check("0\n\n \t0\n \t1\n", @"
paragraph ( 0, 0) 0-0
literal ( 0, 0) 0-0
code ( 2, 4) 5-9
");
}
[Test]
public void TestTabsInList()
{
// 012 34 567 89
Check("- \t0\n- \t1\n", @"
list ( 0, 0) 0-8
listitem ( 0, 0) 0-3
paragraph ( 0, 4) 3-3
literal ( 0, 4) 3-3
listitem ( 1, 0) 5-8
paragraph ( 1, 4) 8-8
literal ( 1, 4) 8-8
");
}
[Test]
public void TestDocument()
{
@@ -606,11 +743,11 @@ literal ( 8, 2) 77-92
foreach (var val in document.Descendants())
{
var name = GetTypeName(val.GetType());
build.Append($"{name,-12} ({val.Line,2},{val.Column,2}) {val.SourceStartPosition,2}-{val.SourceEndPosition}\n");
build.Append($"{name,-12} ({val.Line,2},{val.Column,2}) {val.Span.Start,2}-{val.Span.End}\n");
var attributes = val.TryGetAttributes();
if (attributes != null)
{
build.Append($"{"attributes",-12} ({attributes.Line,2},{attributes.Column,2}) {attributes.SourceStartPosition,2}-{attributes.SourceEndPosition}\n");
build.Append($"{"attributes",-12} ({attributes.Line,2},{attributes.Column,2}) {attributes.Span.Start,2}-{attributes.Span.End}\n");
}
}
var result = build.ToString().Trim();

View File

@@ -33,5 +33,10 @@ namespace Markdig.Extensions.Abbreviations
/// The text associated to this label.
/// </summary>
public StringSlice Text;
/// <summary>
/// The label span
/// </summary>
public SourceSpan LabelSpan;
}
}

View File

@@ -4,6 +4,7 @@
using System.Collections.Generic;
using Markdig.Helpers;
using Markdig.Parsers;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
namespace Markdig.Extensions.Abbreviations
@@ -38,8 +39,9 @@ namespace Markdig.Extensions.Abbreviations
return BlockState.None;
}
SourceSpan labelSpan;
string label;
if (!LinkHelper.TryParseLabel(ref slice, out label))
if (!LinkHelper.TryParseLabel(ref slice, out label, out labelSpan))
{
return BlockState.None;
}
@@ -57,10 +59,10 @@ namespace Markdig.Extensions.Abbreviations
{
Label = label,
Text = slice,
SourceStartPosition = startPosition,
SourceEndPosition = slice.End,
Span = new SourceSpan(startPosition, slice.End),
Line = processor.LineIndex,
Column = processor.Column
Column = processor.Column,
LabelSpan = labelSpan,
};
if (!processor.Document.HasAbbreviations())
{
@@ -126,8 +128,7 @@ namespace Markdig.Extensions.Abbreviations
{
container = new ContainerInline()
{
SourceStartPosition = originalLiteral.SourceStartPosition,
SourceEndPosition = originalLiteral.SourceEndPosition,
Span = originalLiteral.Span,
Line = originalLiteral.Line,
Column = originalLiteral.Column,
};
@@ -137,18 +138,21 @@ namespace Markdig.Extensions.Abbreviations
int column;
var abbrInline = new AbbreviationInline(abbr)
{
SourceStartPosition = processor.GetSourcePosition(i, out line, out column),
Span =
{
Start = processor.GetSourcePosition(i, out line, out column),
},
Line = line,
Column = column
};
abbrInline.SourceEndPosition = abbrInline.SourceStartPosition + match.Length - 1;
abbrInline.Span.End = abbrInline.Span.Start + match.Length - 1;
// Append the previous literal
if (i > content.Start)
{
container.AppendChild(literal);
literal.SourceEndPosition = abbrInline.SourceStartPosition - 1;
literal.Span.End = abbrInline.Span.Start - 1;
// Truncate it before the abbreviation
literal.Content.End = i - 1;
}
@@ -167,8 +171,7 @@ namespace Markdig.Extensions.Abbreviations
// Process the remaining literal
literal = new LiteralInline()
{
SourceStartPosition = abbrInline.SourceEndPosition + 1,
SourceEndPosition = literal.SourceEndPosition,
Span = new SourceSpan(abbrInline.Span.End + 1, literal.Span.End),
Line = line,
Column = column + match.Length,
};

View File

@@ -66,8 +66,7 @@ namespace Markdig.Extensions.DefinitionLists
{
currentDefinitionList = new DefinitionList(this)
{
SourceStartPosition = paragraphBlock.SourceStartPosition,
SourceEndPosition = processor.Line.End,
Span = new SourceSpan(paragraphBlock.Span.Start, processor.Line.End),
Column = paragraphBlock.Column,
Line = paragraphBlock.Line,
};
@@ -78,8 +77,7 @@ namespace Markdig.Extensions.DefinitionLists
{
Line = processor.LineIndex,
Column = column,
SourceStartPosition = startPosition,
SourceEndPosition = processor.Line.End,
Span = new SourceSpan(startPosition, processor.Line.End),
OpeningCharacter = processor.CurrentChar,
};
@@ -90,8 +88,7 @@ namespace Markdig.Extensions.DefinitionLists
{
Column = paragraphBlock.Column,
Line = line.Line,
SourceStartPosition = paragraphBlock.SourceStartPosition,
SourceEndPosition = paragraphBlock.SourceEndPosition,
Span = new SourceSpan(paragraphBlock.Span.Start, paragraphBlock.Span.End),
IsOpen = false
};
term.AppendLine(ref line.Slice, line.Column, line.Line, line.Position);
@@ -101,7 +98,7 @@ namespace Markdig.Extensions.DefinitionLists
processor.Open(definitionItem);
// Update the end position
currentDefinitionList.SourceEndPosition = processor.Line.End;
currentDefinitionList.Span.End = processor.Line.End;
return BlockState.Continue;
}
@@ -136,7 +133,7 @@ namespace Markdig.Extensions.DefinitionLists
definitionItem.RemoveAt(definitionItem.Count - 1);
}
list.SourceEndPosition = list.LastChild.SourceEndPosition;
list.Span.End = list.LastChild.Span.End;
return BlockState.None;
}
@@ -148,8 +145,7 @@ namespace Markdig.Extensions.DefinitionLists
processor.Close(definitionItem);
var nextDefinitionItem = new DefinitionItem(this)
{
SourceStartPosition = startPosition,
SourceEndPosition = processor.Line.End,
Span = new SourceSpan(startPosition, processor.Line.End),
Line = processor.LineIndex,
Column = processor.Column,
OpeningCharacter = processor.CurrentChar,
@@ -182,7 +178,7 @@ namespace Markdig.Extensions.DefinitionLists
definitionItem.RemoveAt(definitionItem.Count - 1);
}
list.SourceEndPosition = list.LastChild.SourceEndPosition;
list.Span.End = list.LastChild.Span.End;
return BlockState.Break;
}
}

View File

@@ -94,12 +94,15 @@ namespace Markdig.Extensions.Emoji
int column;
processor.Inline = new EmojiInline(unicode)
{
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
Span =
{
Start = processor.GetSourcePosition(startPosition, out line, out column),
},
Line = line,
Column = column,
Match = match
};
processor.Inline.SourceEndPosition = processor.Inline.SourceStartPosition + match.Length - 1;
processor.Inline.Span.End = processor.Inline.Span.Start + match.Length - 1;
return true;
}

View File

@@ -53,8 +53,7 @@ namespace Markdig.Extensions.Figures
var figure = new Figure(this)
{
SourceStartPosition = startPosition,
SourceEndPosition = line.End,
Span = new SourceSpan(startPosition, line.End),
Line = processor.LineIndex,
Column = processor.Column,
OpeningCharacter = matchChar,
@@ -66,8 +65,7 @@ namespace Markdig.Extensions.Figures
{
var caption = new FigureCaption(this)
{
SourceStartPosition = line.Start,
SourceEndPosition = line.End,
Span = new SourceSpan(line.Start, line.End),
Line = processor.LineIndex,
Column = column + line.Start - startPosition,
IsOpen = false
@@ -107,8 +105,7 @@ namespace Markdig.Extensions.Figures
{
var caption = new FigureCaption(this)
{
SourceStartPosition = line.Start,
SourceEndPosition = line.End,
Span = new SourceSpan(line.Start, line.End),
Line = processor.LineIndex,
Column = column + line.Start - startPosition,
IsOpen = false
@@ -117,7 +114,7 @@ namespace Markdig.Extensions.Figures
figure.Add(caption);
}
figure.SourceEndPosition = line.End;
figure.Span.End = line.End;
// Don't keep the last line
return BlockState.BreakDiscard;
@@ -126,7 +123,7 @@ namespace Markdig.Extensions.Figures
// Reset the indentation to the column before the indent
processor.GoToColumn(processor.ColumnBeforeIndent);
figure.SourceEndPosition = line.End;
figure.Span.End = line.End;
return BlockState.Continue;
}

View File

@@ -47,8 +47,7 @@ namespace Markdig.Extensions.Footers
}
processor.NewBlocks.Push(new FooterBlock(this)
{
SourceStartPosition = startPosition,
SourceEndPosition = processor.Line.End,
Span = new SourceSpan(startPosition, processor.Line.End),
OpeningCharacter = openingChar,
Column = column,
Line = processor.LineIndex,
@@ -81,7 +80,7 @@ namespace Markdig.Extensions.Footers
{
processor.NextChar(); // Skip following space
}
block.SourceEndPosition = processor.Line.End;
block.Span.End = processor.Line.End;
}
return result;
}

View File

@@ -34,6 +34,11 @@ namespace Markdig.Extensions.Footnotes
/// </summary>
public List<FootnoteLink> Links { get; private set; }
/// <summary>
/// The label span
/// </summary>
public SourceSpan LabelSpan;
internal bool IsLastLineEmpty { get; set; }
}
}

View File

@@ -36,7 +36,8 @@ namespace Markdig.Extensions.Footnotes
var saved = processor.Column;
string label;
int start = processor.Start;
if (!LinkHelper.TryParseLabel(ref processor.Line, false, out label) || !label.StartsWith("^") || processor.CurrentChar != ':')
SourceSpan labelSpan;
if (!LinkHelper.TryParseLabel(ref processor.Line, false, out label, out labelSpan) || !label.StartsWith("^") || processor.CurrentChar != ':')
{
processor.GoToColumn(saved);
return BlockState.None;
@@ -48,7 +49,11 @@ namespace Markdig.Extensions.Footnotes
processor.NextChar(); // Skip ':'
var footnote = new Footnote(this) {Label = label};
var footnote = new Footnote(this)
{
Label = label,
LabelSpan = labelSpan,
};
// Maintain a list of all footnotes at document level
var footnotes = processor.Document.GetData(DocumentKey) as FootnoteGroup;

View File

@@ -62,8 +62,8 @@ namespace Markdig.Extensions.GenericAttributes
// Update position for HtmlAttributes
htmlAttributes.Line = processor.LineIndex;
htmlAttributes.Column = startOfAttributes - processor.CurrentLineStartPosition; // This is not accurate with tabs!
htmlAttributes.SourceStartPosition = startOfAttributes;
htmlAttributes.SourceEndPosition = copy.Start - 1;
htmlAttributes.Span.Start = startOfAttributes;
htmlAttributes.Span.End = copy.Start - 1;
line.End = indexOfAttributes - 1;
return true;

View File

@@ -69,10 +69,10 @@ namespace Markdig.Extensions.GenericAttributes
// Update the position of the attributes
int line;
int column;
currentHtmlAttributes.SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column);
currentHtmlAttributes.Span.Start = processor.GetSourcePosition(startPosition, out line, out column);
currentHtmlAttributes.Line = line;
currentHtmlAttributes.Column = column;
currentHtmlAttributes.SourceEndPosition = currentHtmlAttributes.SourceStartPosition + slice.Start - startPosition - 1;
currentHtmlAttributes.Span.End = currentHtmlAttributes.Span.Start + slice.Start - startPosition - 1;
// We don't set the processor.Inline as we don't want to add attach attributes to a particular entity
return true;

View File

@@ -5,6 +5,7 @@
using Markdig.Helpers;
using Markdig.Parsers;
using Markdig.Renderers.Html;
using Markdig.Syntax;
namespace Markdig.Extensions.Mathematics
{
@@ -106,8 +107,7 @@ namespace Markdig.Extensions.Mathematics
int column;
var inline = new MathInline()
{
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
SourceEndPosition = processor.GetSourcePosition(slice.End),
Span = new SourceSpan(processor.GetSourcePosition(startPosition, out line, out column), processor.GetSourcePosition(slice.End)),
Line = line,
Column = column,
Delimiter = match,

View File

@@ -167,13 +167,13 @@ namespace Markdig.Extensions.SmartyPants
int column;
var pant = new SmartyPant()
{
SourceStartPosition = processor.GetSourcePosition(startingPosition, out line, out column),
Span = {Start = processor.GetSourcePosition(startingPosition, out line, out column)},
Line = line,
Column = column,
OpeningCharacter = openingChar,
Type = type
};
pant.SourceEndPosition = pant.SourceStartPosition + slice.Start - startingPosition - 1;
pant.Span.End = pant.Span.Start + slice.Start - startingPosition - 1;
// We will check in a post-process step for balanaced open/close quotes
if (postProcess)
@@ -258,8 +258,7 @@ namespace Markdig.Extensions.SmartyPants
pants.RemoveAt(j);
toReplace.ReplaceBy(new LiteralInline(toReplace.ToString())
{
SourceStartPosition = toReplace.SourceStartPosition,
SourceEndPosition = toReplace.SourceEndPosition,
Span = toReplace.Span,
Line = toReplace.Line,
Column = toReplace.Column,
});
@@ -282,8 +281,7 @@ namespace Markdig.Extensions.SmartyPants
{
quote.ReplaceBy(new LiteralInline(quote.ToString())
{
SourceStartPosition = quote.SourceStartPosition,
SourceEndPosition = quote.SourceEndPosition,
Span = quote.Span,
Line = quote.Line,
Column = quote.Column,
});

View File

@@ -28,6 +28,8 @@ namespace Markdig.Extensions.Tables
public void Setup(MarkdownPipelineBuilder pipeline)
{
// Pipe tables require precise source location
pipeline.PreciseSourceLocation = true;
if (!pipeline.BlockParsers.Contains<PipeTableBlockParser>())
{
pipeline.BlockParsers.Insert(0, new PipeTableBlockParser());

View File

@@ -102,8 +102,7 @@ namespace Markdig.Extensions.Tables
{
processor.Inline = new PiprTableDelimiterInline(this)
{
SourceStartPosition = position,
SourceEndPosition = position,
Span = new SourceSpan(position, position),
Line = globalLineIndex,
Column = column,
LocalLineIndex = localLineIndex
@@ -236,8 +235,8 @@ namespace Markdig.Extensions.Tables
}
// TODO: This is not accurate for the table
table.SourceStartPosition = column.SourceStartPosition;
table.SourceEndPosition = column.SourceEndPosition;
table.Span.Start = column.Span.Start;
table.Span.End = column.Span.End;
table.Line = column.Line;
table.Column = column.Column;
@@ -287,17 +286,16 @@ namespace Markdig.Extensions.Tables
{
cellContainer.Line = item.Line;
cellContainer.Column = item.Column;
cellContainer.SourceStartPosition = item.SourceStartPosition;
cellContainer.Span.Start = item.Span.Start;
isFirstItem = false;
}
cellContainer.SourceEndPosition = item.SourceEndPosition;
cellContainer.Span.End = item.Span.End;
item = nextSibling;
}
var tableParagraph = new ParagraphBlock()
{
SourceStartPosition = cellContainer.SourceStartPosition,
SourceEndPosition = cellContainer.SourceEndPosition,
Span = cellContainer.Span,
Line = cellContainer.Line,
Column = cellContainer.Column,
Inline = cellContainer
@@ -305,8 +303,7 @@ namespace Markdig.Extensions.Tables
var tableCell = new TableCell()
{
SourceStartPosition = cellContainer.SourceStartPosition,
SourceEndPosition = cellContainer.SourceEndPosition,
Span = cellContainer.Span,
Line = cellContainer.Line,
Column = cellContainer.Column,
};
@@ -317,8 +314,7 @@ namespace Markdig.Extensions.Tables
{
row = new TableRow()
{
SourceStartPosition = cellContainer.SourceStartPosition,
SourceEndPosition = cellContainer.SourceEndPosition,
Span = cellContainer.Span,
Line = cellContainer.Line,
Column = cellContainer.Column,
};
@@ -448,8 +444,8 @@ namespace Markdig.Extensions.Tables
}
// Check the left side of a `|` delimiter
TableColumnAlign align;
if (!ParseHeaderString(delimiter.PreviousSibling, out align))
TableColumnAlign align = TableColumnAlign.Left;
if (delimiter.PreviousSibling != null && !ParseHeaderString(delimiter.PreviousSibling, out align))
{
break;
}

View File

@@ -49,12 +49,12 @@ namespace Markdig.Extensions.TaskLists
int column;
var taskItem = new TaskList()
{
SourceStartPosition = processor.GetSourcePosition(startingPosition, out line, out column),
Span = { Start = processor.GetSourcePosition(startingPosition, out line, out column)},
Line = line,
Column = column,
Checked = !c.IsSpace()
};
taskItem.SourceEndPosition = taskItem.SourceStartPosition + 2;
taskItem.Span.End = taskItem.Span.Start + 2;
processor.Inline = taskItem;
return true;
}

View File

@@ -297,10 +297,24 @@ namespace Markdig.Helpers
public static bool TryParseInlineLink(StringSlice text, out string link, out string title)
{
return TryParseInlineLink(ref text, out link, out title);
SourceSpan linkSpan;
SourceSpan titleSpan;
return TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan);
}
public static bool TryParseInlineLink(StringSlice text, out string link, out string title, out SourceSpan linkSpan, out SourceSpan titleSpan)
{
return TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan);
}
public static bool TryParseInlineLink(ref StringSlice text, out string link, out string title)
{
SourceSpan linkSpan;
SourceSpan titleSpan;
return TryParseInlineLink(ref text, out link, out title, out linkSpan, out titleSpan);
}
public static bool TryParseInlineLink(ref StringSlice text, out string link, out string title, out SourceSpan linkSpan, out SourceSpan titleSpan)
{
// 1. An inline link consists of a link text followed immediately by a left parenthesis (,
// 2. optional whitespace, TODO: specs: is it whitespace or multiple whitespaces?
@@ -313,14 +327,25 @@ namespace Markdig.Helpers
link = null;
title = null;
linkSpan = SourceSpan.Empty;
titleSpan = SourceSpan.Empty;
// 1. An inline link consists of a link text followed immediately by a left parenthesis (,
if (c == '(')
{
text.NextChar();
text.TrimStart();
var pos = text.Start;
if (TryParseUrl(ref text, out link))
{
linkSpan.Start = pos;
linkSpan.End = text.Start - 1;
if (linkSpan.End < linkSpan.Start)
{
linkSpan = SourceSpan.Empty;
}
int spaceCount;
text.TrimStart(out spaceCount);
var hasWhiteSpaces = spaceCount > 0;
@@ -333,12 +358,19 @@ namespace Markdig.Helpers
else if (hasWhiteSpaces)
{
c = text.CurrentChar;
pos = text.Start;
if (c == ')')
{
isValid = true;
}
else if (TryParseTitle(ref text, out title))
{
titleSpan.Start = pos;
titleSpan.End = text.Start - 1;
if (titleSpan.End < titleSpan.Start)
{
titleSpan = SourceSpan.Empty;
}
text.TrimStart();
c = text.CurrentChar;
@@ -582,12 +614,25 @@ namespace Markdig.Helpers
return TryParseLinkReferenceDefinition(ref text, out label, out url, out title);
}
public static bool TryParseLinkReferenceDefinition<T>(ref T text, out string label, out string url,
out string title) where T : ICharIterator
public static bool TryParseLinkReferenceDefinition<T>(ref T text, out string label, out string url, out string title)
where T : ICharIterator
{
SourceSpan labelSpan;
SourceSpan urlSpan;
SourceSpan titleSpan;
return TryParseLinkReferenceDefinition(ref text, out label, out url, out title, out labelSpan, out urlSpan,
out titleSpan);
}
public static bool TryParseLinkReferenceDefinition<T>(ref T text, out string label, out string url, out string title, out SourceSpan labelSpan, out SourceSpan urlSpan, out SourceSpan titleSpan) where T : ICharIterator
{
url = null;
title = null;
if (!TryParseLabel(ref text, out label))
urlSpan = SourceSpan.Empty;
titleSpan = SourceSpan.Empty;
if (!TryParseLabel(ref text, out label, out labelSpan))
{
return false;
}
@@ -601,10 +646,13 @@ namespace Markdig.Helpers
// Skip any whitespaces before the url
text.TrimStart();
urlSpan.Start = text.Start;
if (!TryParseUrl(ref text, out url) || string.IsNullOrEmpty(url))
{
return false;
}
urlSpan.End = text.Start - 1;
var saved = text;
int newLineCount;
@@ -612,8 +660,10 @@ namespace Markdig.Helpers
var c = text.CurrentChar;
if (c == '\'' || c == '"' || c == '(')
{
titleSpan.Start = text.Start;
if (TryParseTitle(ref text, out title))
{
titleSpan.End = text.Start - 1;
// If we have a title, it requires a whitespace after the url
if (!hasWhiteSpaces)
{
@@ -662,24 +712,41 @@ namespace Markdig.Helpers
public static bool TryParseLabel<T>(T lines, out string label) where T : ICharIterator
{
return TryParseLabel(ref lines, false, out label);
SourceSpan labelSpan;
return TryParseLabel(ref lines, false, out label, out labelSpan);
}
public static bool TryParseLabel<T>(T lines, out string label, out SourceSpan labelSpan) where T : ICharIterator
{
return TryParseLabel(ref lines, false, out label, out labelSpan);
}
public static bool TryParseLabel<T>(ref T lines, out string label) where T : ICharIterator
{
return TryParseLabel(ref lines, false, out label);
SourceSpan labelSpan;
return TryParseLabel(ref lines, false, out label, out labelSpan);
}
public static bool TryParseLabel<T>(ref T lines, bool allowEmpty, out string label) where T : ICharIterator
public static bool TryParseLabel<T>(ref T lines, out string label, out SourceSpan labelSpan) where T : ICharIterator
{
return TryParseLabel(ref lines, false, out label, out labelSpan);
}
public static bool TryParseLabel<T>(ref T lines, bool allowEmpty, out string label, out SourceSpan labelSpan) where T : ICharIterator
{
label = null;
char c = lines.CurrentChar;
labelSpan = SourceSpan.Empty;
if (c != '[')
{
return false;
}
var buffer = StringBuilderCache.Local();
var startLabel = -1;
var endLabel = -1;
bool hasEscape = false;
bool previousWhitespace = true;
bool hasNonWhiteSpace = false;
@@ -719,11 +786,19 @@ namespace Markdig.Helpers
break;
}
buffer.Length = i;
endLabel--;
}
// Only valid if buffer is less than 1000 characters
if (buffer.Length <= 999)
{
labelSpan.Start = startLabel;
labelSpan.End = endLabel;
if (labelSpan.Start > labelSpan.End)
{
labelSpan = SourceSpan.Empty;
}
label = buffer.ToString();
isValid = true;
}
@@ -741,6 +816,10 @@ namespace Markdig.Helpers
if (!hasEscape && c == '\\')
{
if (startLabel < 0)
{
startLabel = lines.Start;
}
hasEscape = true;
}
else
@@ -749,6 +828,11 @@ namespace Markdig.Helpers
if (!previousWhitespace || !isWhitespace)
{
if (startLabel < 0)
{
startLabel = lines.Start;
}
endLabel = lines.Start;
buffer.Append(c);
if (!isWhitespace)
{

View File

@@ -212,7 +212,7 @@ namespace Markdig.Helpers
index = Start + offset;
for (int i = index; i <= end; i ++)
{
if (Match(text, End, i))
if (Match(text, End, i - Start))
{
index = i + text.Length;
return true;

View File

@@ -590,8 +590,16 @@ namespace Markdig.Parsers
/// </summary>
private void TryOpenBlocks()
{
int previousStart = -1;
while (ContinueProcessingLine)
{
// Security check so that the parser can't go into a crazy infinite loop if one extension is messing
if (previousStart == Start)
{
throw new InvalidOperationException($"The parser is in an invalid infinite loop while trying to parse blocks at line [{LineIndex}] with line [{Line}]");
}
previousStart = Start;
// Eat indent spaces before checking the character
ParseIndent();

View File

@@ -159,8 +159,8 @@ namespace Markdig.Parsers
fenced.Column = processor.Column;
fenced.FencedChar = matchChar;
fenced.FencedCharCount = count;
fenced.SourceStartPosition = startPosition;
fenced.SourceEndPosition = line.Start;
fenced.Span.Start = startPosition;
fenced.Span.End = line.Start;
};
// Try to parse any attached attributes
@@ -216,7 +216,7 @@ namespace Markdig.Parsers
// The line must contain only fence opening character followed only by whitespaces.
if (count <=0 && !processor.IsCodeIndent && (c == '\0' || c.IsWhitespace()) && line.TrimEnd())
{
block.SourceEndPosition = line.Start - 1;
block.Span.End = line.Start - 1;
// Don't keep the last line
return BlockState.BreakDiscard;

View File

@@ -70,7 +70,7 @@ namespace Markdig.Parsers
HeaderChar = matchingChar,
Level = leadingCount,
Column = column,
SourceStartPosition = sourcePosition
Span = { Start = sourcePosition }
};
processor.NewBlocks.Push(headingBlock);
processor.GoToColumn(column + leadingCount + 1);
@@ -119,7 +119,7 @@ namespace Markdig.Parsers
}
// Setup the source end position of this element
headingBlock.SourceEndPosition = processor.Line.End;
headingBlock.Span.End = processor.Line.End;
// We expect a single line, so don't continue
return BlockState.Break;

View File

@@ -185,35 +185,35 @@ namespace Markdig.Parsers
case HtmlBlockType.Comment:
if (line.Search("-->", out endof))
{
htmlBlock.SourceEndPosition = endof - 1;
htmlBlock.Span.End = endof - 1;
result = BlockState.Break;
}
break;
case HtmlBlockType.CData:
if (line.Search("]]>", out endof))
{
htmlBlock.SourceEndPosition = endof - 1;
htmlBlock.Span.End = endof - 1;
result = BlockState.Break;
}
break;
case HtmlBlockType.ProcessingInstruction:
if (line.Search("?>", out endof))
{
htmlBlock.SourceEndPosition = endof - 1;
htmlBlock.Span.End = endof - 1;
result = BlockState.Break;
}
break;
case HtmlBlockType.DocumentType:
if (line.Search(">", out endof))
{
htmlBlock.SourceEndPosition = endof - 1;
htmlBlock.Span.End = endof - 1;
result = BlockState.Break;
}
break;
case HtmlBlockType.ScriptPreOrStyle:
if (line.SearchLowercase("</script>", out endof) || line.SearchLowercase("</pre>", out endof) || line.SearchLowercase("</style>", out endof))
{
htmlBlock.SourceEndPosition = endof - 1;
htmlBlock.Span.End = endof - 1;
result = BlockState.Break;
}
break;
@@ -234,7 +234,7 @@ namespace Markdig.Parsers
// Update only if we don't have a break discard
if (result != BlockState.BreakDiscard)
{
htmlBlock.SourceEndPosition = line.End;
htmlBlock.Span.End = line.End;
}
return result;
@@ -246,9 +246,8 @@ namespace Markdig.Parsers
{
Column = startColumn,
Type = type,
SourceStartPosition = startPosition,
// By default, setup to the end of line
SourceEndPosition = startPosition + state.Line.End
Span = new SourceSpan(startPosition, startPosition + state.Line.End)
});
return BlockState.Continue;
}

View File

@@ -25,8 +25,7 @@ namespace Markdig.Parsers
processor.NewBlocks.Push(new CodeBlock(this)
{
Column = processor.Column,
SourceStartPosition = startPosition,
SourceEndPosition = processor.Line.End
Span = new SourceSpan(startPosition, processor.Line.End)
});
}
return result;
@@ -49,7 +48,7 @@ namespace Markdig.Parsers
}
if (block != null)
{
block.SourceEndPosition = processor.Line.End;
block.Span.End = processor.Line.End;
}
return BlockState.Continue;
}

View File

@@ -119,6 +119,18 @@ namespace Markdig.Parsers
return GetSourcePosition(sliceOffset, out lineIndex, out column);
}
public SourceSpan GetSourcePositionFromLocalSpan(SourceSpan span)
{
if (span.IsEmpty)
{
return SourceSpan.Empty;
}
int column;
int lineIndex;
return new SourceSpan(GetSourcePosition(span.Start, out lineIndex, out column), GetSourcePosition(span.End, out lineIndex, out column));
}
/// <summary>
/// Gets the source position for the specified offset within the current slice.
/// </summary>
@@ -159,6 +171,8 @@ namespace Markdig.Parsers
/// <param name="leafBlock">The leaf block.</param>
public void ProcessInlineLeaf(LeafBlock leafBlock)
{
if (leafBlock == null) throw new ArgumentNullException(nameof(leafBlock));
// clear parser states
Array.Clear(ParserStates, 0, ParserStates.Length);
@@ -175,8 +189,17 @@ namespace Markdig.Parsers
var text = leafBlock.Lines.ToSlice(lineOffsets);
leafBlock.Lines = new StringLineGroup();
int previousStart = -1;
while (!text.IsEmpty)
{
// Security check so that the parser can't go into a crazy infinite loop if one extension is messing
if (previousStart == text.Start)
{
throw new InvalidOperationException($"The parser is in an invalid infinite loop while trying to parse inlines for block [{leafBlock.GetType().Name}] at position ({leafBlock.ToPositionText()}");
}
previousStart = text.Start;
var c = text.CurrentChar;
var textSaved = text;

View File

@@ -2,6 +2,7 @@
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using Markdig.Helpers;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
namespace Markdig.Parsers.Inlines
@@ -39,8 +40,7 @@ namespace Markdig.Parsers.Inlines
{
IsEmail = isEmail,
Url = link,
SourceStartPosition = processor.GetSourcePosition(saved.Start, out line, out column),
SourceEndPosition = processor.GetSourcePosition(slice.Start - 1),
Span = new SourceSpan(processor.GetSourcePosition(saved.Start, out line, out column), processor.GetSourcePosition(slice.Start - 1)),
Line = line,
Column = column
};
@@ -57,8 +57,7 @@ namespace Markdig.Parsers.Inlines
processor.Inline = new HtmlInline()
{
Tag = htmlTag,
SourceStartPosition = processor.GetSourcePosition(saved.Start, out line, out column),
SourceEndPosition = processor.GetSourcePosition(slice.Start - 1),
Span = new SourceSpan(processor.GetSourcePosition(saved.Start, out line, out column), processor.GetSourcePosition(slice.Start - 1)),
Line = line,
Column = column
};

View File

@@ -2,6 +2,7 @@
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using Markdig.Helpers;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
namespace Markdig.Parsers.Inlines
@@ -106,8 +107,7 @@ namespace Markdig.Parsers.Inlines
{
Delimiter = match,
Content = builder.ToString(),
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
SourceEndPosition = processor.GetSourcePosition(slice.Start - 1),
Span = new SourceSpan(processor.GetSourcePosition(startPosition, out line, out column), processor.GetSourcePosition(slice.Start - 1)),
Line = line,
Column = column
};

View File

@@ -185,8 +185,7 @@ namespace Markdig.Parsers.Inlines
{
DelimiterCount = delimiterCount,
Type = delimiterType,
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
SourceEndPosition = processor.GetSourcePosition(slice.Start - 1),
Span = new SourceSpan(processor.GetSourcePosition(startPosition, out line, out column), processor.GetSourcePosition(slice.Start - 1)),
Column = column,
Line = line,
};
@@ -254,14 +253,14 @@ namespace Markdig.Parsers.Inlines
var closeDelimitercount = closeDelimiter.DelimiterCount;
var delimiterDelta = isStrong ? 2 : 1;
emphasis.SourceStartPosition = openDelimiter.SourceStartPosition;
emphasis.Span.Start = openDelimiter.Span.Start;
emphasis.Line = openDelimiter.Line;
emphasis.Column = openDelimiter.Column;
emphasis.SourceEndPosition = closeDelimiter.SourceEndPosition - closeDelimitercount + delimiterDelta;
emphasis.Span.End = closeDelimiter.Span.End - closeDelimitercount + delimiterDelta;
openDelimiter.SourceStartPosition += delimiterDelta;
openDelimiter.Span.Start += delimiterDelta;
openDelimiter.Column += delimiterDelta;
closeDelimiter.SourceStartPosition += delimiterDelta;
closeDelimiter.Span.Start += delimiterDelta;
closeDelimiter.Column += delimiterDelta;
openDelimiter.DelimiterCount -= delimiterDelta;
@@ -301,8 +300,7 @@ namespace Markdig.Parsers.Inlines
{
Content = new StringSlice(literalDelimiter.ToLiteral()),
IsClosed = true,
SourceStartPosition = literalDelimiter.SourceStartPosition,
SourceEndPosition = literalDelimiter.SourceEndPosition,
Span = literalDelimiter.Span,
Line = literalDelimiter.Line,
Column = literalDelimiter.Column
};
@@ -355,8 +353,7 @@ namespace Markdig.Parsers.Inlines
{
Content = new StringSlice(closeDelimiter.ToLiteral()),
IsClosed = true,
SourceStartPosition = closeDelimiter.SourceStartPosition,
SourceEndPosition = closeDelimiter.SourceEndPosition,
Span = closeDelimiter.Span,
Line = closeDelimiter.Line,
Column = closeDelimiter.Column
};
@@ -383,8 +380,7 @@ namespace Markdig.Parsers.Inlines
{
Content = new StringSlice(delimiter.ToLiteral()),
IsClosed = true,
SourceStartPosition = delimiter.SourceStartPosition,
SourceEndPosition = delimiter.SourceEndPosition,
Span = delimiter.Span,
Line = delimiter.Line,
Column = delimiter.Column
};

View File

@@ -29,11 +29,11 @@ namespace Markdig.Parsers.Inlines
processor.Inline = new LiteralInline()
{
Content = new StringSlice(new string(c, 1)),
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
Span = { Start = processor.GetSourcePosition(startPosition, out line, out column) },
Line = line,
Column = column
};
processor.Inline.SourceEndPosition = processor.Inline.SourceStartPosition + 1;
processor.Inline.Span.End = processor.Inline.Span.Start + 1;
slice.NextChar();
return true;
}
@@ -44,11 +44,11 @@ namespace Markdig.Parsers.Inlines
processor.Inline = new LineBreakInline()
{
IsHard = true,
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
Span = { Start = processor.GetSourcePosition(startPosition, out line, out column) },
Line = line,
Column = column
};
processor.Inline.SourceEndPosition = processor.Inline.SourceStartPosition + 1;
processor.Inline.Span.End = processor.Inline.Span.Start + 1;
slice.NextChar();
return true;
}

View File

@@ -2,6 +2,7 @@
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using Markdig.Helpers;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
namespace Markdig.Parsers.Inlines
@@ -51,8 +52,7 @@ namespace Markdig.Parsers.Inlines
{
Original = matched,
Transcoded = new StringSlice(literal),
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
SourceEndPosition = processor.GetSourcePosition(matched.End + 1),
Span = new SourceSpan(processor.GetSourcePosition(startPosition, out line, out column), processor.GetSourcePosition(matched.End + 1)),
Line = line,
Column = column
};

View File

@@ -42,12 +42,12 @@ namespace Markdig.Parsers.Inlines
int column;
processor.Inline = new LineBreakInline
{
SourceStartPosition = processor.GetSourcePosition(startPosition, out line, out column),
Span = { Start = processor.GetSourcePosition(startPosition, out line, out column)},
IsHard = EnableSoftAsHard || (slice.Start != 0 && hasDoubleSpacesBefore),
Line = line,
Column = column
};
processor.Inline.SourceEndPosition = processor.Inline.SourceStartPosition;
processor.Inline.Span.End = processor.Inline.Span.Start;
return true;
}
}

View File

@@ -51,8 +51,9 @@ namespace Markdig.Parsers.Inlines
var saved = slice;
string label;
SourceSpan labelSpan;
// If the label is followed by either a ( or a [, this is not a shortcut
if (LinkHelper.TryParseLabel(ref slice, out label))
if (LinkHelper.TryParseLabel(ref slice, out label, out labelSpan))
{
if (!processor.Document.ContainsLinkReferenceDefinition(label))
{
@@ -67,9 +68,9 @@ namespace Markdig.Parsers.Inlines
{
Type = DelimiterType.Open,
Label = label,
LabelSpan = processor.GetSourcePositionFromLocalSpan(labelSpan),
IsImage = isImage,
SourceStartPosition = startPosition,
SourceEndPosition = processor.GetSourcePosition(slice.Start - 1),
Span = new SourceSpan(startPosition, processor.GetSourcePosition(slice.Start - 1)),
Line = line,
Column = column
};
@@ -94,7 +95,7 @@ namespace Markdig.Parsers.Inlines
return false;
}
private bool ProcessLinkReference(InlineProcessor state, string label, LinkDelimiterInline parent, int endPosition)
private bool ProcessLinkReference(InlineProcessor state, string label, SourceSpan labelSpan, LinkDelimiterInline parent, int endPosition)
{
bool isValidLink = false;
LinkReferenceDefinition linkRef;
@@ -115,9 +116,11 @@ namespace Markdig.Parsers.Inlines
{
Url = HtmlHelper.Unescape(linkRef.Url),
Title = HtmlHelper.Unescape(linkRef.Title),
Label = label,
LabelSpan = labelSpan,
IsImage = parent.IsImage,
SourceStartPosition = parent.SourceStartPosition,
SourceEndPosition = endPosition,
Reference = linkRef,
Span = new SourceSpan(parent.Span.Start, endPosition),
Line = parent.Line,
Column = parent.Column,
};
@@ -134,8 +137,7 @@ namespace Markdig.Parsers.Inlines
Content = new StringSlice(label),
IsClosed = true,
// Not exact but we leave it like this
SourceStartPosition = parent.SourceStartPosition,
SourceEndPosition = parent.SourceEndPosition,
Span = parent.Span,
Line = parent.Line,
Column = parent.Column,
};
@@ -194,8 +196,7 @@ namespace Markdig.Parsers.Inlines
inlineState.Inline = new LiteralInline()
{
Content = new StringSlice("["),
SourceStartPosition = openParent.SourceStartPosition,
SourceEndPosition = openParent.SourceStartPosition,
Span = openParent.Span,
Line = openParent.Line,
Column = openParent.Column,
};
@@ -214,7 +215,9 @@ namespace Markdig.Parsers.Inlines
case '(':
string url;
string title;
if (LinkHelper.TryParseInlineLink(ref text, out url, out title))
SourceSpan linkSpan;
SourceSpan titleSpan;
if (LinkHelper.TryParseInlineLink(ref text, out url, out title, out linkSpan, out titleSpan))
{
// Inline Link
var link = new LinkInline()
@@ -222,8 +225,10 @@ namespace Markdig.Parsers.Inlines
Url = HtmlHelper.Unescape(url),
Title = HtmlHelper.Unescape(title),
IsImage = openParent.IsImage,
SourceStartPosition = openParent.SourceStartPosition,
SourceEndPosition = inlineState.GetSourcePosition(text.Start -1),
LabelSpan = openParent.LabelSpan,
UrlSpan = inlineState.GetSourcePositionFromLocalSpan(linkSpan),
TitleSpan = inlineState.GetSourcePositionFromLocalSpan(titleSpan),
Span = new SourceSpan(openParent.Span.Start, inlineState.GetSourcePosition(text.Start -1)),
Line = openParent.Line,
Column = openParent.Column,
};
@@ -250,13 +255,17 @@ namespace Markdig.Parsers.Inlines
break;
default:
var labelSpan = SourceSpan.Empty;
string label = null;
bool isLabelSpanLocal = true;
// Handle Collapsed links
if (text.CurrentChar == '[')
{
if (text.PeekChar(1) == ']')
{
label = openParent.Label;
labelSpan = openParent.LabelSpan;
isLabelSpanLocal = false;
text.NextChar(); // Skip [
text.NextChar(); // Skip ]
}
@@ -266,9 +275,14 @@ namespace Markdig.Parsers.Inlines
label = openParent.Label;
}
if (label != null || LinkHelper.TryParseLabel(ref text, true, out label))
if (label != null || LinkHelper.TryParseLabel(ref text, true, out label, out labelSpan))
{
if (ProcessLinkReference(inlineState, label, openParent, inlineState.GetSourcePosition(text.Start - 1)))
if (isLabelSpanLocal)
{
labelSpan = inlineState.GetSourcePositionFromLocalSpan(labelSpan);
}
if (ProcessLinkReference(inlineState, label, labelSpan, openParent, inlineState.GetSourcePosition(text.Start - 1)))
{
// Remove the open parent
openParent.Remove();
@@ -292,6 +306,7 @@ namespace Markdig.Parsers.Inlines
var literal = new LiteralInline()
{
Span = openParent.Span,
Content = new StringSlice(openParent.IsImage ? "![" : "[")
};

View File

@@ -3,6 +3,7 @@
// See the license.txt file in the project root for more information.
using Markdig.Helpers;
using Markdig.Syntax;
using Markdig.Syntax.Inlines;
namespace Markdig.Parsers.Inlines
@@ -67,8 +68,7 @@ namespace Markdig.Parsers.Inlines
processor.Inline = new LiteralInline()
{
Content = length > 0 ? new StringSlice(slice.Text, slice.Start, endPosition) : StringSlice.Empty,
SourceStartPosition = startPosition,
SourceEndPosition = processor.GetSourcePosition(endPosition),
Span = new SourceSpan(startPosition, processor.GetSourcePosition(endPosition)),
Line = line,
Column = column,
};

View File

@@ -153,7 +153,7 @@ namespace Markdig.Parsers
}
// Update list-item source end position
listItem.SourceEndPosition = state.Line.End;
listItem.Span.End = state.Line.End;
return BlockState.Continue;
}
@@ -174,7 +174,7 @@ namespace Markdig.Parsers
}
// Update list-item source end position
listItem.SourceEndPosition = state.Line.End;
listItem.Span.End = state.Line.End;
return BlockState.Continue;
}
@@ -258,8 +258,7 @@ namespace Markdig.Parsers
{
Column = initColumn,
ColumnWidth = columnWidth,
SourceStartPosition = sourcePosition,
SourceEndPosition = sourceEndPosition
Span = new SourceSpan(sourcePosition, sourceEndPosition)
};
state.NewBlocks.Push(newListItem);
@@ -286,8 +285,7 @@ namespace Markdig.Parsers
var newList = new ListBlock(this)
{
Column = initColumn,
SourceStartPosition = sourcePosition,
SourceEndPosition = sourceEndPosition,
Span = new SourceSpan(sourcePosition, sourceEndPosition),
IsOrdered = isOrdered,
BulletType = listInfo.BulletType,
OrderedDelimiter = listInfo.OrderedDelimiter,
@@ -357,7 +355,7 @@ namespace Markdig.Parsers
// Update end-position for the list
if (listBlock.Count > 0)
{
listBlock.SourceEndPosition = listBlock[listBlock.Count - 1].SourceEndPosition;
listBlock.Span.End = listBlock[listBlock.Count - 1].Span.End;
}
return true;

View File

@@ -23,8 +23,7 @@ namespace Markdig.Parsers
processor.NewBlocks.Push(new ParagraphBlock(this)
{
Column = processor.Column,
SourceStartPosition = processor.Line.Start,
SourceEndPosition = processor.Line.End
Span = new SourceSpan(processor.Line.Start, processor.Line.End)
});
return BlockState.Continue;
}
@@ -41,7 +40,7 @@ namespace Markdig.Parsers
return TryParseSetexHeading(processor, block);
}
block.SourceEndPosition = processor.Line.End;
block.Span.End = processor.Line.End;
return BlockState.Continue;
}
@@ -128,8 +127,7 @@ namespace Markdig.Parsers
var heading = new HeadingBlock(this)
{
Column = paragraph.Column,
SourceStartPosition = paragraph.SourceStartPosition,
SourceEndPosition = line.Start,
Span = new SourceSpan(paragraph.Span.Start, line.Start),
Level = level,
Lines = paragraph.Lines,
};
@@ -141,7 +139,7 @@ namespace Markdig.Parsers
return BlockState.BreakDiscard;
}
block.SourceEndPosition = state.Line.End;
block.Span.End = state.Line.End;
return BlockState.Continue;
}

View File

@@ -42,8 +42,7 @@ namespace Markdig.Parsers
{
QuoteChar = quoteChar,
Column = column,
SourceStartPosition = sourcePosition,
SourceEndPosition = processor.Line.End,
Span = new SourceSpan(sourcePosition, processor.Line.End),
});
return BlockState.Continue;
}
@@ -62,7 +61,7 @@ namespace Markdig.Parsers
var c = processor.CurrentChar;
if (c != quote.QuoteChar)
{
block.SourceEndPosition = processor.Start - 1;
block.Span.End = processor.Start - 1;
return processor.IsBlankLine ? BlockState.BreakDiscard : BlockState.None;
}
@@ -72,7 +71,7 @@ namespace Markdig.Parsers
processor.NextChar(); // Skip following space
}
block.SourceEndPosition = processor.Line.End;
block.Span.End = processor.Line.End;
return BlockState.Continue;
}
@@ -81,7 +80,7 @@ namespace Markdig.Parsers
var quoteBlock = block as QuoteBlock;
if (quoteBlock?.LastChild != null)
{
quoteBlock.SourceEndPosition = quoteBlock.LastChild.SourceEndPosition;
quoteBlock.Span.End = quoteBlock.LastChild.Span.End;
}
return true;
}

View File

@@ -88,8 +88,7 @@ namespace Markdig.Parsers
processor.NewBlocks.Push(new ThematicBreakBlock(this)
{
Column = processor.Column,
SourceStartPosition = startPosition,
SourceEndPosition = line.End
Span = new SourceSpan(startPosition, line.End)
});
return BlockState.BreakDiscard;
}

View File

@@ -25,6 +25,6 @@ namespace Markdig
{
public static partial class Markdown
{
public const string Version = "0.5.1";
public const string Version = "0.5.4";
}
}

View File

@@ -69,9 +69,9 @@ namespace Markdig.Syntax
children[Count++] = item;
item.Parent = this;
if (item.SourceEndPosition > SourceEndPosition)
if (item.Span.End > Span.End)
{
SourceEndPosition = item.SourceEndPosition;
Span.End = item.Span.End;
}
}

View File

@@ -25,6 +25,11 @@ namespace Markdig.Syntax.Inlines
/// </summary>
public string Label { get; set; }
/// <summary>
/// The label span
/// </summary>
public SourceSpan LabelSpan;
public override string ToLiteral()
{
return IsImage ? "![" : "[";

View File

@@ -54,9 +54,34 @@ namespace Markdig.Syntax.Inlines
/// </summary>
public string Title { get; set; }
/// <summary>
/// Gets or sets the label.
/// </summary>
public string Label { get; set; }
/// <summary>
/// Gets or sets a value indicating whether this instance is an image link.
/// </summary>
public bool IsImage { get; set; }
/// <summary>
/// Gets or sets the reference this link is attached to. May be null.
/// </summary>
public LinkReferenceDefinition Reference { get; set; }
/// <summary>
/// The URL source span.
/// </summary>
public SourceSpan? UrlSpan;
/// <summary>
/// The title source span.
/// </summary>
public SourceSpan? TitleSpan;
/// <summary>
/// The label span
/// </summary>
public SourceSpan? LabelSpan;
}
}

View File

@@ -11,56 +11,46 @@ namespace Markdig.Syntax
/// </summary>
public static class LinkReferenceDefinitionExtensions
{
private static readonly object DocumentKey = typeof(LinkReferenceDefinition);
private static readonly object DocumentKey = typeof(LinkReferenceDefinitionGroup);
public static bool ContainsLinkReferenceDefinition(this MarkdownDocument document, string label)
{
if (label == null) throw new ArgumentNullException(nameof(label));
var references = document.GetData(DocumentKey) as Dictionary<string, LinkReferenceDefinition>;
var references = document.GetData(DocumentKey) as LinkReferenceDefinitionGroup;
if (references == null)
{
return false;
}
return references.ContainsKey(label);
return references.Links.ContainsKey(label);
}
public static void SetLinkReferenceDefinition(this MarkdownDocument document, string label, LinkReferenceDefinition linkReferenceDefinition)
{
if (label == null) throw new ArgumentNullException(nameof(label));
var references = document.GetLinkReferenceDefinitions();
references[label] = linkReferenceDefinition;
}
public static bool RemoveLinkReferenceDefinition(this MarkdownDocument document, string label)
{
if (label == null) throw new ArgumentNullException(nameof(label));
var references = document.GetData(DocumentKey) as Dictionary<string, LinkReferenceDefinition>;
if (references == null)
{
return false;
}
return references.Remove(label);
references.Set(label, linkReferenceDefinition);
}
public static bool TryGetLinkReferenceDefinition(this MarkdownDocument document, string label, out LinkReferenceDefinition linkReferenceDefinition)
{
if (label == null) throw new ArgumentNullException(nameof(label));
linkReferenceDefinition = null;
var references = document.GetData(DocumentKey) as Dictionary<string, LinkReferenceDefinition>;
var references = document.GetData(DocumentKey) as LinkReferenceDefinitionGroup;
if (references == null)
{
return false;
}
return references.TryGetValue(label, out linkReferenceDefinition);
return references.TryGet(label, out linkReferenceDefinition);
}
public static Dictionary<string, LinkReferenceDefinition> GetLinkReferenceDefinitions(this MarkdownDocument document)
public static LinkReferenceDefinitionGroup GetLinkReferenceDefinitions(this MarkdownDocument document)
{
var references = document.GetData(DocumentKey) as Dictionary<string, LinkReferenceDefinition>;
var references = document.GetData(DocumentKey) as LinkReferenceDefinitionGroup;
if (references == null)
{
references = new Dictionary<string, LinkReferenceDefinition>(StringComparer.OrdinalIgnoreCase);
references = new LinkReferenceDefinitionGroup();
document.SetData(DocumentKey, references);
document.Add(references);
}
return references;
}

View File

@@ -0,0 +1,44 @@
// Copyright (c) Alexandre Mutel. All rights reserved.
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using System;
using System.Collections.Generic;
namespace Markdig.Syntax
{
/// <summary>
/// Contains all the <see cref="LinkReferenceDefinition"/> found in a document.
/// </summary>
/// <seealso cref="Markdig.Syntax.ContainerBlock" />
public class LinkReferenceDefinitionGroup : ContainerBlock
{
/// <summary>
/// Initializes a new instance of the <see cref="LinkReferenceDefinitionGroup"/> class.
/// </summary>
public LinkReferenceDefinitionGroup() : base(null)
{
Links = new Dictionary<string, LinkReferenceDefinition>(StringComparer.OrdinalIgnoreCase);
}
/// <summary>
/// Gets an association between a label and the corresponding <see cref="LinkReferenceDefinition"/>
/// </summary>
public Dictionary<string, LinkReferenceDefinition> Links { get; }
public void Set(string label, LinkReferenceDefinition link)
{
if (link == null) throw new ArgumentNullException(nameof(link));
Links[label] = link;
if (!Contains(link))
{
Add(link);
}
}
public bool TryGet(string label, out LinkReferenceDefinition link)
{
return Links.TryGetValue(label, out link);
}
}
}

View File

@@ -10,6 +10,11 @@ namespace Markdig.Syntax
/// </summary>
public abstract class MarkdownObject : IMarkdownObject
{
protected MarkdownObject()
{
Span = SourceSpan.Empty;
}
/// <summary>
/// The attached datas. Use internally a simple array instead of a Dictionary{Object,Object}
/// as we expect less than 5~10 entries, usually typically 1 (HtmlAttributes)
@@ -29,21 +34,18 @@ namespace Markdig.Syntax
public int Line { get; set; }
/// <summary>
/// Gets or sets the starting character position from the original text source.
/// Note that for inline elements, this is only valid if <see cref="MarkdownExtensions.UsePreciseSourceLocation"/> is setup on the pipeline.
/// The source span
/// </summary>
public int SourceStartPosition { get; set; }
public SourceSpan Span;
/// <summary>
/// Gets or sets the ending character position from the original text source.
/// Note that for inline elements, this is only valid if <see cref="MarkdownExtensions.UsePreciseSourceLocation"/> is setup on the pipeline.
/// Gets a string of the location in the text.
/// </summary>
public int SourceEndPosition { get; set; }
/// <summary>
/// Gets the character length of this element within the original source code.
/// </summary>
public int SourceLength => SourceEndPosition - SourceStartPosition + 1;
/// <returns></returns>
public string ToPositionText()
{
return $"${Line}, {Column}, {Span.Start}-{Span.End}";
}
/// <summary>
/// Stores a key/value pair for this instance.

View File

@@ -0,0 +1,80 @@
// Copyright (c) Alexandre Mutel. All rights reserved.
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using System;
namespace Markdig.Syntax
{
/// <summary>
/// A span of text.
/// </summary>
public struct SourceSpan : IEquatable<SourceSpan>
{
public static readonly SourceSpan Empty = new SourceSpan(0, -1);
/// <summary>
/// Initializes a new instance of the <see cref="SourceSpan"/> struct.
/// </summary>
/// <param name="start">The start.</param>
/// <param name="end">The end.</param>
public SourceSpan(int start, int end)
{
Start = start;
End = end;
}
/// <summary>
/// Gets or sets the starting character position from the original text source.
/// Note that for inline elements, this is only valid if <see cref="MarkdownExtensions.UsePreciseSourceLocation"/> is setup on the pipeline.
/// </summary>
public int Start { get; set; }
/// <summary>
/// Gets or sets the ending character position from the original text source.
/// Note that for inline elements, this is only valid if <see cref="MarkdownExtensions.UsePreciseSourceLocation"/> is setup on the pipeline.
/// </summary>
public int End { get; set; }
/// <summary>
/// Gets the character length of this element within the original source code.
/// </summary>
public int Length => End - Start + 1;
public bool IsEmpty => Start > End;
public bool Equals(SourceSpan other)
{
return Start == other.Start && End == other.End;
}
public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
return obj is SourceSpan && Equals((SourceSpan) obj);
}
public override int GetHashCode()
{
unchecked
{
return (Start*397) ^ End;
}
}
public static bool operator ==(SourceSpan left, SourceSpan right)
{
return left.Equals(right);
}
public static bool operator !=(SourceSpan left, SourceSpan right)
{
return !left.Equals(right);
}
public override string ToString()
{
return $"{Start}-{End}";
}
}
}

View File

@@ -1,6 +1,6 @@
{
"title": "Markdig",
"version": "0.5.1",
"version": "0.5.4",
"authors": [ "Alexandre Mutel" ],
"description": "A fast, powerfull, CommonMark compliant, extensible Markdown processor for .NET",
"copyright": "Alexandre Mutel",
@@ -11,7 +11,7 @@
"projectUrl": "https://github.com/lunet-io/markdig",
"iconUrl": "https://raw.githubusercontent.com/lunet-io/markdig/master/img/markdig.png",
"requireLicenseAcceptance": false,
"releaseNotes": "Add support for task lists.\nAdd support for precise source location pipelineBuilder.UsePreciseSourceLocation.\nBreaking change: Markdown.ToHtml() accept now only a string instead of a TextReader as this improve the overall performance.",
"releaseNotes": "Fix bug in HTML block parsing which could break parsing of remaining document",
"tags": [ "Markdown CommonMark md html md2html" ]
},
"configurations": {