Compare commits

...

30 Commits

Author SHA1 Message Date
Alexandre Mutel
1d7bb021a7 Bump version to 0.10.7 2017-04-01 10:20:18 +02:00
Alexandre Mutel
b595383281 Merge pull request #103 from Daniel15/replace
Add Replace method to replace an element in an OrderedList
2017-03-20 10:39:01 +01:00
Alexandre Mutel
bf85964543 Merge pull request #102 from Daniel15/lock-dotnet
Lock .NET Core SDK to 1.0.0-preview2
2017-03-20 10:37:16 +01:00
Daniel Lo Nigro
d5b020b784 Add Replace method to replace an element in an OrderedList 2017-03-19 14:39:24 -07:00
Daniel Lo Nigro
8ab0467e78 Lock .NET Core SDK to 1.0.0-preview2 2017-03-19 13:42:26 -07:00
Alexandre Mutel
74fe3be7c7 Bump version to 0.10.6 2017-02-24 07:52:29 +01:00
Alexandre Mutel
b47ae8d9ba Fix emphasis with HTML entities (#97) 2017-02-24 07:49:46 +01:00
Alexandre Mutel
a926e1a326 Bump version to 0.10.5 2017-02-14 17:36:54 +01:00
Alexandre Mutel
6048406f52 Merge branch 'pr/n89_johnsimons' 2017-01-31 13:01:17 +01:00
John Simons
1edf14c742 Remove requirement for space after emoji.
Fixes #88
2017-01-31 12:58:32 +01:00
Alexandre Mutel
d4bb7c0a51 Merge pull request #82 from christophano/bugfix/abbreviation_parser_fix
Fixes the IsValidAbbreviation method in AbbreviationParser
2016-12-19 14:37:32 +01:00
Chris Rodgers
ba4e463517 Adds the test I ought to have included to begin with. 2016-12-19 13:25:56 +00:00
Chris Rodgers
2d0dae4ebb Fixes the IsValidAbbreviation method in AbbreviationParser 2016-12-19 13:05:03 +00:00
Alexandre Mutel
1a9d923c44 Merge pull request #81 from volkanceylan/master
Use Char.ToUpperInvariant instead of Char.ToUpper
2016-12-18 22:20:20 +01:00
volkan.ceylan
013ec0bf80 Due to use of Char.ToUpper, ListExtraItemParser fails in cultures like Turkish where uppercase of 'i' is 'İ'. Char.ToUpperInvariant should be used instead. 2016-12-18 11:47:25 +03:00
Alexandre Mutel
f82e874cb7 Merge pull request #80 from christophano/feature/pipeline_builder_use_method
Adds fluent extension registration method to MarkdownPipelineBuilder.
2016-12-15 11:49:32 +01:00
Chris Rodgers
976fc569fa Moves new Use method to MarkdownExtensions and changes it to extension method. 2016-12-15 10:31:02 +00:00
Chris Rodgers
5aab21f0bf Adds fluent extension registration method to MarkdownPipelineBuilder. 2016-12-15 10:07:54 +00:00
Alexandre Mutel
1df8344576 Bump to 0.10.4 2016-12-12 11:03:48 +01:00
Alexandre Mutel
9eb84cf95e Normalize pipe tables so they contain the same number of columns for all rows 2016-12-11 19:01:49 +01:00
Alexandre Mutel
47d45b5577 Fix issue when parsing autolinks embraced by a pending emphasis (issue #78) 2016-12-11 18:08:51 +01:00
Alexandre Mutel
ebcc286df0 Bump version to 0.10.3 2016-11-29 18:02:39 +01:00
Alexandre Mutel
f11d8037ac Fix issue when the last character of a table is not a newline and there are spaces in-between (issue #73) 2016-11-29 17:43:28 +01:00
Alexandre Mutel
f668b3fe38 Bump version to 0.10.2 2016-11-26 11:20:49 +01:00
Alexandre Mutel
5ca4e31332 Fix exception when trying to Urilize an url with an unicode character outside the supported range by string.Normalize + NormD (issue #75) 2016-11-26 11:20:37 +01:00
Alexandre Mutel
a8fbe79e30 Bump to 0.10.1 2016-11-19 16:53:56 +01:00
Alexandre Mutel
bdb09b9820 Update to latest CommonMark specs 0.27 2016-11-19 16:53:44 +01:00
Alexandre Mutel
79348ebea8 Add span to LinkReferenceDefinition (issue #51) 2016-10-27 17:06:37 +02:00
Alexandre Mutel
b366c1a5e3 Bump to 0.10.0 2016-10-16 21:30:55 +02:00
Alexandre Mutel
a5e53b014f Add better support to discover active extensions when setup renderer (issue #66) 2016-10-16 21:30:38 +02:00
61 changed files with 2038 additions and 111 deletions

View File

@@ -14,7 +14,7 @@ You can **try Markdig online** and compare it to other implementations on [babel
- **Abstract Syntax Tree** with precise source code location for syntax tree, useful when building a Markdown editor.
- Checkout [MarkdownEditor for Visual Studio](https://visualstudiogallery.msdn.microsoft.com/eaab33c3-437b-4918-8354-872dfe5d1bfe) powered by Markdig!
- Converter to **HTML**
- Passing more than **600+ tests** from the latest [CommonMark specs](http://spec.commonmark.org/)
- Passing more than **600+ tests** from the latest [CommonMark specs (0.27)](http://spec.commonmark.org/)
- Includes all the core elements of CommonMark:
- including **GFM fenced code blocks**.
- **Extensible** architecture

View File

@@ -4,6 +4,6 @@
<NuGetPackageRoot>$(UserProfile)\.nuget\packages\</NuGetPackageRoot>
</PropertyGroup>
<ImportGroup>
<Import Project="$(NuGetPackageRoot)\Microsoft.Diagnostics.Tracing.TraceEvent\1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets" Condition="Exists('$(NuGetPackageRoot)\Microsoft.Diagnostics.Tracing.TraceEvent\1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets')" />
<Import Project="$(NuGetPackageRoot)microsoft.diagnostics.tracing.traceevent\1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets" Condition="Exists('$(NuGetPackageRoot)microsoft.diagnostics.tracing.traceevent\1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets')" />
</ImportGroup>
</Project>

View File

@@ -65,6 +65,7 @@
<Compile Include="TestHtmlHelper.cs" />
<Compile Include="TestLineReader.cs" />
<Compile Include="TestLinkHelper.cs" />
<Compile Include="TestOrderedList.cs" />
<Compile Include="TestPragmaLines.cs" />
<Compile Include="TestSourcePosition.cs" />
<Compile Include="TestStringSliceList.cs" />

View File

@@ -67,3 +67,13 @@ We should not abbreviate 1.1A or 11A!
.
<p>We should not abbreviate 1.1A or 11A!</p>
````````````````````````````````
Abbreviations should match whole word only, even if the word is the entire content:
```````````````````````````````` example
*[1A]: First
1.1A
.
<p>1.1A</p>
````````````````````````````````

View File

@@ -68,3 +68,11 @@ This is an HTML [http://www.google.com](http://www.google.com) link
.
<p>This is an HTML <a href="http://www.google.com">http://www.google.com</a> link</p>
````````````````````````````````
A link embraced by pending emphasis should let the emphasis takes precedence if characters are placed at the end of the matched link:
```````````````````````````````` example
Check **http://www.a.com** or __http://www.b.com__
.
<p>Check <strong><a href="http://www.a.com">http://www.a.com</a></strong> or <strong><a href="http://www.b.com">http://www.b.com</a></strong></p>
````````````````````````````````

View File

@@ -12,10 +12,28 @@ This is a test with a :) and a :angry: smiley
<p>This is a test with a 😃 and a 😠 smiley</p>
````````````````````````````````
An emoji needs to be preceded by a space and followed by a space:
An emoji needs to be preceded by a space:
```````````````````````````````` example
These are not:) an :)emoji with a:) x:angry:x
These are not:) an emoji with a:) x:angry:x
.
<p>These are not:) an :)emoji with a:) x:angry:x</p>
<p>These are not:) an emoji with a:) x:angry:x</p>
````````````````````````````````
Emoji can be followed by close ponctuation (or any other characters):
```````````````````````````````` example
We all need :), it makes us :muscle:. (and :ok_hand:).
.
<p>We all need 😃, it makes us 💪. (and 👌).</p>
````````````````````````````````
Sentences can end with Emoji:
```````````````````````````````` example
This is a sentance :ok_hand:
and keeps going to the next line :)
.
<p>This is a sentance 👌
and keeps going to the next line 😃</p>
````````````````````````````````

View File

@@ -52,3 +52,16 @@ Marked text can be used to specify that a text has been marked in a document. T
.
<p><mark>Marked text</mark></p>
````````````````````````````````
## Emphasis on Html Entities
```````````````````````````````` example
This is text MyBrand ^&reg;^ and MyTrademark ^&trade;^
This is text MyBrand^&reg;^ and MyTrademark^&trade;^
This is text MyBrand~&reg;~ and MyCopyright^&copy;^
.
<p>This is text MyBrand <sup>®</sup> and MyTrademark <sup>TM</sup>
This is text MyBrand<sup>®</sup> and MyTrademark<sup>TM</sup>
This is text MyBrand<sub>®</sub> and MyCopyright<sup>©</sup></p>
````````````````````````````````

View File

@@ -136,6 +136,7 @@ a | b
<tr>
<th>a</th>
<th>b</th>
<th></th>
</tr>
</thead>
<tbody>
@@ -147,9 +148,12 @@ a | b
<tr>
<td>3</td>
<td>4</td>
<td></td>
</tr>
<tr>
<td>5</td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
@@ -548,3 +552,31 @@ Tests trailing spaces after pipes
</tbody>
</table>
````````````````````````````````
** Normalized columns count **
The tables are normalized to the maximum number of columns found in a table
```````````````````````````````` example
a | b
-- | -
0 | 1 | 2
.
<table>
<thead>
<tr>
<th>a</th>
<th>b</th>
<th></th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
<td>2</td>
</tr>
</tbody>
</table>
````````````````````````````````

View File

@@ -6,8 +6,8 @@ namespace Markdig.Tests
// ---
// title: CommonMark Spec
// author: John MacFarlane
// version: 0.26
// date: '2016-07-15'
// version: 0.27
// date: '2016-11-18'
// license: '[CC-BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/)'
// ...
//
@@ -6036,11 +6036,11 @@ namespace Markdig.Tests
// If the list item is ordered, then it is also assigned a start
// number, based on the ordered list marker.
//
// Exceptions: When the list item interrupts a paragraph---that
// is, when it starts on a line that would otherwise count as
// [paragraph continuation text]---then (a) the lines *Ls* must
// not begin with a blank line, and (b) if the list item is
// ordered, the start number must be 1.
// Exceptions: When the first list item in a [list] interrupts
// a paragraph---that is, when it starts on a line that would
// otherwise count as [paragraph continuation text]---then (a)
// the lines *Ls* must not begin with a blank line, and (b) if
// the list item is ordered, the start number must be 1.
//
// For example, let *Ls* be the lines
[TestFixture]
@@ -16633,6 +16633,7 @@ namespace Markdig.Tests
// <tr>
// <th>a</th>
// <th>b</th>
// <th></th>
// </tr>
// </thead>
// <tbody>
@@ -16644,15 +16645,18 @@ namespace Markdig.Tests
// <tr>
// <td>3</td>
// <td>4</td>
// <td></td>
// </tr>
// <tr>
// <td>5</td>
// <td></td>
// <td></td>
// </tr>
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 7, "Extensions Pipe Table");
TestParser.TestSpec("a | b \n-- | --\n0 | 1 | 2\n3 | 4\n5 |", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n<td>2</td>\n</tr>\n<tr>\n<td>3</td>\n<td>4</td>\n</tr>\n<tr>\n<td>5</td>\n</tr>\n</tbody>\n</table>", "pipetables|advanced");
TestParser.TestSpec("a | b \n-- | --\n0 | 1 | 2\n3 | 4\n5 |", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n<th></th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n<td>2</td>\n</tr>\n<tr>\n<td>3</td>\n<td>4</td>\n<td></td>\n</tr>\n<tr>\n<td>5</td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>", "pipetables|advanced");
}
}
// **Rule #2**
@@ -17227,6 +17231,45 @@ namespace Markdig.Tests
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 22, "Extensions Pipe Table");
TestParser.TestSpec("| abc | def | \n|---|---|\n| cde| ddd| \n| eee| fff|\n| fff | fffff | \n|gggg | ffff | ", "<table>\n<thead>\n<tr>\n<th>abc</th>\n<th>def</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>cde</td>\n<td>ddd</td>\n</tr>\n<tr>\n<td>eee</td>\n<td>fff</td>\n</tr>\n<tr>\n<td>fff</td>\n<td>fffff</td>\n</tr>\n<tr>\n<td>gggg</td>\n<td>ffff</td>\n</tr>\n</tbody>\n</table>", "pipetables|advanced");
}
}
// ** Normalized columns count **
//
// The tables are normalized to the maximum number of columns found in a table
[TestFixture]
public partial class TestExtensionsPipeTable
{
[Test]
public void Example023()
{
// Example 23
// Section: Extensions Pipe Table
//
// The following CommonMark:
// a | b
// -- | -
// 0 | 1 | 2
//
// Should be rendered as:
// <table>
// <thead>
// <tr>
// <th>a</th>
// <th>b</th>
// <th></th>
// </tr>
// </thead>
// <tbody>
// <tr>
// <td>0</td>
// <td>1</td>
// <td>2</td>
// </tr>
// </tbody>
// </table>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 23, "Extensions Pipe Table");
TestParser.TestSpec("a | b\n-- | - \n0 | 1 | 2", "<table>\n<thead>\n<tr>\n<th>a</th>\n<th>b</th>\n<th></th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0</td>\n<td>1</td>\n<td>2</td>\n</tr>\n</tbody>\n</table>", "pipetables|advanced");
}
}
// # Extensions
//
@@ -17573,6 +17616,30 @@ namespace Markdig.Tests
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 5, "Extensions Marked");
TestParser.TestSpec("==Marked text==", "<p><mark>Marked text</mark></p>", "emphasisextras|advanced");
}
}
// ## Emphasis on Html Entities
[TestFixture]
public partial class TestExtensionsEmphasisonHtmlEntities
{
[Test]
public void Example006()
{
// Example 6
// Section: Extensions Emphasis on Html Entities
//
// The following CommonMark:
// This is text MyBrand ^&reg;^ and MyTrademark ^&trade;^
// This is text MyBrand^&reg;^ and MyTrademark^&trade;^
// This is text MyBrand~&reg;~ and MyCopyright^&copy;^
//
// Should be rendered as:
// <p>This is text MyBrand <sup>®</sup> and MyTrademark <sup>TM</sup>
// This is text MyBrand<sup>®</sup> and MyTrademark<sup>TM</sup>
// This is text MyBrand<sub>®</sub> and MyCopyright<sup>©</sup></p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 6, "Extensions Emphasis on Html Entities");
TestParser.TestSpec("This is text MyBrand ^&reg;^ and MyTrademark ^&trade;^\nThis is text MyBrand^&reg;^ and MyTrademark^&trade;^\nThis is text MyBrand~&reg;~ and MyCopyright^&copy;^", "<p>This is text MyBrand <sup>®</sup> and MyTrademark <sup>TM</sup>\nThis is text MyBrand<sup>®</sup> and MyTrademark<sup>TM</sup>\nThis is text MyBrand<sub>®</sub> and MyCopyright<sup>©</sup></p>", "emphasisextras|advanced");
}
}
// # Extensions
//
@@ -18467,7 +18534,7 @@ namespace Markdig.Tests
TestParser.TestSpec("This is a test with a :) and a :angry: smiley", "<p>This is a test with a 😃 and a 😠 smiley</p>", "emojis|advanced+emojis");
}
}
// An emoji needs to be preceded by a space and followed by a space:
// An emoji needs to be preceded by a space:
[TestFixture]
public partial class TestExtensionsEmoji
{
@@ -18478,13 +18545,55 @@ namespace Markdig.Tests
// Section: Extensions Emoji
//
// The following CommonMark:
// These are not:) an :)emoji with a:) x:angry:x
// These are not:) an emoji with a:) x:angry:x
//
// Should be rendered as:
// <p>These are not:) an :)emoji with a:) x:angry:x</p>
// <p>These are not:) an emoji with a:) x:angry:x</p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 2, "Extensions Emoji");
TestParser.TestSpec("These are not:) an :)emoji with a:) x:angry:x", "<p>These are not:) an :)emoji with a:) x:angry:x</p>", "emojis|advanced+emojis");
TestParser.TestSpec("These are not:) an emoji with a:) x:angry:x", "<p>These are not:) an emoji with a:) x:angry:x</p>", "emojis|advanced+emojis");
}
}
// Emoji can be followed by close ponctuation (or any other characters):
[TestFixture]
public partial class TestExtensionsEmoji
{
[Test]
public void Example003()
{
// Example 3
// Section: Extensions Emoji
//
// The following CommonMark:
// We all need :), it makes us :muscle:. (and :ok_hand:).
//
// Should be rendered as:
// <p>We all need 😃, it makes us 💪. (and 👌).</p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 3, "Extensions Emoji");
TestParser.TestSpec("We all need :), it makes us :muscle:. (and :ok_hand:).", "<p>We all need 😃, it makes us 💪. (and 👌).</p>", "emojis|advanced+emojis");
}
}
// Sentences can end with Emoji:
[TestFixture]
public partial class TestExtensionsEmoji
{
[Test]
public void Example004()
{
// Example 4
// Section: Extensions Emoji
//
// The following CommonMark:
// This is a sentance :ok_hand:
// and keeps going to the next line :)
//
// Should be rendered as:
// <p>This is a sentance 👌
// and keeps going to the next line 😃</p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 4, "Extensions Emoji");
TestParser.TestSpec("This is a sentance :ok_hand:\nand keeps going to the next line :)", "<p>This is a sentance 👌\nand keeps going to the next line 😃</p>", "emojis|advanced+emojis");
}
}
// # Extensions
@@ -18628,6 +18737,28 @@ namespace Markdig.Tests
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 6, "Extensions Abbreviation");
TestParser.TestSpec("*[1A]: First\n\nWe should not abbreviate 1.1A or 11A!", "<p>We should not abbreviate 1.1A or 11A!</p>", "abbreviations|advanced");
}
}
// Abbreviations should match whole word only, even if the word is the entire content:
[TestFixture]
public partial class TestExtensionsAbbreviation
{
[Test]
public void Example007()
{
// Example 7
// Section: Extensions Abbreviation
//
// The following CommonMark:
// *[1A]: First
//
// 1.1A
//
// Should be rendered as:
// <p>1.1A</p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 7, "Extensions Abbreviation");
TestParser.TestSpec("*[1A]: First\n\n1.1A", "<p>1.1A</p>", "abbreviations|advanced");
}
}
// # Extensions
//
@@ -20258,4 +20389,24 @@ namespace Markdig.Tests
TestParser.TestSpec("This is an HTML [http://www.google.com](http://www.google.com) link", "<p>This is an HTML <a href=\"http://www.google.com\">http://www.google.com</a> link</p>", "autolinks|advanced");
}
}
// A link embraced by pending emphasis should let the emphasis takes precedence if characters are placed at the end of the matched link:
[TestFixture]
public partial class TestExtensionsAutoLinks
{
[Test]
public void Example007()
{
// Example 7
// Section: Extensions AutoLinks
//
// The following CommonMark:
// Check **http://www.a.com** or __http://www.b.com__
//
// Should be rendered as:
// <p>Check <strong><a href="http://www.a.com">http://www.a.com</a></strong> or <strong><a href="http://www.b.com">http://www.b.com</a></strong></p>
Console.WriteLine("Example {0}" + Environment.NewLine + "Section: {0}" + Environment.NewLine, 7, "Extensions AutoLinks");
TestParser.TestSpec("Check **http://www.a.com** or __http://www.b.com__", "<p>Check <strong><a href=\"http://www.a.com\">http://www.a.com</a></strong> or <strong><a href=\"http://www.b.com\">http://www.b.com</a></strong></p>", "autolinks|advanced");
}
}
}

View File

@@ -38,8 +38,7 @@ SOFTWARE.
<#@ import namespace="System.CodeDom.Compiler" #>
<#@ output extension=".cs" #><#
var specFiles = new KeyValuePair<string, string>[] {
// new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/master/spec.txt", string.Empty),
new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/cfc84164475d3bec8be9482c21a705adc93a54f5/spec.txt", string.Empty), // 0.26 specs
new KeyValuePair<string, string>("https://raw.githubusercontent.com/jgm/CommonMark/791b1c121f16d3d7e80837c6f52917e57bbb2f61/spec.txt", string.Empty), // 0.27
new KeyValuePair<string, string>(Host.ResolvePath("PipeTableSpecs.md"), "pipetables|advanced"),
new KeyValuePair<string, string>(Host.ResolvePath("FootnotesSpecs.md"), "footnotes|advanced"),
new KeyValuePair<string, string>(Host.ResolvePath("GenericAttributesSpecs.md"), "attributes|advanced"),

View File

@@ -390,6 +390,7 @@ namespace Markdig.Tests
[TestCase("b>r", "br")]
[TestCase(@"b\r", "br")]
[TestCase(@"b""r", "br")]
[TestCase(@"Requirement 😀", "requirement")]
public void TestUrilizeNonAscii_NonValidCharactersForFragments(string input, string expectedResult)
{
Assert.AreEqual(expectedResult, LinkHelper.Urilize(input, false));

View File

@@ -0,0 +1,43 @@
// Copyright (c) Alexandre Mutel. All rights reserved.
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using Markdig.Helpers;
using NUnit.Framework;
namespace Markdig.Tests
{
[TestFixture]
public class TestOrderedList
{
[Test]
public void TestReplace()
{
var list = new OrderedList<ITest>
{
new A(),
new B(),
new C(),
};
// Replacing B with D. Order should now be A, D, B.
var result = list.Replace<B>(new D());
Assert.That(result, Is.True);
Assert.That(list.Count, Is.EqualTo(3));
Assert.That(list[0], Is.InstanceOf<A>());
Assert.That(list[1], Is.InstanceOf<D>());
Assert.That(list[2], Is.InstanceOf<C>());
// Replacing B again should fail, as it's no longer in the list.
Assert.That(list.Replace<B>(new D()), Is.False);
}
#region Test fixtures
private interface ITest { }
private class A : ITest { }
private class B : ITest { }
private class C : ITest { }
private class D : ITest { }
#endregion
}
}

View File

@@ -118,7 +118,7 @@ blabla
[Test]
public void TestBugAdvancaed()
public void TestBugAdvanced()
{
TestParser.TestSpec(@"`https://{domain}/callbacks`
#### HEADING
@@ -126,6 +126,30 @@ Paragraph
", "<p><code>https://{domain}/callbacks</code></p>\n<h4 id=\"heading\">HEADING</h4>\n<p>Paragraph</p>", "advanced");
}
[Test]
public void TestBugPipeTables()
{
// https://github.com/lunet-io/markdig/issues/73
TestParser.TestSpec(@"| abc | def |
| --- | --- |
| 1 | ~3 |
", @"<table>
<thead>
<tr>
<th>abc</th>
<th>def</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>~3</td>
</tr>
</tbody>
</table>", "advanced");
}
[Test]
public void TestSamePipelineAllExtensions()
{

View File

@@ -17,7 +17,7 @@ namespace Markdig.Extensions.Abbreviations
pipeline.BlockParsers.AddIfNotAlready<AbbreviationParser>();
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null && !htmlRenderer.ObjectRenderers.Contains<HtmlAbbreviationRenderer>())

View File

@@ -190,7 +190,7 @@ namespace Markdig.Extensions.Abbreviations
{
// The word matched must be embraced by punctuation or whitespace or \0.
var index = matchIndex - 1;
while (index > content.Start)
while (index >= content.Start)
{
var c = content.PeekCharAbsolute(index);
if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))
@@ -204,7 +204,7 @@ namespace Markdig.Extensions.Abbreviations
index--;
}
index = matchIndex + match.Length;
while (index < content.End)
while (index <= content.End)
{
var c = content.PeekCharAbsolute(index);
if (!(c == '\0' || c.IsAsciiPunctuation() || c.IsWhitespace()))

View File

@@ -59,7 +59,7 @@ namespace Markdig.Extensions.AutoIdentifiers
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}

View File

@@ -22,7 +22,7 @@ namespace Markdig.Extensions.AutoLinks
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}
}

View File

@@ -3,6 +3,7 @@
// See the license.txt file in the project root for more information.
using System;
using System.Collections.Generic;
using Markdig.Helpers;
using Markdig.Parsers;
using Markdig.Syntax.Inlines;
@@ -38,8 +39,9 @@ namespace Markdig.Extensions.AutoLinks
return false;
}
List<char> pendingEmphasis;
// Check that an autolink is possible in the current context
if (!IsAutoLinkValidInCurrentContext(processor))
if (!IsAutoLinkValidInCurrentContext(processor, out pendingEmphasis))
{
return false;
}
@@ -84,6 +86,27 @@ namespace Markdig.Extensions.AutoLinks
return false;
}
// If we have any pending emphasis, remove any pending emphasis characters from the end of the link
if (pendingEmphasis != null)
{
for (int i = link.Length - 1; i >= 0; i--)
{
if (pendingEmphasis.Contains(link[i]))
{
slice.Start--;
}
else
{
if (i < link.Length - 1)
{
link = link.Substring(0, i + 1);
}
break;
}
}
}
// Post-check URL
switch (c)
{
@@ -144,8 +167,10 @@ namespace Markdig.Extensions.AutoLinks
return true;
}
private bool IsAutoLinkValidInCurrentContext(InlineProcessor processor)
private bool IsAutoLinkValidInCurrentContext(InlineProcessor processor, out List<char> pendingEmphasis)
{
pendingEmphasis = null;
// Case where there is a pending HtmlInline <a>
var currentInline = processor.Inline;
while (currentInline != null)
@@ -188,6 +213,23 @@ namespace Markdig.Extensions.AutoLinks
countBrackets--;
}
}
else
{
// Record all pending characters for emphasis
var emphasisDelimiter = currentInline as EmphasisDelimiterInline;
if (emphasisDelimiter != null)
{
if (pendingEmphasis == null)
{
// Not optimized for GC, but we don't expect this case much
pendingEmphasis = new List<char>();
}
if (!pendingEmphasis.Contains(emphasisDelimiter.DelimiterChar))
{
pendingEmphasis.Add(emphasisDelimiter.DelimiterChar);
}
}
}
currentInline = currentInline.Parent;
}

View File

@@ -22,7 +22,7 @@ namespace Markdig.Extensions.Bootstrap
pipeline.DocumentProcessed += PipelineOnDocumentProcessed;
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}

View File

@@ -24,7 +24,7 @@ namespace Markdig.Extensions.Citations
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -38,7 +38,7 @@ namespace Markdig.Extensions.CustomContainers
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -20,7 +20,7 @@ namespace Markdig.Extensions.DefinitionLists
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -17,7 +17,7 @@ namespace Markdig.Extensions.Diagrams
{
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -21,7 +21,7 @@ namespace Markdig.Extensions.Emoji
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}
}

View File

@@ -81,12 +81,6 @@ namespace Markdig.Extensions.Emoji
return false;
}
// Following char must be a space
if (!slice.PeekCharExtra(match.Length).IsWhiteSpaceOrZero())
{
return false;
}
// If we have a smiley, we decode it to emoji
string emoji;
if (!SmileyToEmoji.TryGetValue(match, out emoji))

View File

@@ -85,7 +85,7 @@ namespace Markdig.Extensions.EmphasisExtras
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -29,7 +29,7 @@ namespace Markdig.Extensions.Figures
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -29,7 +29,7 @@ namespace Markdig.Extensions.Footers
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -20,7 +20,7 @@ namespace Markdig.Extensions.Footnotes
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -37,7 +37,7 @@ namespace Markdig.Extensions.GenericAttributes
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}

View File

@@ -24,7 +24,7 @@ namespace Markdig.Extensions.Hardlines
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}
}

View File

@@ -22,7 +22,7 @@ namespace Markdig.Extensions.ListExtras
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}
}

View File

@@ -62,7 +62,7 @@ namespace Markdig.Extensions.ListExtras
{
// otherwise we expect a regular alpha lettered list with a single character.
var isUpper = c.IsAlphaUpper();
result.OrderedStart = (Char.ToUpper(c) - 64).ToString();
result.OrderedStart = (Char.ToUpperInvariant(c) - 64).ToString();
result.BulletType = isUpper ? 'A' : 'a';
result.DefaultOrderedStart = isUpper ? "A" : "a";
state.NextChar();

View File

@@ -29,7 +29,7 @@ namespace Markdig.Extensions.Mathematics
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -31,7 +31,7 @@ namespace Markdig.Extensions.MediaLinks
{
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -16,7 +16,7 @@ namespace Markdig.Extensions.NoRefLinks
{
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var linkRenderer = renderer.ObjectRenderers.Find<LinkInlineRenderer>();
if (linkRenderer != null)

View File

@@ -15,7 +15,7 @@ namespace Markdig.Extensions.NonAsciiNoEscape
{
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -23,7 +23,7 @@ namespace Markdig.Extensions.PragmaLines
pipeline.DocumentProcessed += PipelineOnDocumentProcessed;
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}

View File

@@ -62,7 +62,7 @@ namespace Markdig.Extensions.SelfPipeline
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
}

View File

@@ -35,7 +35,7 @@ namespace Markdig.Extensions.SmartyPants
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -19,7 +19,7 @@ namespace Markdig.Extensions.Tables
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null && !htmlRenderer.ObjectRenderers.Contains<HtmlTableRenderer>())

View File

@@ -41,7 +41,7 @@ namespace Markdig.Extensions.Tables
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null && !htmlRenderer.ObjectRenderers.Contains<HtmlTableRenderer>())

View File

@@ -274,7 +274,16 @@ namespace Markdig.Extensions.Tables
}
var endOfTable = new LineBreakInline();
lastElement.InsertAfter(endOfTable);
// If the last element is a container, we have to add the EOL to its child
// otherwise only next sibling
if (lastElement is ContainerInline)
{
((ContainerInline)lastElement).AppendChild(endOfTable);
}
else
{
lastElement.InsertAfter(endOfTable);
}
delimiters.Add(endOfTable);
tableState.EndOfLines.Add(endOfTable);
}
@@ -449,6 +458,9 @@ namespace Markdig.Extensions.Tables
// Clear cells when we are done
cells.Clear();
// Normalize the table
table.Normalize();
// We don't want to continue procesing delimiters, as we are already processing them here
return false;
}

View File

@@ -69,5 +69,33 @@ namespace Markdig.Extensions.Tables
}
return true;
}
/// <summary>
/// Normalizes the number of columns of this table by taking the maximum columns and appending empty cells.
/// </summary>
public void Normalize()
{
var maxColumn = 0;
for (int i = 0; i < this.Count; i++)
{
var row = this[i] as TableRow;
if (row != null && row.Count > maxColumn)
{
maxColumn = row.Count;
}
}
for (int i = 0; i < this.Count; i++)
{
var row = this[i] as TableRow;
if (row != null)
{
for (int j = row.Count; j < maxColumn; j++)
{
row.Add(new TableCell());
}
}
}
}
}
}

View File

@@ -21,7 +21,7 @@ namespace Markdig.Extensions.TaskLists
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
var htmlRenderer = renderer as HtmlRenderer;
if (htmlRenderer != null)

View File

@@ -22,7 +22,7 @@ namespace Markdig.Extensions.Yaml
}
}
public void Setup(IMarkdownRenderer renderer)
public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
{
if (!renderer.ObjectRenderers.Contains<YamlFrontMatterRenderer>())
{

View File

@@ -96,9 +96,9 @@ namespace Markdig.Helpers
int result = 0;
for (int i = 0; i < text.Length; i++)
{
var character = Char.ToUpper(text[i]);
var character = Char.ToUpperInvariant(text[i]);
var candidate = romanMap[character];
if (i + 1 < text.Length && candidate < romanMap[Char.ToUpper(text[i + 1])])
if (i + 1 < text.Length && candidate < romanMap[Char.ToUpperInvariant(text[i + 1])])
{
result -= candidate;
}
@@ -162,6 +162,9 @@ namespace Markdig.Helpers
return IsWhitespace(c) || IsZero(c);
}
// Note that we are not considering the character & as a punctuation in HTML
// as it is used for HTML entities, print unicode, so we assume that when we have a `&`
// it is more likely followed by a valid HTML Entity that represents a non punctuation
public static void CheckUnicodeCategory(this char c, out bool space, out bool punctuation)
{
// Credits: code from CommonMark.NET
@@ -170,7 +173,7 @@ namespace Markdig.Helpers
if (c <= 'ÿ')
{
space = c == '\0' || c == ' ' || (c >= '\t' && c <= '\r') || c == '\u00a0' || c == '\u0085';
punctuation = c == '\0' || (c >= 33 && c <= 47) || (c >= 58 && c <= 64) || (c >= 91 && c <= 96) || (c >= 123 && c <= 126);
punctuation = c == '\0' || (c >= 33 && c <= 47 && c != 38) || (c >= 58 && c <= 64) || (c >= 91 && c <= 96) || (c >= 123 && c <= 126);
}
else
{

File diff suppressed because it is too large Load Diff

View File

@@ -20,60 +20,59 @@ namespace Markdig.Helpers
public static string Urilize(string headingText, bool allowOnlyAscii)
{
#if SUPPORT_NORMALIZE
// Normalzie the string if we don't allow UTF8
if (allowOnlyAscii)
{
headingText = headingText.Normalize(NormalizationForm.FormD);
}
#endif
var headingBuffer = StringBuilderCache.Local();
bool hasLetter = false;
bool previousIsSpace = false;
for (int i = 0; i < headingText.Length; i++)
{
var c = headingText[i];
if (char.IsLetter(c))
var normalized = allowOnlyAscii ? CharNormalizer.ConvertToAscii(c) : null;
for (int j = 0; j < (normalized?.Length ?? 1); j++)
{
#if SUPPORT_NORMALIZE
if (allowOnlyAscii && (c < ' ' || c >= 127))
if (normalized != null)
{
continue;
c = normalized[j];
}
#endif
c = char.IsUpper(c) ? char.ToLowerInvariant(c) : c;
headingBuffer.Append(c);
hasLetter = true;
previousIsSpace = false;
}
else if (hasLetter)
{
if (IsReservedPunctuation(c))
if (char.IsLetter(c))
{
if (previousIsSpace)
if (allowOnlyAscii && (c < ' ' || c >= 127))
{
headingBuffer.Length--;
continue;
}
if (headingBuffer[headingBuffer.Length - 1] != c)
c = char.IsUpper(c) ? char.ToLowerInvariant(c) : c;
headingBuffer.Append(c);
hasLetter = true;
previousIsSpace = false;
}
else if (hasLetter)
{
if (IsReservedPunctuation(c))
{
if (previousIsSpace)
{
headingBuffer.Length--;
}
if (headingBuffer[headingBuffer.Length - 1] != c)
{
headingBuffer.Append(c);
}
previousIsSpace = false;
}
else if (c.IsDigit())
{
headingBuffer.Append(c);
previousIsSpace = false;
}
previousIsSpace = false;
}
else if (c.IsDigit())
{
headingBuffer.Append(c);
previousIsSpace = false;
}
else if (!previousIsSpace && c.IsWhitespace())
{
var pc = headingBuffer[headingBuffer.Length - 1];
if (!IsReservedPunctuation(pc))
else if (!previousIsSpace && c.IsWhitespace())
{
headingBuffer.Append('-');
var pc = headingBuffer[headingBuffer.Length - 1];
if (!IsReservedPunctuation(pc))
{
headingBuffer.Append('-');
}
previousIsSpace = true;
}
previousIsSpace = true;
}
}
}

View File

@@ -101,5 +101,25 @@ namespace Markdig.Helpers
}
return false;
}
/// <summary>
/// Replaces <typeparamref name="TElement"/> with <paramref name="replacement"/>.
/// </summary>
/// <typeparam name="TElement">Element type to find in the list</typeparam>
/// <param name="replacement">Object to replace this element with</param>
/// <returns><c>true</c> if a replacement was made; otherwise <c>false</c>.</returns>
public bool Replace<TElement>(T replacement) where TElement : T
{
for (var i = 0; i < Count; i++)
{
if (this[i] is TElement)
{
RemoveAt(i);
Insert(i, replacement);
return true;
}
}
return false;
}
}
}

View File

@@ -20,7 +20,8 @@ namespace Markdig
/// <summary>
/// Setups this extension for the specified renderer.
/// </summary>
/// <param name="pipeline">The pipeline used to parse the document.</param>
/// <param name="renderer">The renderer.</param>
void Setup(IMarkdownRenderer renderer);
void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer);
}
}

View File

@@ -39,6 +39,17 @@ namespace Markdig
/// </summary>
public static class MarkdownExtensions
{
/// <summary>
/// Adds the specified extension to the extensions collection.
/// </summary>
/// <typeparam name="TExtension">The type of the extension.</typeparam>
/// <returns>The instance of <see cref="MarkdownPipelineBuilder" /></returns>
public static MarkdownPipelineBuilder Use<TExtension>(this MarkdownPipelineBuilder pipeline) where TExtension : class, IMarkdownExtension, new()
{
pipeline.Extensions.AddIfNotAlready<TExtension>();
return pipeline;
}
/// <summary>
/// Uses all extensions except the BootStrap, Emoji, SmartyPants and soft line as hard line breaks extensions.
/// </summary>
@@ -66,8 +77,7 @@ namespace Markdig
.UseAutoLinks()
.UseGenericAttributes(); // Must be last as it is one parser that is modifying other parsers
}
/// <summary>
/// Uses this extension to enable autolinks from text `http://`, `https://`, `ftp://`, `mailto:`, `www.xxx.yyy`
/// </summary>

View File

@@ -2,6 +2,7 @@
// This file is licensed under the BSD-Clause 2 license.
// See the license.txt file in the project root for more information.
using System;
using System.Collections.ObjectModel;
using System.IO;
using Markdig.Helpers;
using Markdig.Parsers;
@@ -34,7 +35,10 @@ namespace Markdig
internal bool PreciseSourceLocation { get; set; }
internal OrderedList<IMarkdownExtension> Extensions { get; }
/// <summary>
/// The read-only list of extensions used to build this pipeline.
/// </summary>
public OrderedList<IMarkdownExtension> Extensions { get; }
internal BlockParserList BlockParsers { get; }
@@ -56,7 +60,7 @@ namespace Markdig
if (renderer == null) throw new ArgumentNullException(nameof(renderer));
foreach (var extension in Extensions)
{
extension.Setup(renderer);
extension.Setup(this, renderer);
}
}
}

View File

@@ -25,6 +25,6 @@ namespace Markdig
{
public static partial class Markdown
{
public const string Version = "0.9.1";
public const string Version = "0.10.7";
}
}

View File

@@ -99,6 +99,8 @@ namespace Markdig.Syntax
SourceSpan urlSpan;
SourceSpan titleSpan;
var startSpan = text.Start;
if (!LinkHelper.TryParseLinkReferenceDefinition(ref text, out label, out url, out title, out labelSpan, out urlSpan, out titleSpan))
{
return false;
@@ -108,7 +110,8 @@ namespace Markdig.Syntax
{
LabelSpan = labelSpan,
UrlSpan = urlSpan,
TitleSpan = titleSpan
TitleSpan = titleSpan,
Span = new SourceSpan(startSpan, titleSpan.End > 0 ? titleSpan.End: urlSpan.End)
};
return true;
}

View File

@@ -1,6 +1,6 @@
{
"title": "Markdig",
"version": "0.9.1",
"version": "0.10.7",
"authors": [ "Alexandre Mutel" ],
"description": "A fast, powerfull, CommonMark compliant, extensible Markdown processor for .NET with 20+ builtin extensions (pipetables, footnotes, definition lists... etc.)",
"copyright": "Alexandre Mutel",
@@ -11,7 +11,7 @@
"projectUrl": "https://github.com/lunet-io/markdig",
"iconUrl": "https://raw.githubusercontent.com/lunet-io/markdig/master/img/markdig.png",
"requireLicenseAcceptance": false,
"releaseNotes": "> 0.9.1\n- Fix regression bug with conflicts between autolink extension and html inline/regular links\n > 0.9.0\n- Add new Autolink extension\n",
"releaseNotes": "> 0.10.7\n - Fix issue when an url ends by a dot `.`\n> 0.10.6\n - Fix emphasis with HTML entities\n> 0.10.5\n - Several minor fixes\n> 0.10.4\n - Fix issue with autolinks\n - Normalize number of columns for tables\n> 0.10.3\n - Fix issue with pipetables shifting a cell to a new column (issue #73)\n> 0.10.2\n - Fix exception when trying to urlize an url with an unicode character outside the supported range by NormD (issue #75)\n > 0.10.1\n- Update to latest CommonMark specs\n- Fix source span for LinkReferenceDefinition\n> 0.10.0\n- Breaking change of the IMarkdownExtension to allow to receive the MarkdownPipeline for the renderers setup\n",
"tags": [ "Markdown CommonMark md html md2html" ]
},
"configurations": {
@@ -34,7 +34,7 @@
"frameworks": {
"net35": {
"buildOptions": {
"define": [ "SUPPORT_NORMALIZE", "SUPPORT_FIXED_STRING" ]
"define": [ "SUPPORT_FIXED_STRING" ]
},
"frameworkAssemblies": {
"mscorlib": "",
@@ -44,7 +44,7 @@
},
"net40": {
"buildOptions": {
"define": [ "SUPPORT_NORMALIZE", "SUPPORT_FIXED_STRING" ]
"define": [ "SUPPORT_FIXED_STRING" ]
},
"frameworkAssemblies": {
"mscorlib": "",

View File

@@ -0,0 +1,104 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
namespace UnicodeNormDApp
{
class Program
{
static void Main(string[] args)
{
var httpClient = new WebClient();
var data = httpClient.DownloadString("http://www.unicode.org/Public/UCD/latest/ucd/NormalizationTest.txt");
var stringReader = new StringReader(data);
var sep = new char[] {';'};
var spaceSpec = new char[] {' '};
string line;
int count = 0;
int min = int.MaxValue;
int max = int.MinValue;
var values = new Dictionary<char, string>();
var builder = new StringBuilder();
while ((line = stringReader.ReadLine()) != null)
{
if (line.StartsWith("#") || line.StartsWith("@"))
{
continue;
}
var commentIndex = line.IndexOf('#');
var dataLine = commentIndex > 0 ? line.Substring(0, commentIndex) : line;
var columns = dataLine.Split(sep, StringSplitOptions.RemoveEmptyEntries);
if (columns.Length < 4)
{
continue;
}
// Skip multi code point
if (columns[0].IndexOf(' ') > 0)
{
continue;
}
var source = Convert.ToInt32(columns[0], 16);
if (source < min)
{
min = source;
}
if (source > max)
{
max = source;
}
var column4Space = columns[4].Split(spaceSpec, StringSplitOptions.RemoveEmptyEntries);
builder.Clear();
for (int i = 0; i < column4Space.Length; i++)
{
var nfdFirst = Convert.ToInt32(column4Space[i], 16);
// We support only single char codepoints
string unicodeString = char.ConvertFromUtf32(nfdFirst);
// We restrict to ascii only
if (unicodeString.Length == 1 && nfdFirst > 32 && nfdFirst < 127)
{
builder.Append(unicodeString[0]);
}
}
var str = builder.ToString();
var sourceString = char.ConvertFromUtf32(source);
// We don't keep spaces
if (sourceString.Length == 1 && str.Length > 0 && !values.ContainsKey(sourceString[0]))
{
//Trace.WriteLine(columns[0] + "/" + source + ": " + char.ConvertFromUtf32(source) + " => " + (char)nfdFirst);
count++;
values.Add(sourceString[0], str);
}
}
//var newValues = new Dictionary<int, char>(values.Count)
//{
// {15, 'a'}
//}
Trace.WriteLine($"CodeToAscii = new Dictionary<char, string>({values.Count})");
Trace.WriteLine("{");
foreach (var pair in values)
{
var escape = pair.Value.Replace("\\", @"\\").Replace("\"", "\\\"");
Trace.WriteLine($" {{'{pair.Key}',\"{escape}\"}},");
}
Trace.WriteLine("};");
//Trace.WriteLine("count: " + count);
//Trace.WriteLine("max: " + max);
//Trace.WriteLine("min: " + min);
}
}
}

View File

@@ -0,0 +1,36 @@
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
// General Information about an assembly is controlled through the following
// set of attributes. Change these attribute values to modify the information
// associated with an assembly.
[assembly: AssemblyTitle("UnicodeNormDApp")]
[assembly: AssemblyDescription("")]
[assembly: AssemblyConfiguration("")]
[assembly: AssemblyCompany("")]
[assembly: AssemblyProduct("UnicodeNormDApp")]
[assembly: AssemblyCopyright("Copyright © 2016")]
[assembly: AssemblyTrademark("")]
[assembly: AssemblyCulture("")]
// Setting ComVisible to false makes the types in this assembly not visible
// to COM components. If you need to access a type in this assembly from
// COM, set the ComVisible attribute to true on that type.
[assembly: ComVisible(false)]
// The following GUID is for the ID of the typelib if this project is exposed to COM
[assembly: Guid("33ffc0b9-0187-44f9-9424-bb5af5b4fb84")]
// Version information for an assembly consists of the following four values:
//
// Major Version
// Minor Version
// Build Number
// Revision
//
// You can specify all the values or you can default the Build and Revision Numbers
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]

View File

@@ -0,0 +1,56 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}</ProjectGuid>
<OutputType>Exe</OutputType>
<AppDesignerFolder>Properties</AppDesignerFolder>
<RootNamespace>UnicodeNormDApp</RootNamespace>
<AssemblyName>UnicodeNormDApp</AssemblyName>
<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<PlatformTarget>AnyCPU</PlatformTarget>
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<Optimize>false</Optimize>
<OutputPath>bin\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<PlatformTarget>AnyCPU</PlatformTarget>
<DebugType>pdbonly</DebugType>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DefineConstants>TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Core" />
<Reference Include="System.Net" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="Microsoft.CSharp" />
<Reference Include="System.Data" />
<Reference Include="System.Xml" />
</ItemGroup>
<ItemGroup>
<Compile Include="Program.cs" />
<Compile Include="Properties\AssemblyInfo.cs" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
</Project>

5
src/global.json Normal file
View File

@@ -0,0 +1,5 @@
{
"sdk": {
"version": "1.0.0-preview2-003121"
}
}

View File

@@ -1,7 +1,7 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 14
VisualStudioVersion = 14.0.25123.0
VisualStudioVersion = 14.0.25420.1
MinimumVisualStudioVersion = 10.0.40219.1
Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Markdig", "Markdig\Markdig.xproj", "{8A58A7E2-627C-4F41-933F-5AC92ADFAB48}"
EndProject
@@ -23,6 +23,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Markdig.Benchmarks", "Markd
EndProject
Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Markdig.WebApp", "Markdig.WebApp\Markdig.WebApp.xproj", "{3CAD9801-9976-46BE-BACA-F6D0D21FDC00}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "UnicodeNormDApp", "UnicodeNormDApp\UnicodeNormDApp.csproj", "{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@@ -45,6 +47,10 @@ Global
{3CAD9801-9976-46BE-BACA-F6D0D21FDC00}.Debug|Any CPU.Build.0 = Debug|Any CPU
{3CAD9801-9976-46BE-BACA-F6D0D21FDC00}.Release|Any CPU.ActiveCfg = Release|Any CPU
{3CAD9801-9976-46BE-BACA-F6D0D21FDC00}.Release|Any CPU.Build.0 = Release|Any CPU
{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}.Debug|Any CPU.Build.0 = Debug|Any CPU
{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}.Release|Any CPU.ActiveCfg = Release|Any CPU
{33FFC0B9-0187-44F9-9424-BB5AF5B4FB84}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE