<feed xmlns='http://www.w3.org/2005/Atom'>
<title>cmark/src/references.c, branch master</title>
<subtitle>My own fork of cmark for commonmark conversion</subtitle>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/'/>
<entry>
<title>Add casts for MSVC10.</title>
<updated>2020-02-16T16:54:19+00:00</updated>
<author>
<name>John MacFarlane</name>
<email>jgm@berkeley.edu</email>
</author>
<published>2020-02-16T16:54:19+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=67ec0eef4b448d32152897c8bbc20190f06d5b3e'/>
<id>67ec0eef4b448d32152897c8bbc20190f06d5b3e</id>
<content type='text'>
This is kivikakk's commit 62166fe3b6b07068ed4c4207113e3c4b060ad4a8
in cmark-gfm.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is kivikakk's commit 62166fe3b6b07068ed4c4207113e3c4b060ad4a8
in cmark-gfm.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix #220 (hash collisions for references).</title>
<updated>2020-02-16T16:50:54+00:00</updated>
<author>
<name>John MacFarlane</name>
<email>jgm@berkeley.edu</email>
</author>
<published>2020-02-16T16:50:54+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=b2378e459be775004af39bbe280846a98c8cbda6'/>
<id>b2378e459be775004af39bbe280846a98c8cbda6</id>
<content type='text'>
This commit ports Vicent Marti's fix in cmark-gfm.
(384cc9db4cd7a90f59c0751e58eb7b3023d38b85)

His commit message follows:

As explained on the previous commit, it is trivial to DoS the CMark
parser by generating a document where all the link reference names hash
to the same bucket in the hash table.

This will cause the lookup process for each reference to take linear
time on the amount of references in the document, and with enough link
references to lookup, the end result is a pathological O(N^2) that
causes medium-sized documents to finish parsing in 5+ minutes.

To avoid this issue, we propose the present commit.

Based on the fact that all reference lookup/resolution in a Markdown
document is always performed as a last step during the parse process,
we've reimplemented reference storage as follows:

1. New references are always inserted at the end of a linked list. This
is an O(1) operation, and does not check whether an existing (duplicate)
reference with the same label already exists in the document.

2. Upon the first call to `cmark_reference_lookup` (when it is expected
that no further references will be added to the reference map), the
linked list of references is written into a fixed-size array.

3. The fixed size array can then be efficiently sorted in-place in O(n
log n). This operation only happens once. We perform this sort in a
_stable_ manner to ensure that the earliest link reference in the
document always has preference, as the spec dictates. To accomplish
this, every reference is tagged with a generation number when initially
inserted in the linked list.

4. The sorted array is then compacted in O(n). Since it was sorted in a
stable way, the first reference for each label is preserved and the
duplicates are removed, matching the spec.

5. We can now simply perform a binary search for the current
`cmark_reference_lookup` query in O(log n). Any further lookup calls
will also be O(log n), since the sorted references table only needs to
be generated once.

The resulting implementation is notably simple (as it uses standard
library builtins `qsort` and `bsearch`), whilst performing better than
the fixed size hash table in documents that have a high number of
references and never becoming pathological regardless of the input.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This commit ports Vicent Marti's fix in cmark-gfm.
(384cc9db4cd7a90f59c0751e58eb7b3023d38b85)

His commit message follows:

As explained on the previous commit, it is trivial to DoS the CMark
parser by generating a document where all the link reference names hash
to the same bucket in the hash table.

This will cause the lookup process for each reference to take linear
time on the amount of references in the document, and with enough link
references to lookup, the end result is a pathological O(N^2) that
causes medium-sized documents to finish parsing in 5+ minutes.

To avoid this issue, we propose the present commit.

Based on the fact that all reference lookup/resolution in a Markdown
document is always performed as a last step during the parse process,
we've reimplemented reference storage as follows:

1. New references are always inserted at the end of a linked list. This
is an O(1) operation, and does not check whether an existing (duplicate)
reference with the same label already exists in the document.

2. Upon the first call to `cmark_reference_lookup` (when it is expected
that no further references will be added to the reference map), the
linked list of references is written into a fixed-size array.

3. The fixed size array can then be efficiently sorted in-place in O(n
log n). This operation only happens once. We perform this sort in a
_stable_ manner to ensure that the earliest link reference in the
document always has preference, as the spec dictates. To accomplish
this, every reference is tagged with a generation number when initially
inserted in the linked list.

4. The sorted array is then compacted in O(n). Since it was sorted in a
stable way, the first reference for each label is preserved and the
duplicates are removed, matching the spec.

5. We can now simply perform a binary search for the current
`cmark_reference_lookup` query in O(log n). Any further lookup calls
will also be O(log n), since the sorted references table only needs to
be generated once.

The resulting implementation is notably simple (as it uses standard
library builtins `qsort` and `bsearch`), whilst performing better than
the fixed size hash table in documents that have a high number of
references and never becoming pathological regardless of the input.
</pre>
</div>
</content>
</entry>
<entry>
<title>Use C string instead of chunk for link URL and title</title>
<updated>2020-01-23T16:25:54+00:00</updated>
<author>
<name>Nick Wellnhofer</name>
<email>wellnhofer@aevum.de</email>
</author>
<published>2020-01-18T22:12:37+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=b237924585e61532ada774bf9e70eadff00666dc'/>
<id>b237924585e61532ada774bf9e70eadff00666dc</id>
<content type='text'>
Use zero-terminated C strings instead of cmark_chunks without storing
the length. This introduces a few additional strlen computations,
but overhead should be low.

Allows to reduce size of struct cmark_node later.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Use zero-terminated C strings instead of cmark_chunks without storing
the length. This introduces a few additional strlen computations,
but overhead should be low.

Allows to reduce size of struct cmark_node later.
</pre>
</div>
</content>
</entry>
<entry>
<title>Use cmark_mem to free where used to alloc</title>
<updated>2016-09-26T05:44:37+00:00</updated>
<author>
<name>Yuki Izumi</name>
<email>kivikakk@github.com</email>
</author>
<published>2016-09-26T05:44:36+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=f4e9ba2068880d25034ebeca172e521d34d6b540'/>
<id>f4e9ba2068880d25034ebeca172e521d34d6b540</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Reformatted.</title>
<updated>2016-06-25T04:55:44+00:00</updated>
<author>
<name>John MacFarlane</name>
<email>jgm@berkeley.edu</email>
</author>
<published>2016-06-25T04:55:44+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=2b1b00fb19fe470f7745c208377345975b9e0b2c'/>
<id>2b1b00fb19fe470f7745c208377345975b9e0b2c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>cmark_reference_lookup: Return NULL if reference is null string.</title>
<updated>2016-06-23T05:18:57+00:00</updated>
<author>
<name>John MacFarlane</name>
<email>jgm@berkeley.edu</email>
</author>
<published>2016-06-23T05:18:24+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=9a8610c9715e714e40ad7a93be89985fe5371907'/>
<id>9a8610c9715e714e40ad7a93be89985fe5371907</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>msvc: Fix warnings and errors</title>
<updated>2016-06-06T22:39:06+00:00</updated>
<author>
<name>Vicent Marti</name>
<email>tanoku@gmail.com</email>
</author>
<published>2016-06-06T09:50:13+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=d4cb8e767ce224d31501c19fb11a220824df9bf8'/>
<id>d4cb8e767ce224d31501c19fb11a220824df9bf8</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>cmark: Implement support for custom allocators</title>
<updated>2016-06-06T22:39:05+00:00</updated>
<author>
<name>Vicent Marti</name>
<email>tanoku@gmail.com</email>
</author>
<published>2016-05-27T14:55:16+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=25429c96f6554ffac415f9d865934b1183f3398e'/>
<id>25429c96f6554ffac415f9d865934b1183f3398e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>cmake: Global handler for OOM situations</title>
<updated>2016-06-06T22:39:05+00:00</updated>
<author>
<name>Vicent Marti</name>
<email>tanoku@gmail.com</email>
</author>
<published>2016-05-24T13:50:44+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=42b07cc9c8d2e6251d190e5ea0d13fd66cb51e6d'/>
<id>42b07cc9c8d2e6251d190e5ea0d13fd66cb51e6d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Prefix utf8proc functions to avoid conflict with existing library</title>
<updated>2015-08-06T16:07:23+00:00</updated>
<author>
<name>Kevin Wojniak</name>
<email>kainjow@kainjow.com</email>
</author>
<published>2015-08-06T16:07:23+00:00</published>
<link rel='alternate' type='text/html' href='http://ns2.complex-networks.net/cmark/commit/?id=2f393322635cbc1eea0fd75ac17c00d01fe42176'/>
<id>2f393322635cbc1eea0fd75ac17c00d01fe42176</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
