Thanks! I was using Earmark.parse to convert MD to plaintext, but the Earmark.as_ast is a much more convenient and simpler API to do the same job.

please be aware that Earmark.parse is private now

BTW

1.4.0 2019/09/05

• 145 Expose AST for output manipulation

• 238 Pure Links are default now

• 256 Align needed Elixir Version with ex_doc (>= 1.7)

• 259 Deprecated option sanitize removed

• 261 Deprecated Plugins removed

• 265 Make deprecated Earmark.parse/2 private

Wow this is great news, thanks for the update here, AST is going to be incredibly useful!

I’m using Earmark in my project and I’m planning to implement a few custom markups. For example, auto-link specific characters ( @ mentions, # issues, etc.) and support emojis. See almightycouch/gitgud#44.

The plugin mechanism as been deprecated in 1.4 and working with the AST should work just fine for implementing custom markups. I see that #27 should provide a way to render the AST as HTML.

When writing custom markups, say @ mentions for example. Should I walk the entire AST searching for a @ character in each tag (third element in tuple), ignore code elements and build-up the new AST from there?

Will a future version of Earmark provide a generic way for this kind of things? I think that most custom markups will need the same mechanism for traversing the AST and skipping content in inline code and code blocks.

Also, if I want to replace a markup with my own implementation, for example by supporting syntax highlighting in code blocks. Should I simply replace each pre AST element with my own element?

I am really happy that folks are starting to use the AST.

I would walk the AST, I am planning to release such an AstWalker in a future version, right now I am busy with exposing an AstToHtmlTransformer v1.4.1 ETA middle of this week (as you spotted correctly), but I am hunting jobs right now, so that is not sure yet.
Walking the AST yourself should not be a very difficult task, have a look at things like Macro.prewalk or look at my Traverse lib to get some ideas.

Be careful if you want to use the to be released transformer though, if you change the AST’s type it will break, please remember that we are still in experimentation mode.

Summary:

• yes #277 will allow you to just change the AST and get your HTML for free iff you do not change the AST’s type

• Walking the AST will eventually be facilitated by tools in Earmark or maybe an associated lib, let us not forget that every project using ex_doc will pull in all of Earmark’s code, so tools shall probably go elsewhere.

• Yes absolutely change e.g. {"pre", [], [whatever]} to {"post", atts, whateverelse} just watch out for the correct type.

Please keep me updated if you have any problem or question, either here or open a ticket in Earmark.
The more feedback I get, the faster the AST API will converge.

Sure, this could be implemented in a separated package. Maybe a module such as AstWalker will suffice to cover most cases.

I was wondering if the current implementation was working with the AST internally when rendering HTML. Are Earmark.Options reflected in the AST or are they only applied to the rendered HTML?

In the meanwhile, can i use Floki.raw_html/2 to render the AST?

The internal implementation does not work with the AST yet, but it is a clear goal, and also the reason why the Transformer will stay inside Earmark, as it will be used for as_html eventually.

There are some subtle differences to Floki, which was the inspiration for the AST, it would be great if the use of Floki.raw_html/2 would work. Please let me know.

One thing I am almost sure would break Floki are comments, as Floki has a different shape for comment nodes, I did not see a reason to not have a more uniform type, so instead of {:comment, ...} Earmark produces {:comment, [], ...}

I apologize but I am too stupid to quote your questions in my answer

And finally, concerning the Options.
Most options are needed for the AST rendering, e.g. pure_links: however all options concerning the inline rendering are ignored, e.g. smartypants:, and yes the Transformer will take those into account.

I guess I can be more precise in the doc in the next version (or I get I nice PR maybe )

Will give it a try asap.

Just select the text you want to quote in the comment you want and click the “Quote” popover link.