Your thoughts on Mdex? (Markdown parser and Formatter)

Recently came across this great library leandrocp/mdex: A fast 100% CommonMark-compatible GitHub Flavored Markdown parser and formatter for Elixir. by our @leandrocp
I wanted to discuss about the library and realized there is no forum post introducing the library.
So, creating this thread. In the Phoenix LiveView, I just called Mdx.to_html() and voila - live preview is available.
Please explore.

5 Likes

@leandrocp - embedding heex components is in the pipeline? Leandro Pereira on X: “@pepicrft @elixirphoenix @chris_mccord Working on embedding components https://t.co/kthGZ7p1uj :slight_smile: A sigil never crossed my mind, but that’s easy to implement and a PR is very welcome!” / X (twitter.com)

Use it and love it :+1:

2 Likes

The reason I’m not talking much about that library yet is because I’m still working on the API to manipulate AST and consequently allow extending it, think of how Req works but for Markdown. I’m glad it works for you!

8 Likes

Yep it’s in the pipeline. As soon as I solve AST manipulation it will be possible to format to HTML, HEEX, and even call arbitrary Elixir code. For context, https://mdxjs.com is a big inspiration for this library.

4 Likes

I’m waiting for a elixir formatter plugin that will format my docs. I always have to redeploy to hex due to broken formatting :slight_smile:

BTW @leandrocp, how do we extend the library? am-kantox/md: Stream-aware markdown parser with custom syntax setting (github.com) and mudasobwa/markright: A customizable markdown parser in Elixir: pure pattern matching. (github.com) look two interesting projects.
One question I have is - if we want to support hashtags (#) or mentions (@) - is it better to go to Javascript libraries like Tiptap, Quill etc. or is there anything we can try using your sort of libraries?
Sorry - if the question is very naive.

@cvkmohan I’ve considered md and others but none of them did the job for my needs. I got excited with md and its extension capabilities but quoting its readme “If one needs to perfectly parse the common markdown, Md is probably not the correct choice” so that was a major blocker.

Regarding extending MDEx that’s what I have in mind: RFC - Extension API · Issue #37 · leandrocp/mdex · GitHub - the first part which is parsing the document (AST) and formatting to HTML is half way done (Elixir and Rust), so it’s in progress.

if we want to support hashtags (#) or mentions (@)

I’m not sure I follow your question, both Tiptap and Quill are rich text editors while MDEx is a Markdown parser/formatter. With MDEx you’ll be able to manipulate the document after it was written, either with Tiptap or any other editor. So let’s say for example you want to add a link to every tag in your document, ie: transform #tag_a to [#tag_a](link) you’ll be able to traverse the AST and transform a text node into a link node. See RFC - Extension API · Issue #37 · leandrocp/mdex · GitHub for an example.

Does it make sense?

1 Like

Hi @dkuku what kind of broken formatting you get publishing to hex? I’ve had an issue or two with earmark but in overall it works fine.

The “formatter” in “A fast 100% CommonMark-compatible GitHub Flavored Markdown parser and formatter.” means converting markdown to HTML and potentially other formats, but I’m assuming you’re talking about a mix format plugin, right? Can you elaborate a bit more?

Thanks @leandrocp. Yes, you answered even my imaginary question. :slight_smile:

Yes, I am looking at something like transforming text node into a link node. By using it - I intend to avoid using a Rich Text Editor. MDEx looks sufficiently fast - and - a split text box with source on the left - and - parsed results on the right should make Rich Text Editor unnecessary. And once you bring slab/delta-elixir: Simple yet expressive format to describe contents and changes :card_file_box: (github.com) into the mix, may be collaborative editing is possible entirely in the Elixir world!
Thanks for the answer. Will be looking forward to future releases.

1 Like

If you’re looking for a way to parse and transform mentions, hashtags, and links: GitHub - bonfire-networks/linkify: Linkify is a basic package for turning website names into links.

I really like Mdex and use it for my personal blog.
the only improvement I need is easier adding new languages for highlighting. :101:

1 Like

This would be awesome ! Thanks for the great job!!

1 Like

I really like Mdex and use it for my personal blog.

That’s nice!

the only improvement I need is easier adding new languages for highlighting

Currently it requires updating GitHub - Colonial-Dev/inkjet: A batteries-included syntax highlighting library for Rust, based on tree-sitter. and the main reason is to reuse that repo instead of duplicating work, which adds more steps to the process but allows us to move forward faster too :slight_smile:

@clsource and others, just release MDEx v0.1.16 that includes Objective-C to the syntax highlighter. I’m not very familiar with the language but I’m not sure it’s generating all tokens, ie: applying colors 100% correctly. Can you please give it a try?

1 Like