Makeup - A syntax highlighter for Elixir

For my use case, the priority is having something that compiles fast in dev and is fast in :prod. I’d prefer something that’s no faster than what we have now but compiles faster in :dev than something which is faster but compiles slowly in :dev. But for other users, raw speed might be more important…

Indeed. The HTML is always the same, only the CSS changes, so I definitely could do it. I have chosen not to paste the same code many times precisely because it serves as a good benchmark for Makeup.

Currently it’s rendering ~120 code blocks in 2-4s. For real projects, there might be some oportunities to parallelize it, but I don’t know if that’s something I should handle in makeup core…

1 Like

Another example of Makeup’s output: the official elixir docs highlighted by Makeup instead of Highlight.js: https://tmbb.github.io/exdocs_makedown_demo/api-reference.html.

For an example of actual code being highlighted, see here: https://tmbb.github.io/exdocs_makedown_demo/Agent.html#module-examples

It also highlights the iex prompt correctly: https://tmbb.github.io/exdocs_makedown_demo/Calendar.ISO.html#day_of_week/3, so it now doubles as an iex lexer.

You can’t try this at home yet because it still depends on an unreleased package and on manually editing the minified javascript returned by ExDoc to play well with my own javascript.

EDIT: the CSS stylefor the highlighted code is not the same. I didn’t try to port the highlightjs stylesheet, and I don’t think I should because my own style is richer and shows more distinctions between identifiers. The them is my new “Samba” theme, which is a patched version of the Pygments “Tango” style to render module names in a different color from variable names and play better with the rest of the ExDocs theme.

5 Likes

Very nice, but scroll down on any of those links all the way and click “Switch to night mode” at the very bottom, something is not working well there. ^.^;

3 Likes

Nice catch! I have to find out how that night mode thing works and add a new theme that’s more readable during the night mode

1 Like

Ok, big news:

  • There is now a library, Makedown, which is basically Earmark with a custom code rendering function that uses Makeup.

  • ExDoc master (still unreleased) is now compatible with Makedown. You can’t use it yet to document hex packages because Hex doesn’t accept dependencies from GitHub, even if they are dev-only and have no impact on runtime (should this be changed?)

  • You can now clone my demo and build it locally: (it will take a long time to compile because of Makeup but the rendering part is quick)

  • There is a new style that plays well with night mode (maybe a little too colorful, but very readable with the dark colors)

3 Likes

Nice, much better, very readable. :slight_smile:

1 Like

A new version of ExDoc has been released. This version (0.16.3) supports using Makedown (a markdown implementation that uses Makeup) as the markdown processor. Instructions on how to integrate here:

This means you can now use Makeup as the syntax highlighting library for the hexdocs documentation of your packages.

5 Likes

Hmm, I should use it in SpiritEx to complete the circle, I wonder if it would let me have a circular dependency like that… Have a library depend on a library that depends on the first library, yeah it should work if the API is the same I think then. ^.^

Just tried it. It doesn’t work… ExSpirit is downloaded as a dependency and you get conflicts. You can probably hack your way around it but it won’t work when uploading the documentation to hexdocs… You can add it to other packages, thoug.

1 Like

I’m going to try it on some of your packages and submit a PR. Not necessarily today.

If it works, I should write a bot that crawls hex.pm, adds the config options to use makedown to all elixir packages and submit a PR to GitHub xD

1 Like

Done

Note that the function name in the function head is not highlighted. This follows the behavior of the original Pygments lexer. I’ll probably changed it one of these days, though. IN any case, I think it’s already better than highlightjs.

Lol, ProtocolEx is definitely one where the original highlighter fails pretty hard since it does not know the special syntax I use, it is a good test. ^.^

I left some comments on the PR. :slight_smile:

My highlighter might fail there as well… It does NOT highlight defimplEx. At a certain point I thought about highlighting everything that started with def but it’s not a good idea. I might add a configuration option for users who want to especify extra keywords.

But it does avoid some dramatic fails (unrelated to your special syntax)

1 Like

The latest version of Makup has broken compatibility. Each language now lives in a different package and you must explicitky import the ones you need.

1 Like

In the last few days Github has refused to serve assets from the _static directory, even though I do have a .no-jekyll file. This has broken the demo website, which is hosted on GithubPages. I have no idea why this happened and I intend to find out, but meanwhile I’ve changed the assets to the static directory and it works now.

1 Like

That… seems weird to not serve from? o.O

For people following this thread, I’ve released a new library that aims to make it easier to use Makeup for syntax highlighting of package documentation. As a result, it’s now easier than ever to use Makeup together with ExDoc: ExDocMakeup - ExDoc + Makeup syntax highlighting

2 Likes

New version released, based on NimbleParsec. It’s way faster than the previous version based on ExSpirit. I’ll add some docs soon, but the code is out there in the repo if you want to play with it. It’s a great way to look at how flexible @josevalim’s NimbleParsec is.

6 Likes

In which repo?

That one linked in your initial post here has no releases in.

Is ExSpirit part of ExDocs? Since ExSpirit is absent on Github and your README.md still says ExDocs is in use. :slight_smile: