Makeup - A syntax highlighter for Elixir

I’ve published the first version of my Makeup library. It’s a syntax highlighter for Elixir in the spirit of Pygments, Currently it highlights Elixir and HTML5 and formats the results as HTML.

GitHub repo: https://github.com/tmbb/makeup
Hex package: https://hex.pm/packages/makeup

The Elixir highlighter is already quite good in my opinion. The HTML5 hasn’t been tested as thoroughly.

Demo here: https://tmbb.github.io/makeup_demo/ (navigate the links to find the demos for Elixir and HTML5).

Contributions are accepted and highly encouraged, especially new lexers. Documentation on how to write them is pretty much inexistent at the moment, though.

22 Likes

Nice! I’m sure this will be useful.

1 Like

It can integrate with earmark with no code changes on the earmark side (and trivial changes on makeup itself). This could probably be used in ex_docs for better syntax highlighting of elixir code (the current syntax highlighting isn’t exactly great).

But I’d need to write a lexer for iex sessions so that doctests would render fine (the current elixir lexer doesn’t work very well for doctests in some situations.

1 Like

@tmbb:

  1. Is it possible to integrate it with Atom editor?
  2. Can we color every code part? For example I don’t found macro calls (there is only definition). I would like to color in similar tones parts like:
    def vs defp
    defmacro vs defmacrop
    function_call vs macro_call
    At least in some of my libraries I use macros to generate modules and/or functions and I would like to see easy function_call vs macro_call especially after bigger time break between developing code.
  3. Can I add blinking red background color to mark part of code that does not compiles?
  4. Overwiev button on demo page does not work - 404 error page
  5. Integration with earmark and iex would be awesome!
    Currently I have different colors in atom and different colors in console, so making them same would be really good point.

Let me know if you need a help with ex_doc. I was need to know how at least some parts works in it, so I look at code deeper and in case you want to develop fork then I could help. Maybe we could make ex_doc much more configurable starting from choose tool for syntax highlighting.

Probably, since atom uses HTML and CSS… But makeup is optimized for offline use, not for interactive editing. And the overhead of communicating with Atom is probably intolerable. The use case for this is to display code on webpages and such.

1 Like

Assuming you want to color the words def, defp, etc., yes. I mean, not currently, but I can easily add an option in which you pass a map of identifiers to colors (%{"def" => "red", "defp" => "pink", ...}). It won’t play well with styles (you need to set colors that play well with the style), but it’s possible. In fact, Pygments styles have some serious limitations, but they were readily available so I used them. Maybe I’ll overhaul the whole thing.

1 Like

No out of the box. Makeup can put pretty colors on your code, but it operates on lexical analysis only. what you want need a deep knowledge about your code.

It can be done pretty easily with the new elixir debug tools, though. I will certainly integrate with it so that you can draw distinctions between functions and mactos, and even have hyperlinks from the function call to its definition. This eas the whole reason for me to want a tool in pure elixir, otherwise I’d just use Python :smile:

But that’s still in the future (I have the architecture more or less sketched out in my head, but implementing it might take a while).

1 Like

Again, not possible out of the box. Unlike the feature above, this is not on the roadmap. The focus is on rendering code that’s already been tested and compiles. Again, makeup os veeery stupid and obly cares about the surface details of your code [insert joke about makeup here]

2 Likes

Thanks! This button wasn’t even meant to exist. The HTML is a gutted version of the webpage of a PHP project, and I’ve forgotten to remove this button. I’ll remove it tonight.

1 Like

I’ve never mentioned integration with iex. It’s a completely different scope! Adding syntax highlighting to iex is something different (even thiugh makeup might be able to help with that someday).

What I said is that I wanted to have a way of highlighting doctests, which are written as if they were an iex session, including the iex prompt. My elixir lexer won’t deal well with that.

This is a problem if this is meant to integrate with ex_docs.

1 Like

ex_doc can already choose the markdown implementarion, and I think Ican easily create a dummy “markdown implementation” which is just earmark configured to use makeup as a code rendering function (that’s an option you can set). The only thing that would require changin ex_docs source is adding makeup’s stylesheet. I don’t think a full fork is needed… But any help is welcome!

1 Like

Impossible for the common “highlight a single file”-highlighters. It does only know about the currently highlighted module Foo, nothing about Bar. Therefore it cant know if Bar.foo() is calling into a macro or into a function.

3 Likes

My priorities regarding features are (roughly in order):

  • Make it so that when the user places the cursor over a matched delimiter ((...) %{...} {...} <<...>> [...] %Struct{...}), both delimiters (left and right) are highlighted. I can do this with few code changes and ~ 20 lines of javascript).
  • Add iex lexer for doctests
  • Add lexer for elixir stacktraces
  • Add general eex lexer
  • Add html.eex lexer
  • Add CSS lexer
  • Add Javascript lexer (this will allow me highlight CSS and Javascript inside and HTML file)
2 Likes

It looks very nice. Awesome work :slight_smile:

2 Likes

Added the first feature:

  • Make it so that when the user places the cursor over a matched delimiter ((...) %{...} {...} <<...>> [...] %Struct{...}), both delimiters (left and right) are highlighted.

This change is already reflected in the demo. You can visit the website and place the cursor over the delimiters. The matching delimiter will be highlighted. In the HTML5 examples matching open and closing tags will also be highlighted if you place the cursor over one of the tags. Try it, it’s cool.

2 Likes

Indeed it is! Now it needs to work on do/fn to end too. ^.^

Done. Try here, for example: https://tmbb.github.io/makeup_demo/elixir.html#abap-nesting.exs

Whoo nice, I like the complex ‘test’ you made for blocks. ^.^

2 Likes

Ahaha, it’s a PEG parser. As long as the grammar definition is sound, it handles averything you throw at it. Supporting the highlighting of matching blocks and delimiters does have a noticeable impact on performance (for something as complex as the demo pages), but I think it’s worth it. But it’s still quite fast, probably thanks to ExSpirit!

1 Like

Still tons of optimizations I can do, it is pretty ‘simple’ as far as previous PEG parsers I’ve written. I’m really tempted to change it to be a DSEL instead of function/macro composition… >.>

Though I’m guessing that you are parsing over every code block in your examples individually (all 100+ of them?!?) which would definitely be a lot slower than parsing over it once and pasting it in different places. ^.^

Good test to do in any case. :slight_smile: