sodapopcan

sodapopcan

Earmark - Parsing HTML inside code blocks

Using Earmark, I’m trying to parse HTML inside markdown code blocks so that they can be syntax-highlighted. I’m having a bit of a rough time with it and, admittedly, I could probably stand to spend a bit more time trying to figure it out myself but I feel I’m maybe on the wrong path as it stands so I thought it couldn’t hurt to ask—also, whenever I do ask for help, I usually figure it out minutes later :sweat_smile:

So, I think understand that the following doesn’t work—the 3rd (content) element of the 4-tuple is usually ignored:

def parse() do
  """
  ```sql
  <span class="k">SELECT</span> * <span class="k">FROM</span> table
  ```
  """
  |> Earmark.as_ast!()
  |> Earmark.map_ast(&parse_html/1)
  |> Earmark.transform()
end

defp parse_html({"code", [{"class", "sql"}], [html], meta} = node, true) do
  {:ok, html} = Floki.parse_fragment(html)

  {:replace, {"code", [{"class", "sql"}], [html], meta}}
end

I say usually since the documentation doesn’t explicitly mention that the content node is ignored when using {:replace, node} yet that seems to be the case (I’m actually not sure .

I there a way to simply replace the content node?

Is there just a better way of doing this (without using highlightjs)?

I’m looking at pre- and post-processors but currently not having much luck.

Thanks for reading!

Most Liked Responses

sodapopcan

sodapopcan

After sleeping on it, I figured it out. It was right there in the documentation, it just didn’t click with me that that is what I needed.

Using map_with_ast we can use the accumulator to conditionally match on a specific text node. The part that the accumulator was used to match in this was is that part that flew over my head when first reading it.

So I’ve ended up with this:

"""
```sql
<span class="k">SELECT</span> * <span class="k">FROM</span> table
```
"""
markdown
|> Earmark.as_ast!()
|> Earmark.Transform.map_ast_with(false, fn
  {"code", [{"class", "sql"}], _, meta}, _ ->
    {{"code", [{"class", "sql"}], nil, meta}, true}

  html, true ->
    {:ok, html} = Floki.parse_fragment(html)

    html =
      Floki.traverse_and_update(html, fn
        {tag, args, children} -> {tag, args, children, %{}}
      end)

    {html, false}

  node, _ ->
    {node, false}
end)
|> Earmark.transform(options())

Which works! The Floki.traverse_and_update/2 call is necessary to convert from Floki’s tuple representation to Earmark’s.

The only thing left is that the spans are put on their own lines which is causing the formatting to be all wonky, though that is expected and will have to figure something else out there.

RobertDober

RobertDober

Great you found it, was just about to try it out …

Thank you for the PR too, very much appreciated

sodapopcan

sodapopcan

Hey @RobertDober, thanks for the reply and thanks for all your work on Earmark—that is very much appreciated!

Yes, compact_output: true did not help since, as you likely well know, spans are not @compact_tags. I was able to fix it by converting the spans to ems (which I don’t mind at all since semantically they are emphasized although I’m doing it programmatically since I want to eventually integrate with makeup or vim’s :TOhtml) but then I was faced with the problem that if two tags are in a row they render without spaces. e.g.: <em class="k">SELECT</em> <em class="k">FROM</em> they render as SELECTFROM. I was actually able to solve this but in a very convoluted way:

    {result, _} =
      Earmark.Transform.map_ast_with(result, nil, fn
        {"em", args, _, meta}, nil ->
          {{"em", args, nil, meta}, :em_first}

        {"em", args, _, meta}, :em_next ->
          {{"em", args, nil, meta}, :em_text}

        {tag, args, _, meta}, _ ->
          {{tag, args, nil, meta}, nil}

        text, :em_first ->
          {text, :em_next}

        text, :em_text ->
          {" #{text}", :em_next}

        node, _ ->
          {node, nil}
      end)

So basically saying "If we see an em for the first time, mark it as such (:em_first), then when we see its text node, do nothing other than that mark it to look out for another em (:em_next). If the next node is indeed an em, mark it that it’s part of a string of ems (still :em_next) and leftPad™ it. Anything else, just reset.

It’s a little convoluted and I haven’t revisited it since I got it working.

For all intents and purposes this solves my problems, but having a another little issue that I was going to open in the repo since it seems more appropriate to discuss there (and I want to look at the source a little more to understand if it’s reasonable or not).

Where Next?

Popular in Questions Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement