dogweather

dogweather

Code critique: group_after/2 for parsing flat HTML

My problem

When parsing old government webpages, my input is often just like that one:

<p><b>Section 1, name, and text</b></p>
<p>Section 1 more text</p>
<p>Section 1 more text</p>
<p><b>Section 2, name, and text</b></p>
<p>Section 2 more text</p>
<p><b>Section 3,  name, and text</b></p>
// etc.

I’d really like feedback about the approach I came up with last night:

This looks like a take/while/scan kind of problem. But I couldn’t find an Enum or Floki function that seemed to handle this kind of repeated pattern.

I decided to write a function that would generically group items and following items using a predicate. In the case of the HTML above, the predicate would be “Does the element contain a <b>?” So, abstractly:

input =  [1,2,2,2,2,1,1,1,2]
output = [[1,2,2,2,2], [1], [1], [1, 2]]

I realized that’s not too hard with Enum.reduce:

  def group_after(list, predicate) do
    reduce(list, [], fn e, acc ->
      case predicate.(e) do
        true ->
          [[e]] ++ acc

        false ->
          {curr, rest} = List.pop_at(acc, 0)
          [curr ++ [e] | rest]
      end
    end)
  end

It works fine. Although, the reduce function’s code is very procedural and not expressive. What do you all think? Is there another approach I’m not considering?

An alternate idea: Consider a string "tfffftttf" as an isomorph of map(list, predicate). Then use an expressive regex like ~r/tf*/ to group the true & false — instead of the procedural reduce. Finally, undo the mapping back into the original list elements.

Most Liked

derek-zhou

derek-zhou

This is not too bad; at least all the <p> have closing </p>. You know they don’t have to; and I;ve seen html that freely mix the 2 styles, with or without closing </p>

Eiji

Eiji

Slow load (possibly timeout). I know what you feel. Hope those are not ASP.net pages with invalid HTML code. :smiling_imp:

Anyway, here is my solution:

Mix.install([:floki])

defmodule Example do
  def sample(list, acc \\ [])

  # for empty input after parsing
  def sample([], []), do: []

  # when all p elements are passed
  # reverse last section texts and wrap them into list
  # as otherwise a resulting list would be added
  # to a main list where each element contains a list of sections texts
  def sample([], acc), do: [:lists.reverse(acc)]

  # in case of first bold text simply add text to
  # as the only element in new acc
  # and call function recursively
  def sample([{"p", _, [{"b", [], [text]}]} | tail], []) when is_binary(text) do
    sample(tail, [text])
  end

  # however if there is some data in acc
  # reverse its contents and return it as a list of
  # last section texts and recursive call
  def sample([{"p", _, [{"b", [], [text]}]} | tail], acc) when is_binary(text) do
    [:lists.reverse(acc) | sample(tail, [text])]
  end

  # when we got a normal text simply add it to acc
  # and call function recursively
  def sample([{"p", _, [text]} | tail], acc) when is_binary(text) do
    sample(tail, [text | acc])
  end
end

"""
<p><b>Section 1, name, and text</b></p>
<p>Section 1 more text</p>
<p>Section 1 more text</p>
<p><b>Section 2, name, and text</b></p>
<p>Section 2 more text</p>
<p><b>Section 3,  name, and text</b></p>
"""
|> Floki.parse_fragment!()
|> Example.sample()
|> dbg()

Pattern matching is a fastest solution. You can take a look at this post to see possible alternative solutions.

dogweather

dogweather

Thanks! That’s pretty interesting. This meta tag is pretty chilling. :slight_smile:

<meta name=Generator content="Microsoft Word 15 (filtered)">

It seems they’re somehow scripting MS Word to “Save as HTML”.

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&amp;query=perfume&amp;page=2, I would like to get: ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement