dogweather
Code critique: group_after/2 for parsing flat HTML
My problem
When parsing old government webpages, my input is often just like that one:
<p><b>Section 1, name, and text</b></p>
<p>Section 1 more text</p>
<p>Section 1 more text</p>
<p><b>Section 2, name, and text</b></p>
<p>Section 2 more text</p>
<p><b>Section 3, name, and text</b></p>
// etc.
I’d really like feedback about the approach I came up with last night:
This looks like a take/while/scan kind of problem. But I couldn’t find an Enum or Floki function that seemed to handle this kind of repeated pattern.
I decided to write a function that would generically group items and following items using a predicate. In the case of the HTML above, the predicate would be “Does the element contain a <b>?” So, abstractly:
input = [1,2,2,2,2,1,1,1,2]
output = [[1,2,2,2,2], [1], [1], [1, 2]]
I realized that’s not too hard with Enum.reduce:
def group_after(list, predicate) do
reduce(list, [], fn e, acc ->
case predicate.(e) do
true ->
[[e]] ++ acc
false ->
{curr, rest} = List.pop_at(acc, 0)
[curr ++ [e] | rest]
end
end)
end
It works fine. Although, the reduce function’s code is very procedural and not expressive. What do you all think? Is there another approach I’m not considering?
An alternate idea: Consider a string
"tfffftttf"as an isomorph ofmap(list, predicate). Then use an expressive regex like~r/tf*/to group the true & false — instead of the proceduralreduce. Finally, undo the mapping back into the original list elements.
Most Liked
derek-zhou
This is not too bad; at least all the <p> have closing </p>. You know they don’t have to; and I;ve seen html that freely mix the 2 styles, with or without closing </p>
Eiji
Slow load (possibly timeout). I know what you feel. Hope those are not ASP.net pages with invalid HTML code. ![]()
Anyway, here is my solution:
Mix.install([:floki])
defmodule Example do
def sample(list, acc \\ [])
# for empty input after parsing
def sample([], []), do: []
# when all p elements are passed
# reverse last section texts and wrap them into list
# as otherwise a resulting list would be added
# to a main list where each element contains a list of sections texts
def sample([], acc), do: [:lists.reverse(acc)]
# in case of first bold text simply add text to
# as the only element in new acc
# and call function recursively
def sample([{"p", _, [{"b", [], [text]}]} | tail], []) when is_binary(text) do
sample(tail, [text])
end
# however if there is some data in acc
# reverse its contents and return it as a list of
# last section texts and recursive call
def sample([{"p", _, [{"b", [], [text]}]} | tail], acc) when is_binary(text) do
[:lists.reverse(acc) | sample(tail, [text])]
end
# when we got a normal text simply add it to acc
# and call function recursively
def sample([{"p", _, [text]} | tail], acc) when is_binary(text) do
sample(tail, [text | acc])
end
end
"""
<p><b>Section 1, name, and text</b></p>
<p>Section 1 more text</p>
<p>Section 1 more text</p>
<p><b>Section 2, name, and text</b></p>
<p>Section 2 more text</p>
<p><b>Section 3, name, and text</b></p>
"""
|> Floki.parse_fragment!()
|> Example.sample()
|> dbg()
Pattern matching is a fastest solution. You can take a look at this post to see possible alternative solutions.
dogweather
Thanks! That’s pretty interesting. This meta tag is pretty chilling. ![]()
<meta name=Generator content="Microsoft Word 15 (filtered)">
It seems they’re somehow scripting MS Word to “Save as HTML”.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








