aphillipo
Hello there! I’ve been working on an Excerpt implementation on top of Floki that greedily includes tags. I’d love for the next step to be able to preserve tags but this is a good starting point; any advice on that welcome.
Let me know if there is anything I could improve or if there are any performance issues I’m encountering by using recursion for this. It’s also worth noting this is an adaption
of the DeepText search that comes with Floki for my own needs.
defmodule PayPip.Text do
@moduledoc """
PayPip.Text is a strategy to get text nodes from a HTML tree using a deep search
algorithm limited to a certain length + the full content of the final node. It will
get all string nodes and concat them into an accumulator up to that point.
"""
@type html_tree :: tuple | list
@doc """
Get text nodes of a desired length + final node length.
## Examples
iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 5)
{"The meaning of life is...", 20}
iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 140, " ")
{"The meaning of life is... something else", -101}
"""
@spec get(html_tree, number, binary) :: {binary, number}
def get(html_tree, max_length \\ 255, sep \\ "") do
get_text(html_tree, {"", max_length}, sep)
end
# initial piece of text
defp get_text(text, {"", max_length}, _sep) when is_binary(text), do: {text, max_length - String.length(text)}
# all other text
defp get_text(text, {acc, max_length}, sep) when is_binary(text) do
{Enum.join([acc, text], sep), max_length - String.length(text)}
end
# deal with nodes lists and sort out over count to be correct (negative when under original max_length)
defp get_text(nodes, {acc, max_length}, sep) when is_list(nodes) do
Enum.reduce_while nodes, {acc, max_length}, fn(child, istr) ->
if max_length > 0 do
{final_acc, final_length} = get_text(child, istr, sep)
{:cont, {final_acc, -final_length}}
else
{:halt, {acc, -max_length}}
end
end
end
# ignore comments
defp get_text({:comment, _}, {acc, max_length}, _), do: {acc, max_length}
# turn BR tags into new lines
defp get_text({"br", _, _}, {acc, max_length}, _), do: {acc <> "\n", max_length - 1}
# process children
defp get_text({_, _, nodes}, acc, sep) do
get_text(nodes, acc, sep)
end
end
Trending in Discussions
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
Hi there! :wave:
@frigidcode and I (but mostly him) have been running an Elixir Book club, we’re almost done with Designing Elixir Syste...
New
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with.
But I don’t use it anymore.
Serverless was the promis...
New
What IDE or editor are you using for Elixir development?
Personally, I use Zed, and I really like it, but sometimes I wish there were a ...
New
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog
It says that Fly is going all-in on sprites, which is a worry ...
New
This was an interesting post by Jose. Curious to hear what others think. Evolving programming languages in the AI era - Dashbit Blog
Gen...
New
Hey guys!
Curiosum team has just published the State of Elixir results on Elixir Hub: see the results here.
1018 participants, 45 quest...
New
Other Trending Topics
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
In 2021 I started a new library called Tempo with the objective of modelling time as a set of intervals - not as instants. In 2022 I gave...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixirconf-us
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #security
- #performance









