aphillipo
Hello there! I’ve been working on an Excerpt implementation on top of Floki that greedily includes tags. I’d love for the next step to be able to preserve tags but this is a good starting point; any advice on that welcome.
Let me know if there is anything I could improve or if there are any performance issues I’m encountering by using recursion for this. It’s also worth noting this is an adaption
of the DeepText search that comes with Floki for my own needs.
defmodule PayPip.Text do
@moduledoc """
PayPip.Text is a strategy to get text nodes from a HTML tree using a deep search
algorithm limited to a certain length + the full content of the final node. It will
get all string nodes and concat them into an accumulator up to that point.
"""
@type html_tree :: tuple | list
@doc """
Get text nodes of a desired length + final node length.
## Examples
iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 5)
{"The meaning of life is...", 20}
iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 140, " ")
{"The meaning of life is... something else", -101}
"""
@spec get(html_tree, number, binary) :: {binary, number}
def get(html_tree, max_length \\ 255, sep \\ "") do
get_text(html_tree, {"", max_length}, sep)
end
# initial piece of text
defp get_text(text, {"", max_length}, _sep) when is_binary(text), do: {text, max_length - String.length(text)}
# all other text
defp get_text(text, {acc, max_length}, sep) when is_binary(text) do
{Enum.join([acc, text], sep), max_length - String.length(text)}
end
# deal with nodes lists and sort out over count to be correct (negative when under original max_length)
defp get_text(nodes, {acc, max_length}, sep) when is_list(nodes) do
Enum.reduce_while nodes, {acc, max_length}, fn(child, istr) ->
if max_length > 0 do
{final_acc, final_length} = get_text(child, istr, sep)
{:cont, {final_acc, -final_length}}
else
{:halt, {acc, -max_length}}
end
end
end
# ignore comments
defp get_text({:comment, _}, {acc, max_length}, _), do: {acc, max_length}
# turn BR tags into new lines
defp get_text({"br", _, _}, {acc, max_length}, _), do: {acc <> "\n", max_length - 1}
# process children
defp get_text({_, _, nodes}, acc, sep) do
get_text(nodes, acc, sep)
end
end
Trending in Discussions
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
New
Hey there,
It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Quite interesting article Google brought me. Didn’t find any mentions about it here.
What do you think in general? Would you use togethe...
New
:warning: Security advisory: Decimal DoS vulnerability
A vulnerability has been published for decimal where very large exponents can cau...
New
It would be helpful to have a list of companies worldwide that hire engineers without prior experience in Elixir. Often, it can be quite ...
New
Anyone running long-lived stateful processes on BEAM? We’re building an AI agent runtime and would love to compare notes.
We’re a small ...
New
Other Trending Topics
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself.
My main conc...
New
Aludel - LLM Evaluation Workbench
Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming










