aphillipo

aphillipo

Hello there! I’ve been working on an Excerpt implementation on top of Floki that greedily includes tags. I’d love for the next step to be able to preserve tags but this is a good starting point; any advice on that welcome.

Let me know if there is anything I could improve or if there are any performance issues I’m encountering by using recursion for this. It’s also worth noting this is an adaption
of the DeepText search that comes with Floki for my own needs.

defmodule PayPip.Text do
  @moduledoc """
  PayPip.Text is a strategy to get text nodes from a HTML tree using a deep search
  algorithm limited to a certain length + the full content of the final node. It will 
  get all string nodes and concat them into an accumulator up to that point.
  """

  @type html_tree :: tuple | list

  @doc """
  Get text nodes of a desired length + final node length.
  ## Examples
      iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 5)
      {"The meaning of life is...", 20}
      iex> PayPip.Text.get([{"a", [], ["The meaning of life is...", {"strong", [], ["something else"]}] }], 140, " ")
      {"The meaning of life is... something else", -101}
  """
  @spec get(html_tree, number, binary) :: {binary, number}
  def get(html_tree, max_length \\ 255, sep \\ "") do
    get_text(html_tree, {"", max_length}, sep)
  end
  
  # initial piece of text
  defp get_text(text, {"", max_length}, _sep) when is_binary(text), do: {text, max_length - String.length(text)}
  # all other text
  defp get_text(text, {acc, max_length}, sep) when is_binary(text) do
    {Enum.join([acc, text], sep), max_length - String.length(text)}
  end
  # deal with nodes lists and sort out over count to be correct (negative when under original max_length)
  defp get_text(nodes, {acc, max_length}, sep) when is_list(nodes) do
    Enum.reduce_while nodes, {acc, max_length}, fn(child, istr) ->
      if max_length > 0 do
        {final_acc, final_length} = get_text(child, istr, sep)
        {:cont, {final_acc, -final_length}}
      else
        {:halt, {acc, -max_length}}
      end
    end
  end
  # ignore comments
  defp get_text({:comment, _}, {acc, max_length}, _), do: {acc, max_length}
  # turn BR tags into new lines
  defp get_text({"br", _, _}, {acc, max_length}, _), do: {acc <> "\n", max_length - 1}
  # process children
  defp get_text({_, _, nodes}, acc, sep) do
    get_text(nodes, acc, sep)
  end
end

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91898 914
New
AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4616 55835 594
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New
Herve37
We’re evaluating API mocking tools for OpenAPI-based projects and would love to hear what other teams are using. We’re particularly inte...
New
matt-savvy
Is there a word for the ~> symbol used in Version strings? Do you also just call it a Squiggle Arrow™ ?!
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews