halostatue

halostatue

Prosody - Text analysis for Reading Time - Extension for Tableau

Over the last several months, I’ve been using Tableau to rebuild my long-neglected website. Tableau is part of the Elixir Tools project headed by @mhanberg, and is built on top of MDEx by @leandrocp. This library is one of ten libraries and Elixir extensions that I built. Most of these were built with the assistance of Kiro.

Prosody is a simple content analysis library that attempts to measure the reading effort and cognitive load for mixed text and code content — like a technical blog. It provides a Tableau extension, Prosody.Tableau, that will return the reading time for code blocks.

It implements text processing in three stages:

  • parse where content is parsed with awareness of its content type (extensible, just like Tableau itself) to convert content into block types (text or code, initially)
  • analyze over the blocks to count words, with some cognitive load adjustments applied to code blocks
  • summarize over the analysis results to aggregate into reading time and metrics.

The current analysis is limited to word counting, but other analysis could be performed in the future (such as calculating content complexity such as the Flesch-Kincaid readability score) and steps could be added to merge blocks prior to analysis.

Counting words is more complex than one would imagine, as the text two words and/or fast-paced 1,234.56 www.example.com will produce 6, 8, or 12 words depending on the algorithm used. The default “balanced” algorithm produces 8, but this is configurable.

Example

content = """
# Hello World

This is some text.

```elixir
IO.puts("Hello")
```
"""

Prosody.analyze!(content, parser: :markdown)

This generates:

%{
  code: %{words: 10, lines: 1},
  text: %{words: 6},
  metadata: %{},
  words: 16,
  reading_time: 1
}

https://github.com/halostatue/prosody

Where Next?

Popular in Announcing Top

OvermindDL1
I created a new library (rather I pulled out a couple files from my big project), it manages an operating system PID file for the BEAM. ...
New
seancribbs
Today I released a new dialyzer Mix task as the dialyzex package! At the time we started writing this task, the existing dialyzer integra...
New
brainlid
LangChain is short for Language Chain. An LLM, or Large Language Model, is the “Language” part. This library makes it easier for Elixir a...
New
mbuhot
Leverage Open Api 3.0 (Swagger) to document, test, validate and explore your Plug and Phoenix APIs. Generate and serve a JSON Open API ...
New
Qqwy
Hello everyone, I wrote a small library today called MapDiff. It returns a map listing the (smallest amount of) changes to get from map...
New
mindok
What is ContEx? A pure Elixir server-side data plotting/charting library outputting SVG. It has nice barcharts in particular and works g...
New
josevalim
Hello everyone, We have just released NimbleCSV which is a small and fast CSV parsing library for Elixir. It allows developers to define...
New
mplatts
With HEEX released we decided to start a components library using Tailwind CSS - check it out here: Petal Components. We also have a boi...
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New
trisolaran
Hi! :waving_hand: I would like to present LiveSelect, a little library that I wrote to easily add a dynamic selection input to your LV f...
198 10978 107
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement