halostatue

halostatue

Prosody - Text analysis for Reading Time - Extension for Tableau

Over the last several months, I’ve been using Tableau to rebuild my long-neglected website. Tableau is part of the Elixir Tools project headed by @mhanberg, and is built on top of MDEx by @leandrocp. This library is one of ten libraries and Elixir extensions that I built. Most of these were built with the assistance of Kiro.

Prosody is a simple content analysis library that attempts to measure the reading effort and cognitive load for mixed text and code content — like a technical blog. It provides a Tableau extension, Prosody.Tableau, that will return the reading time for code blocks.

It implements text processing in three stages:

  • parse where content is parsed with awareness of its content type (extensible, just like Tableau itself) to convert content into block types (text or code, initially)
  • analyze over the blocks to count words, with some cognitive load adjustments applied to code blocks
  • summarize over the analysis results to aggregate into reading time and metrics.

The current analysis is limited to word counting, but other analysis could be performed in the future (such as calculating content complexity such as the Flesch-Kincaid readability score) and steps could be added to merge blocks prior to analysis.

Counting words is more complex than one would imagine, as the text two words and/or fast-paced 1,234.56 www.example.com will produce 6, 8, or 12 words depending on the algorithm used. The default “balanced” algorithm produces 8, but this is configurable.

Example

content = """
# Hello World

This is some text.

```elixir
IO.puts("Hello")
```
"""

Prosody.analyze!(content, parser: :markdown)

This generates:

%{
  code: %{words: 10, lines: 1},
  text: %{words: 6},
  metadata: %{},
  words: 16,
  reading_time: 1
}

https://github.com/halostatue/prosody

Where Next?

Popular in Announcing Top

OvermindDL1
I created a new library (rather I pulled out a couple files from my big project), it manages an operating system PID file for the BEAM. ...
New
mspanc
I am pleased to announce an initial release of the Membrane Framework - an Elixir-based framework with special focus on processing multim...
New
ostinelli
Let’s write a database! Well not really, but I think it’s a little sad that there doesn’t seem to be a simple in-memory distributed KV da...
New
dbern
I’m excited to announce that TaxJar has developed and open-sourced DateTimeParser. We developed it because we found a need to parse user ...
New
bryanjos
Hi, I just published version 0.23.0 of Elixirscript. https://github.com/bryanjos/elixirscript/blob/master/CHANGELOG.md Most of the chan...
New
kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
kip
Image is an image processing library for Elixir. It is based upon the fabulous vix library that provides a libvips wrapper for Elixir. I...
622 18934 194
New
Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Thank...
New
anshuman23
Hello all, I have been working on my proposed project called Tensorflex as part of Google Summer of Code 2018.. Tensorflex can be used f...
New
mplatts
With HEEX released we decided to start a components library using Tailwind CSS - check it out here: Petal Components. We also have a boi...
New

Other popular topics Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement