koudelka

koudelka

I’ve just released version 1.1.1 of the Honeydew job queue library and added “instant” Ecto work queues to turn your schema into a job queue.

By adding a couple fields to your schema, Honeydew will automatically run background jobs related to your schema on your worker nodes whenever you insert a new row. For instance, if a user uploads a song file, and you want to transcode it into a number of different formats, or if a user uploads a photo and you want to run your object classifier against it.

This is done without a separate database table, and entirely without an external stateful queue process for you to worry about. Since your schema’s table is the work queue itself, they’re always in sync and issues relating to the the lack of atomicity between your row’s insertion and the enqueue are eliminated, in fact, your application code never enqueues jobs at all.

This specific queue type integrates well into the rest of the Honeydew ecosystem, supporting almost all of the library’s features (failure modes, suspend/resume, etc).

Check out the README for a lengthier description and an example project.

Hope folks find it useful, feedback is always welcome. :slight_smile:

Showing Posts 1 to 2

mbuhot

mbuhot

Awesome! Another option for transactionally consistent durable queueing :+1:
Have you tested the throughput? I’ve read that without DB specific tricks like PostgreSQL skip locked that simple update order by limit 1 queries can be a bottleneck.

koudelka

koudelka OP

Thanks for the heads up! :slight_smile:

I haven’t run any benchmarks yet, outside of hammering the queue in an attempt to flush out any flagrant concurrency issues. My first goal is to get it correct, and then optimize for speed.

I’m using it with CockroachDB, which doesn’t support skip locked, unfortunately. But I don’t think it’d be a big deal to change out which SQL statements are used based on the database (either trying to auto-detect at runtime, or requiring the user to indicate).

— All posts loaded —

Where Next? Top

Trending in Announcing Top

woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New
MRdotB
I needed to reuse React components from my Chrome extension in my Phoenix/LiveView backend. I noticed that for Svelte/Vue, there are live...
New
woylie
I released Doggo, a collection of unstyled Phoenix components. https://github.com/woylie/doggo Features Unstyled Phoenix components....
New
GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
anuaralfetahe
Hello Published a new library - ProcessHub! ProcessHub is a library designed to manage process distribution within the Elixir cluster. ...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
AstonJ
This showed up on my feed.. anyone heard of it? Just hype? Ox Alpha is a reasoning model designed for coding, sustained ag...
New
bartblast
Hey folks, I just published a post about Hologram’s funding and where the project goes next - the short version: Curiosum as Main Spons...
New
CodeSync
:microphone: ElixirConf 2026 - Call for Talks is open! We’re heading to Chicago :united_states: :round_pushpin: In person + virtual :d...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews