Show HN: Lightweight Task queue on Erlang/OTP, SQLite-backed, no overengineering

69 pointsent1c3d3 days ago10 commentsView on HN

Setting up Kafka or such enterprise oriented software with their clusters or dedicated servers is heavy and bothering enough that most small teams or indie hackers skip it entirely and making compromise to use in-memory queues.

I wanted something in between: a persistent queue that is simple to run (one binary, which makes one sqlite db), gets real fault isolation and crash recovery due to Elixir, easy to inspect (open ezra.db in any SQLite browser and see every task), and requires no new client library - it speaks the Redis Streams wire protocol, so any Redis client in any language just works out of the box.

Very short demo video: [https://www.youtube.com/watch?v=MLYyD3DVWmE]

neoecos19 hours ago

Oban is really awesome, are you inspired by it?

skrebbel19 hours ago

Title said “no overengineering” so I doubt it.

xlii16 hours ago

I would argue that Oban isn't overengineered.

If so should we also consider PostgreSQL overengineered?

It's a shame OP decided to use Elixir as base, many ecosystems don't have mature task queues (e.g. for Rust I had to roll my own: simple_queue) so the space IMO would be more welcoming.

On OTP doubt anything can even make a dent in Oban user base.

gamache15 hours ago

Satisfied former Oban user here. Oban is engineered. Your use-case may be petty, though, like your comment.

cpursley19 hours ago

This is nice. For those wanting to stay on Postgres for DAG type of workflows, check out pgmq based PgFlow: https://github.com/agoodway/pgflow

tenwz119 hours ago

good

abrookewood19 hours ago

Congrats on the launch. Using the Redis protocol was a pretty clever choice. Does it have to run as a stand-alone server?

bhaney15 hours ago

> This project is maintained by a single author and pull requests are not accepted

Save yourself the headache of people not reading this and just disable pull requests in the repo settings

memset13 hours ago

This is really cool! I built something very similar only I replicated the AWS sqs protocol instead.

https://github.com/poundifdef/smoothmq

hankbond10 hours ago

Maybe this says more about me than your project, but I find the use of implementing this with the Redis Streams wire protocol very interesting and creative. Being able to leverage the surface area of existing client SDKs should really help adoption and reduce what you have to maintain in order to widely integrate.

I agree that in many areas there often is not much in-between "roll yourself in-memory" and "enterprise grade maxed out scalability focus".