aschrijver

aschrijver

Sqlite in Production

(Moved to separate topic from: https://forum.elixirforum.com/t/lower-the-initial-barrier-by-using-ets-and-not-postgresql-for-phoenix-examples/53287 )

FWIW I’ve seen sqlite and related projects (e.g. litestream) getting rave reactions on HN the past 1.5 year or so (even today there’s such discussion), and what I find intriguing in many of the comments I read was people using it in production. So much discussion in the tech community comes from startups and Big Tech who need hyperscale, and vendors playing into this. Many average web apps don’t need all that. I am interested myself in small technology, and sqlite is a great fit there. I think more sqlite focus would not only be attractive wrt onboarding, but for actual use as well.

First 10 of 18 Posts! Switch mode

kwando

kwando

Thats true :slight_smile: but gives you enough to get started.

josevalim

josevalim

Creator of Elixir

Yes, the litestream stuff is exciting and if they finally remove the single-writer limitation with hc-tree, it would be huge. I do have a question though: how do you deal with backups in such cases? Because there is an argument for simpler deployments with sqlite3 but that inevitably pushes some concerns to you, no?

aschrijver

aschrijver

Yes. I could not give you a good answer now. Maybe a comment on HN, now the discussion is still active?

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

FYI Moved these sqlite specific bits out to their own thread, I think the suitability of Sqlite in production is a super interesting topic right now, but I think is a bit tangential to the original thread.

josevalim

josevalim

Creator of Elixir

Apparently Litestream can replicate to S3/Google/etc: Install - Litestream

I also found this for a lightweight cron-based approach: Cron-based backup - Litestream

LostKobrakai

LostKobrakai

There are also some sqlite options without litestream. From the point of having a separate file of the db it’s mostly a matter of putting that file somewhere safe.

D4no0

D4no0

The only thing that scares me is the fact that you need to lock the file, it would be nice to be able to backup partially without having to lock.

dimitarvp

dimitarvp

SQLite has its own backup functionality as linked by @LostKobrakai – it’s not just copying the files though – and a lot of energy has been expended to prove that it works stably.

Additionally, it also has the concepts of patch-sets and change-sets (section 2.1) – somewhat to the confusion of us in the Elixir space due to the naming clash – that allow you to accumulate changes since a last known snapshot, in a dedicated file.

SQLite has a strong backup apparatus even without litestream. The latter seeks to make backups over the network more or less transparent. The rest is well-covered by sqlite itself.

dimitarvp

dimitarvp

That’s already covered. You issue a backup command and SQLite itself makes new file(s) while you’re using the same session. It’s fully atomic.

People get needlessly scared there, their team is very aware of the potential issues and took care of them a while ago.

kwando

kwando

It is not recommend to backup sqlite by just copying the file. You want to use the backup and restore commands.

From command line backup can be done like this, super simple.

$ sqlite /path/to/db.sqlite ".backup /path/to/file.bkp"

For most of my pet projects it is absolutely fine to use sqlite, it is plenty fast and super reliable. I cant say postgres ever caused me any big troubles, but the simplicity of sqlite is just great when you can get away with it :slight_smile:

Last Post!

iFire

iFire

I am trying to use ecto with mvsqlite which is a remix of sqlite that allows clustering.

Here’s my work in progress GitHub - V-Sekai/elixir-mvsqlite: A mvsqlite driver for Elixir · GitHub, I also enabled sqlite strict mode in ecto.

It is only enabled for msvc.

Where Next?

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2976 91332 914
New
f0rest8
Hi everyone :waving_hand: Posting here to showcase and announce that Metamorphic is now officially live on a public-facing domain at htt...
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
zachdaniel
Introducing AshStorage! Attachment and file management that slots directly into your resources :smiling_face_with_sunglasses: I had hope...
New

We're in Beta

About us Mission Statement