That’s exactly my current thinking. Keep things super simple - but v useful perhaps.
mmport80
mmport80
DBs are v good at many things, but are also a little bit complicated.
I suspect many apps don’t need to scale to need DB amounts of data.
For that niche, it’d be nice to have a simpler solution, eh?
0
Qqwy
TypeCheck Core Team
Maybe it’s important to take a step back, and let us consider what a ‘data base’ actually is:
It’s just some place in which you store (some of) your data.
Whether this is inside your (OS) application or outside of it does not really matter that much from a nomenclature point-of-view.
Historically, data base-frameworks have grown to become their own separate (OS) applications because this meant they could be started, stopped and versioned separately from the main (OS) application.
In a world like the BEAM which is its own operating-system like environment, it makes a lot of sense to work with a data base that runs inside of this system, because of:
- No separate serialization/deserialization-steps necessary when reading/writing from/to the data base.
- Distribution is possible in the same way as your normal application.
- No two (OS)-processes competing for the attention of the (OS) schedulers; rather, they can be (semi-)preemptively scheduled by the BEAM schedulers.
- Less mental complexity than when using two separate technologies that have to talk to one-other.
However, the idea to run a data base inside the BEAM is somewhat new, or at least widely unexpored territory: There is of course (D)ETS and Mnesia built on top of that, and then we have riak_core (but using the normal riak data base inside your own BEAM instances is not possible AFAIK). CouchDB was also not built with in-BEAM support in mind.
So Mnesia, riak_core and lasp are our only current contenders for distributed in-app data bases, and none of these are easy to get started with.
But if you don’t need to go to distributed scale, you can just use ETS, or even simpler: A process that periodically backs up its state using :erlang.term_to_binary and writes it to a file.
2
mmport80
That’s what I am thinking about.
Everyone focuses on planetary scale distribution. But in many cases, the usual use-case is quite small…
2
tristan
Rebar3 Core Team
Do you just mean how grains are stored?
There isn’t any, you can just as easily use a NoSQL store for grain persistence. Postgres is what it supports right now just because that is what we use at work. Orleans supports a number of different databases for grain persistence.
3
mmport80
Nope, when reading up about it, it seemed that Orleans might be getting close to NoSQL land.
I.e. indexed, but largely unstructured storage of data.
1
ricosuaveyatusabe
The default phoenix generator will create an application with ecto - the reason is to give the user something “with batteries included”. You can easily remove this by passing the --no-ecto flag to the mix phoenix.new generator.
Extracted from:
1
tristan
Rebar3 Core Team
quatermain
Hi, I know it’s 3 years old topic but somehow I have similar issue. I need “in-house” small data storage which has to be distribute between multiple apps(2 kind of apps with multiple instances) to share state. I’m thing about mnesia and Cachex for example. I wonder how you ended up and what solutions are available in these days.
Main reson for me to find this kind of storage it’s because I have two apps:
- api app
- show app
You can create structure with api and when you create you can show it with shop app. We use postgresql right now to store these information. It works well, until postgresql server is not available and we have to prepare for bigger traffic. So for example when we lost connection with psql for 3 seconds we can lost about 300-500 api calls. Right now I see only one solution and that’s distributed in-house(in-beam) storage which will handle it and async storing in posgresql for other use. I’m not sure if I need persistent storage for that kind of in-house db. I see only one reason for it: when some entries are not saved into postgresql and some app crash. But distribution can help with it.
Cachex is nice cache solution. But I have feelings that using it for my use case is not right and can make me more other problems. This kind of cache is more cache for already stored data, not pre-save cache I think.
Mnesia is battle tested, but I need only key-value storage where I store some information. I take token from URL, decrypt ids, load data by that ids and use data to show call 3rd party APIs and show informations.
1