I have developed already three Applications with Phoenix and always used the PostgresDB - now i start a large Project used by many cities in my country and it feels wrong to use PostgresDB for a highly scalable Language and Application. I have already used Riak with Elixir and would be cool if the new Phoenix-Project would use it. I also used Mnesia for some caching, but never in Mnesia with Phoenix.
I know ChrisMcCord has >=1 Phoenix App running with Riak. I do not know if it is a good Idea to run Phoenix with Riak and also how I also found an Ecto Adapter (https://github.com/pma/riak_ecto) for Riak - anyhow I wasn’t able to find a Doc or some good Example Code. I would also love to use the feature that data does not have to have a strict structure like in SQL Table (Variable Fields)
I’m also open for some new Ideas and Suggestion btw the App maybe won’t Hit the Postgres Query/Data Limits but it will be a huge App which it would be awesome to test Riak, and i don’t want to scale Postgres over 2-3 Hosts (again)
RethinkDB looks nice too - just wanted to try Riak The write performance of Rethink won’t be a problem. Any Phoenix Project with RethinkDB out there and OpenSource to look at?
I’ve never used Riak before, but I am interested as well.
At the moment I work on a phoenix project with Redis, which is very similar to Riak. The main idea of Redis is to offer basic data structures like lists, hashes, string, etc. The advantage of Redis as an in-memory database is the performance. The advantage of Riak as a distributed database is the reliability and the high availability.
At the beginning I chose Redis only for a partial aspect of the business logic. I have some sets of entities, on which I have to search very fast to offer some kind of real time feeling.
When you want to use a database like Redis or Riak you should be aware that you have to do everything on your own. A simple example is user management. In a RDB you have a users table which you can ask for a single entry by username or by id or by whatever. In Redis you could store the user entry in a hash - let’s say users:1, where users is something like the name of the “table” and the number 1 after the colon is the user-id. So, if you want the user with the id 123 you pick users:123 and you are done in O(1). But you can’t search in a Redis-hash. If you want to search by username or by email address, you have to build the indexes by your own. If you search for the username foo, such a index could be a Redis-string with key username:foo and the value is the user-id.
Finally I would like to recommend the book “Redis in Action” by Josiah Carlson. You will find a lot of ideas and inspiration how build more complex structures on the Redis data structures. It is still very helpful for me.
I have worked a lot with Redis while i was developing in Ruby and RoR, do you know the feeling when you would like to start something new?
My Idea behind this project is a Project everything in ErlangVM - This is not a Project with a Deadline or Strict planning so i just wanted so see whats Possible inside the Elixir/Erlang Ecosystem.
You were saying you can’t find example code for the Riak adapter but isn’t the purpose of Ecto to be like an interface for the DB? You should be able to use the Riak adapter just by calling Ecto functions and all that like when using it for Postgres, no?
I am about to use Riak as my database, and am building various applications, one being a Phoenix app.
So, I am seeking opinions about how to use Riak in Phoenix
My rationale for using Riak is Solr, it’s easier to figure text analyzers compared to Postgresql and better language support.
I was just playing with https://github.com/drewkerrigan/riak-elixir-client last night and found it to be acceptable, although documentation is rather poor, like mysterious. But then again I suppose you can always call the erlang riak directly and maybe that’s what people usually do. Just my 2 cents.
I am using Elixir and Riak with the Erlang client in production and it works well for my use case. The Elixir client wasn’t available at the time, so I wrote my own simple wrapper in Elixir.
However, the Elixir client looks fine and well maintained, so if I would have to choose now, I would go for the Elixir client. As for documentation, I think you can easily use the documentation for the Erlang client and apply most of it to the Elixir client.
If you have any specific questions, I am happy to answer them.
Actually Riak has secondary indexes built in, they are part of the metadata that you can add to a row, at least the last time I used it, which was… oy probably a decade ago, I’m assuming Riak has developed even further since then, I just use PostgreSQL nowadays, I’ve not had a need for an untold monstrous amount of data munging needed that would make something like Riak shine and at the scales I work (~1tb/data) PostgreSQL well outperforms it.
Also Redis is pretty worthless, especially if you have PostgreSQL, as PostgreSQL can do everything Redis can do, and better, and can do a whole lot more!
Also, Riak is natively erlang, you can make native calls, though those are documented as breakable between versions and you have to run in-node, but even the client calls are utterly trivial.
Ecto is built more for relational DB’s, for things like Riak, which is more of a KV-Store + Secondary Indexes + FileStorage + Concurrent MapReduce + plus plus plus, well Riak is an entirely different beast.
Now an up to date post. ^.^
I’d call Riak’s Client directly if you want to make an ecto interface. Also, Ecto2 made a huge paradigm shift, it will be mostly if not entirely a rewrite, I’d probably just rewrite.
Jut call the erlang client, the elixir one is probably just a thin wrapper around it anyway and erlang is native in elixir since they are just different syntaxes on the same core.
I’m curious of this. In my testing Riak is only suited for utterly massive read-writes and distributed mapreduce, potentially as a KV file storage as well. These are not use-cases I’ve had as of yet that PostgreSQL’s been unable to fulfill for me yet.
I chose RIak because I had to store a potentially unbounded amount of log events that need to be aggregated from time to time, generated by a web application. For this, I think Riak is a good use case, but I use Postgres and Ecto for everything else.
Thanks for sharing. I see, typical time series data. There is a trend of using KV store for time series data. Postgresql has also improved its support for that recently.
Thanks or your reply. Right, I could create a new Ecto interface, but why? I am relatively new to Phoenix, and I do realise that if I rip Ecto out, I loose Changesets and most of the built-in functionality in the Model, But how much of that really makes sense if I use Riak in any case?
I am quite happy to call the Erlang interface directly, and question the value of Ecto for a database like Riak.
For distributed storage. Our application is not like a typical Rails backed website. Its a http protocol API providing JSON to a Javascript application running in a web browser. A small part of the app will also serve up html, css, js to kick-start the Javascript client, but mostly what it will be doing is REST. I don’t want the headache of replicating a PostgreSQL database (maintaining mirrors, WAL etc). If I have to do that, its like towing a semi-trailer behind a chain-ganged line of Leyland Minis.
I also found an Ecto Adapter (https://github.com/pma/riak_ecto) for Riak - anyhow I wasn’t able to find a Doc or some good Example Code. I would also love to use the feature that data does not have to have a strict structure like in SQL Table (Variable Fields)
The write performance of Rethink won’t be a problem. Any Phoenix Project with RethinkDB out there and OpenSource to look at?
