Huge ecommerce backend migration - is Phoenix the right choice?

So I know how PHP7 is fast, because I was working with it:

And I continue to say is not as fast as Elixir, and mile-seconds count under load, but PHP will crumble under load.

Regarding scaling you said:

But has I said, that’s is viable only for who is willing to literally burn money,

Regarding databases I can agree that they can take the most chunk of time in a request, but once more PHP as overhead in the ORM layer, that Ecto doesn’t seem to have, plus in some situations you can take advantage of Elixir to execute parallel queries to a database.

So while PHP 7 is fast for non high traffic situations, is far from being the right tool for high traffic situations.

1 Like

Crystal is exceptionally non-parallel though. It’s concurrent sure, in the vein of nodejs or Python with GIL threads, but it is exceptionally non-parallel last I checked?

2 Likes

ORMs in many of these “slower” languages are often a performance hit, but that’s not the language’s fault. If performance is a concern, you can drop the ORM. For example, right now I’m working with a Python app that’s not using any ORM because they didn’t want to pay that performance cost. The tradeoff is complexity in the application, but they felt that was warranted.

AWS bills for these languages are usually not that bad for the servers (they’re usually high for the databases - it’s always the database!). I’m working on a team that has a Java service and an Erlang service, both of which have traffic in the 1000s of requests per second, and the AWS bill for the Java app is only €335/month (roughly 2x) more than the Erlang app. We even did load testing on these apps today and found that with this current setup, we could be handling roughly 10,000 RPS without even hitting 80% CPU usage, so the server isn’t much of a cost carrier here.

Even for slower languages that need 10x the servers, it’s not that bad in reality. This is another one of those situations where €600/month is 10x(!) more than €60/month, but to a business that €540/month difference is just a rounding error in their monthly cost. For example, the total AWS bill for the company I’m working for right now is over €1,300,000/month.

If your company is extremely price sensitive, sure, Elixir will help. But for most companies, payroll costs are usually several orders of magnitude more than hosting costs. Cloud servers are dirt cheap these days.

2 Likes

They’re getting there. Big steps in Crystal landed recently: https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html

Hmm, those don’t look like very good results so far, it still looks like there’s a GIL around the GC and GC operations like specifying roots (they state it’s a multi-threaded GIL, but I remember the boehm collector ‘barely’ being multi-threaded, and it required a lot of locks or spinning to accomplish). The GC and GIL were the two big reasons it wasn’t able to be parallel yet, and so far it seems like the benchmark is only really doing parallel IO, which isn’t crystal work but kernel work?

1 Like

This full journey about a complete e-commerce application built with elixir/phoenix might be of interest to you, if you decide ultimately to go with elixir.

3 Likes

You made me remember of this forum post Freshcom API - an open source eCommerce project using Phoenix

2 Likes

That’s not the only available library, but we don’t have details about any companies using them in production.

3 Likes

Rant with random thoughts incoming. No particular argument against you, just got some neurons fired by the comments here and I felt like sharing the thoughts behind them. :slight_smile:

Yes they are. But we’re going to be in 2020 soon. The IT area has to start becoming truly useful and more generally applicable. Rust’s Tokio scheduler has been improved with up to 10x performance recently. Languages like Crystal, while intriguing, will inevitably become fringe curiosities in a few years if the only thing they bring to the table is a faster Ruby-like compiled language. The LLVM / IR compilers and runtimes aren’t the ultimate art in the area but they are still times faster than Crystal in real workflows no matter what a biased benchmark like TechEmpower shows – their only valuable test is the so-called Fortunes where they include various activities done in a web app; and even that doesn’t match the patterns of real web apps usage that much. (But still, it’s a respectable benchmark; much more than their others anyway.)


People praise Python left and right but without a good solid concurrency and parallelism story akin to the OTP – and in a world where only the CPU cores will increase and the frequency will likely never go beyond 10GHz per core – it will always remain just a glue for scientific or deep learning libraries and nothing much else. This, I would argue, has happened several years ago and will only continue. Serious people with tough projects reach for other languages and runtimes. Almost the same applies to PHP (a lot of old inertia and code that people don’t want to rewrite) and Ruby (quick and convenient scripting with a slow web framework tacked on top).


I am not talking about myself here but for many other people – and they are sick and tired of IT being so grossly inefficient and coming with a ton of gotchas no matter what technology you choose. It’s an eye-opener to communicate more with people outside IT. I believe that, as a consultant, you (@devonestes) are very well aware of that.

There’s such a thing as the best tool for the job and the business reality of quick buck in exchange for a tech debt is not going anywhere until capitalism itself ceases to exist. That will be a fact of life for a while still.


But when you can use such a resilient runtime like the OTP while reaching for languages that become better every month – like Rust, and maybe Zig, Nim and OCaml in the near future – when you need the full muscle of your hardware, then a lot of other languages and tech start to look inadequate and kind of superfluous in the grand scheme of things.

As I’ve shared before in this forum, as I am [hopefully] maturing as a human being and not only as a programmer, I want to achieve more while sitting on my computers – while spending less time on them. Languages like Crystal, F#, Haskell and a ton of others can teach you interesting things for sure but I wouldn’t reach for them for real production work. Same reason while I eventually rejected Racket even though I fell in love with it – it’s an amazing language in every way but… no really good transparent concurrency/parallelism like the OTP!

The only thing that truly gives Erlang/Elixir a run for their money in this department is Rust’s actix / actix-web frameworks. And even they have ways to go still. Everybody has ways to go until they catch up with the OTP, sadly. I’d kill for an OCaml with OTP, for example…


Eventually, Elixir might become a fringe curiosity language as well. I wouldn’t kick and scream about it. For now though, it’s a very solid contender even if it lags behind Rust’s actix by a factor of 13x performance in the Fortunes section of TechEmpower (and likely 100x or more behind on true muscle power). A lot of people out there like reliability and future-proofing more and not a Ferrari that can crash into a wall after it takes three turns.


/end random rant

6 Likes

Did I read that right or did you mean €1,300/month? :lol:

€1.3 million a month? :scream: Can you share what kind of app is it and how many requests per second? I think (I’m guessing) even WhatsApp’s server costs are nowhere near that :upside_down_face:

What about making use of BEAM goodies like ETS or Mnesia? For speed I’d want to utilise in-memory stores if at all possible (not sure how suitable this would be for an e-commerce shop tho).

2 Likes

I’m with you on this one. Database access tends to dominate esp. in these kinds of applications; and the OPs mention of ‘mindblowing business logic’ is suggestive of that.

@mendicm Hi there and welcome to the forums. Migrating this project (which from the sound of it is the company’s main product) to a new tech stack sounds riskier than it needs to be. Perhaps it would be safer to first try to extract small pieces of this into separate services using Elixir to get a good feel for the technology and see what production and support looks like. That could boost your and your team’s confidence before attempting to port the eshop- plus, if you’ve already broken some pieces out of it, it should be easier than trying to port all of it at once

1 Like

€1.3mil is the monthly cost for the whole company, not just any single app. It’s a very big company with thousands of developers and dozens of products. The team I’m working with spends around €5k/month on AWS for all their apps, databases and other associated infra (like EMR clusters and such).

Keeping stuff in memory is fine, but there’s little perceivable difference to a user between a Redis Cache and an ETS cache. It’s another one of those “10x faster” situation, but it’s 20ms vs 2ms. I don’t even think the difference is that big, though.

I wouldn’t use mnesia for something where losing data isn’t an option… It’s just not meant for that.

2 Likes

I know you probably don’t agree that it’s a viable comparison, but there’s a reason that most “modern” cloud architectures look an awful lot like the BEAM’s supervision and concurrency models, just on a large scale. K8s, Serverless and the like are essentially OS level BEAMs with less efficient message passing and process spawning.

If folks are on the cloud, proper parallelism matters a whole lot less in practice.

The place where it makes the biggest difference is in embedded systems, which is the last place the BEAM is essentially uniquely qualified to be. I would never tell someone to write an embedded application in anything other than a BEAM language.

2 Likes

What about complementing the platform with OTP like features? For example using F# along akka.net or protoactor sounds reasonable :shushing_face:

I understand that an incredible advantage of BEAM languages is that they already come with OTP (less configuration, faster development cycle).

Haven’t really tried it (and don’t know the trade offs fully) but CubDB sounds good enough for simple use cases.

Ah I see, that makes sense. (Although I’d consider building my own DC at that scale.)

Being able to do so much of it in Elixir/Erlang is great tho :smiley:

From Elixir In Action (Manning) (for the benefit of the OP):

6 Likes

They had their own DC, but are nearly done moving to AWS because their DC was just in one region but they’re live in multiple regions now, and they didn’t want to have to build and manage 5 data centers around the world.

2 Likes

Yes, and I discovered another one, Crimson Ecommerce by @amacgregor , but its in very early days.

2 Likes

Do you mind telling us what other options your team is considering, and why? Just for context.

Thank you

Oh, I’m just a consultant. They don’t involve me in those sorts of decisions.

Developer behind CrimsonCommerce here, been working on the ecommerce domain for 13 years or so with Magento, Shopify, Workarea, and many others.

Happy to answer any questions or bounce any ideas.

5 Likes