Donovan

Donovan OP

Hello everyone,

I’m so glad to have discovered this awesome community. Thanks for creating it! This is my second post, and apologies for any inadvertent breach of protocol by posting this (a variation of a prior reply to a related topic) as a separate post.

I’m just learning Elixir and have encountered a few pronouncements that Elixir is slow and may not be well suited to heavy data/analytics applications. At the same time I’ve seen references to Elixir conference presentations on Big Data applications. As a result I am seeking calibration on the kinds of math-oriented/analytics apps are well suited to Erlang.

A specific use case: I am considering using Elixir/Phoenix for the backend of a Mint-like web app. The app will principally track budget/actual entries, perform predictive / Monte Carlo / scenario modeling and produce datasets for front-end rendered charts.

I’m excited to learn and apply Elixir to this task; but would like to qualify the suitability of Elixir/Phoenix to this type of app.

Thoughts?

First 10 of 31 Posts Switch mode

easco

easco

Elixir, like Java and other systems, complies down to a byte code that runs on a virtual machine. As such low level operations like floating point math usually go through a level or two of indirection. My understanding is that Elixir, Erlang, and their peers may not be the most performant option for number-intensive calculations.

What Elixir and Erlang thrive at is parallelism. In the case of Big Data applications you typically see gains from that kind of parallelism. Running really intensive calculations may not be a forte, but running a bunch of them at the same time could be a win.

Donovan

Donovan OP

Thanks @easco. This is helpful.

I’m not sure if a Mint-like app is considered math intensive or not; but with multiple simultaneous users each viewing scenarios based on on their unique data sets, it’s possible that the app may benefit from Erlang’s multi-threaded/parallel processing capabilities.

I think the math can be characterized as mostly arithmetic and time value of money operations applied to tables/arrays of data up to 1000x1000.

I suspect there is a threshold of math/computation-intensivity where Erlang is not well suited. I’m trying to get a working sense of that threshold, and an appreciation of the classes of apps that fall beyond that threshold.

Thanks again for your comments!

andre1sk

andre1sk

For something like Mint Elixir is going to be totally fine the number crunching is least intensive of the tasks you will have to perform, polling all the APIs for transaction data and rendering out interfaces will be much higher % of your workload.

StefanHoutzager

StefanHoutzager

I don’t know what a mint-like app is. But concerning number-crunching: you can call modules written in languages suited for that from elixir. For example: I call R modules using Rserve (Rserve - Binary R server - RForge.net) using erlang libraries (GitHub - del/erserve: Erlang/Rserve communication interface · GitHub GitHub - del/erserve_pool: Connection pooling for erserve, the Erlang/Rserve interface. · GitHub). So you can combine the strengths of different languages.

Donovan

Donovan OP

Thanks @andre1sk. That’s what I thought; but needed validation absent any calibration of what math/computationally-intensive means in the Elixir context.

Donovan

Donovan OP

Thanks for the insight and example @StefanHoutzager.

Mint.com is a web based budgeting and personal finance tracking application that was acquired by Intuit a few years ago. Mint is the equivalent of Quicken on the web but better and more user friendly.

The target app I’m evaluating Elixr/Phoenix for is not as robust as Mint but does provide basic budgeting, personal finance tracking, analysis and heavy use of graphs to provide guidance.

My thinking is to use Elixr/Phoenix for the backend (database, analytics and preparation of graphing-ready results datasets, API, etc.) plus something like React for the front end (user interaction and reports rendering, etc.).

Thoughts?

josevalim

josevalim

Creator of Elixir

When we say Elixir may not be suited to do number crunching, we are usually thinking a bit beyond analytics, averages, medians, etc. Because it is not necessarily the mathematical processing that hurts but rather the lack of support for large mutable data structures that would allow us to do implement things like image processing or support vector machines efficiently.

For example, think how you would implement a 100x100 matrix in Elixir and how much copying you would need to do to change a single {x,y} pair using immutable data structures. If you need performance, your best bet is ETS or falling back to C (which is what many languages do anyway).

If you are worried about analytics on something like Mint, Elixir/Phoenix should be fine.

15
Post #7
Donovan

Donovan OP

Thanks Jose!

I really appreciate you taking the time to provide such an informative response. I am still learning Elixir syntax and features (before tackling Phoenix) and will add ETS and incorporating C libraries into Elixir/Erlang to my reading list. Can you suggest an Erlang/Elixir finance library or point me in a fruitful direction?

Thanks again!

StefanHoutzager

StefanHoutzager

For numerical computing f.e. R enables people to be more productive (higher level language) than C. R itself is written mostly in C iirc. Small sideremark.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Are you agreeing or disagreeing with what Jose said? By “fallback to C” I took Jose to simply mean that most languages end up using C anyway when needing to write optimal number crunching code. R is a good example of precisely that phenomenon.

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91898 914
New
AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4616 55835 594
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
mudasobwa
While I am working on the Language Agnostic Code Audit SaaS, which uses MetaAST (spoiler: I am expecting it to be in a good shape for ann...
New

We're in Beta

About us Mission Statement