eDev
Erlang/Elixir and supercomputers
This guy didn’t get the memo about Erlang!
“As a computer scientist it’s difficult writing software that can take advantage of and control large numbers of computer cores,” said Professor Les Carr from the University of Southampton.
"This is why supercomputers are restricted to specialised applications - you need very specialised computing needs to take advantage of them.
In all seriousness - would BEAM be a good fit for super computers like these?
Most Liked
bbense
I started out my career working at the San Diego Supercomputer Center on some of the very early multiprocessor supercomputers of the day (i.e. when a 1000 cores was a supercomputer). My interest in concurrency started there and eventually lead me to Erlang and Elixir.
While there are some simulations that could effectively use the BEAM, most supercomputer type problems don’t translate well to the actor model due to the connectivity of the data. For large physical simulations, every data point affects the value of every other data point in the simulation.
The Actor model assumes a network of minimal connections. If every process needs to talk to every other process then communication dominates costs as you scale the problem.
Now on the other hand, almost all of the science of numerical computation is figuring out when it’s “okay” to ignore the connections between data in a problem and partition the problem into smaller solvable problems.
I think the BEAM could work in these environments as scheduler and coordinator, but you’d need a specialized core of mathematical routines that currently doesn’t exist.
uranther
These supercomputers are working on parallelizable problems, yes, but need raw speed such as for modeling climate change. They might even use specialized languages for the computations. A BEAM language may be good for supervising the process farm but other than that there needs to be mutable data structures and fast matrix computations.
Qqwy
I really do wonder what would happen if we had a pure dependently typed actor-model functional language. This would enable us to use crazy tricks like process-based memoization, and have the compiler do all kind of weird things for us to make programs faster.
It is completely possible to use mutable data structures in a purely functional language by using a State Monad Transformer (But I have to admit, this is the kind of stuff that Mathematics Professors discuss about with squiggely lines and vague terms – I don’t understand how this works myself either right now).
Last Post!
pat_rick
This question was raised in June 2016. Would the answers to this question today (October 2023) be different than they were back then?
Popular in Discussions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex










