ityonemo
Spinning this off in a new thread because I think it’s interesting and merits its own discussion. Agree with @hauleth that the basis should be spinning up a c-node inside of julia, but it’s going to need a bit more than that. A few questions:
- How to manage Julia versions to run on
- Julia has a --worker node setting, should we use that?
- How do you load up the correct dependencies? Also, julia annoyingly uses global dependencies, more like python than elixir, which has project-level dependencies.
- Is there something really high performance that one could do like memmapping memory segments and passing data across that way? How do you detect if that capability exists? (windows? Linux? MacOS? etc)
Separately, would anybody be interested in doing a collab on this for SpawnFest 2020?
Where Next?
Trending in Discussions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 29 to 20- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
Changxin
Oh, Thanks!
I’ve been working on a reasoning system using
Elixir/OTP(I’m really enjoy it !!!). The system should have been connected to the robot hardware, but for testing purposes a simulated environmentWebotswas chosen.Unfortunately, webots doesn‘t have a language interface for Elixir. So, I use
Juliacall itsdlland transfer robot status toElixirbySocket.Well, the status information also includes camera images, so
msgpackmay be a good choice, just like whatAirSimdoes. But the images are represented byStringand its resolution is relative small, soJSONis used by the system.I want a good interoperability between
JuliaandElixir, both are great programming languages! However, now,Socket+JSONorMsgPackmay be more realistic.josevalim
If you are sending textual data, we didn’t see a large improvement between JSON and Erlang.
Also note that sending Erlang terms to a server is insecure.
So if you are sending binaries and want efficiency, I would go with msgpack or similar.
Changxin
Yes,
Socketis more flexible thanStantard IO, emm, I have a question that when communicate with Erlang bySockets,Json FormatorErlang External Formatwhich one is preferred?dokie
Yeah no worries, I took it is the spirit it was intended
ityonemo
haha sorry that was more of a shower thought. Unfortunately the julia worker interface is not very well documented, so I would say, for a team that has to deliver a product it’s probably not worth the hyperoptimization yet, but you know, it would probably be “fun to look into”. That sort of a thing.
dokie
Good point and for some of our Julia services it might be a good option. I will discuss this idea with the team and see it we can add it to our backlog. Cheers
ityonemo
I think though you can configure julia workers to listen on a socket with a much more lightweight protocol (and even instruct K8S to use that protocol as a liveness check), without having to drop in such a heavy system. I guess it depends on what your latency requirements and how long each job takes; if your julia jobs are on the order of milliseconds you might care; if your julia jobs are on the order of minutes or more HTTP is a-ok and probably easier to debug if something goes wrong.
dokie
Unfortunately our architecture of the platform we have built on k8s is better handled by having the 2 runtimes and services written in the 2 languages on different pods so we can scale independently
hcs42
We actually started doing this before 1.0… and it was not trivial. Then the situation improved with 1.0 and the new package compiler, which we started to use. But it still required some non-trivial work. We haven’t modified this part of our code since then, so I’m not sure how much changed since 1.0.
Good luck with your next project!
ityonemo
Sorry I wasn’t able to figure out the spawnfest registration. I was in the process of evaluating if I should quit my job due to some unexpected developments, and that consumed a lot of energy.