PragTob

PragTob

How to make an Elixir library available to erlang users?

Hello Elixir friends,

I want to make one of my Elixir available/usable for our Erlang friends and be a “good BEAM citizen”. I was also told, that there isn’t really a good benchmarking library for Erlang so it seemed like a great idea.

Problem: I can’t figure out how to install Elixir libraries in Erlang. Threads like Writing a library for use in both Elixir and Erlang as well as the prominent Erlang/rebar3 usage on hex.pm made me think this was possible (although I always sort of wondered where it gets all the Elixir from)…

I setup a little test respository. However, benchee can not be found and neither can any other popular elixir hex package I can think of:

tobi@speedy ~/github/benchee_erlang $ rebar3 update
===> Updating package registry...
===> Writing registry to /home/tobi/.cache/rebar3/hex/default/registry
===> Generating package index...
===> Writing index to /home/tobi/.cache/rebar3/hex/default/packages.idx
tobi@speedy ~/github/benchee_erlang $ rebar3 compile
===> Package <<"benchee">> not found. Fetching registry updates and trying again...
===> Updating package registry...
===> Writing registry to /home/tobi/.cache/rebar3/hex/default/registry
===> Generating package index...
===> Writing index to /home/tobi/.cache/rebar3/hex/default/packages.idx
===> Package not found in registry: <<"benchee">>.
tobi@speedy ~/github/benchee_erlang $ rebar3 pkgs | grep poison
tobi@speedy ~/github/benchee_erlang $ rebar3 pkgs | grep phoenix

(btw. yes I have the rebar3_hex plugin installed but I think it does something else)

Adding benchee as a git dependency I get this error:

tobi@speedy ~/github/benchee_erlang $ rebar3 compile
===> Verifying dependencies...
===> Fetching benchee ({git,"git://github.com/PragTob/benchee.git"})
===> WARNING: It is recommended to use {branch, Name}, {tag, Tag} or {ref, Ref}, otherwise updating the dep may not work as expected.
===> Dependency failure: Application benchee not found at the top level of directory /home/tobi/github/benchee_erlang/_build/default/lib/benchee

It’s from this rebar3 issue - basically I don’t follow the standard rebar3/erlang project structure and yeah I’m an elixir project, so sure it doesn’t work.

So what do I do? Elixir and Erlang/rebar3 both use hex.pm and elixir people can use Erlang stuff but not the other way around? Do I just not know how?

How can I allow an Erlang project to use an Elixir library? Do they have to copy the files and compile them themselves to beam files? Do I setup a separate rebar3 package?

Sorry my erlang knowledge is also rather… improvable :slight_smile:

I feel like (and hope) I’m really missing something so please feel free to point it out. Thank you in advance :tada:

First 10 of 15 Posts Switch mode

idi527

idi527

A bit off-topic, but some erlang programmers would want to avoid using any elixir libs because elixir needs to include it’s standard library at runtime, so if someone wants their library to be used by all languages on beam, it might be better to write it in erlang. In your case though it probably doesn’t matter, since it’s a benchmarking library.

PragTob

PragTob OP

Should they avoid it for that reason, then their loss :slight_smile: It’s also not used at run time, so there is no impact on the runtime release that gets distributed :slight_smile:

The library has a non trivial code base by now so I’m not gonna rewrite, just trying to be a good BEAM citizen.

josevalim

josevalim

Creator of Elixir

I believe you need something like rebar3_mix or rebar3_elixir for it to work. Try pinging t_slought/tristan or alisdair on elixir-lang IRC channel.

PragTob

PragTob OP

Thanks, will do/try! :slight_smile:

PragTob

PragTob OP

Update: Thanks to @josevalim ( :green_heart: :tada: :star2:) I found rebar3_elixir_compile which seems to do what I want it to with the following configuration:

{deps, [
  {benchee, {elixir, "benchee", "0.8.0"}}
  %% {benchee, {git, "git://github.com/PragTob/benchee.git"}}
]}.

{plugins, [
    { rebar3_elixir_compile, ".*", {git, "https://github.com/barrel-db/rebar3_elixir_compile.git", {branch, "master"}}}
]}.

{provider_hooks, [
  {pre, [{compile, {ex, compile}}]},
  {pre, [{release, {ex, compile}}]}
]}.

{elixir_opts,
  [
    {env, dev}
  ]
}.

It fetches all dev dependencies atm and values other than the dev environment don’t seem to work. Will ask there. Seems to work now:

tobi@speedy ~/github/benchee_erlang $ rebar3 shell
===> Verifying dependencies...
===> Upgrading benchee ({elixir,"benchee","0.8.0"})
Dependency benchee already exists

Running dependency resolution...
All dependencies up to date
===> App bunt is no longer needed and can be deleted.
===> App credo is no longer needed and can be deleted.
===> App deep_merge is no longer needed and can be deleted.
===> App dialyxir is no longer needed and can be deleted.
===> App earmark is no longer needed and can be deleted.
===> App ex_doc is no longer needed and can be deleted.
===> App fs is no longer needed and can be deleted.
===> App mix_test_watch is no longer needed and can be deleted.
===> Compiling benchee_erlang
===> Adding Elixir Libs
Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V8.1  (abort with ^G)
1> 'Elixir.Benchee':init().
#{'__struct__' => 'Elixir.Benchee.Suite',
  configuration => #{'__struct__' => 'Elixir.Benchee.Configuration',
    assigns => #{},
    formatter_options => #{console => #{comparison => true,unit_scaling => best}},
    formatters => [#Fun<Elixir.Benchee.Formatters.Console.output.1>],
    inputs => nil,
    parallel => 1,
    print => #{benchmarking => true,configuration => true,fast_warning => true},
    time => 5000000,
    warmup => 2000000},
  jobs => #{},
  run_times => nil,
  statistics => nil,
  system => nil}

Thanks a lot!

If someone has other ideas/ways please still let me know :slight_smile:

Crowdhailer

Crowdhailer

Creator of Raxx

It might be time for a new thread on this but I am wonder what the resolution to this thread was. What is the best way to

a) use Elixir libs from an erlang project
b) structure an Elixir project so it is easy to use from erlang

PragTob

PragTob OP

I think rebar3_elixir_compile is still the goto solution. Albeit no one ever responded to the issues I created but there have been some updates.

My test repo works some of the time - albeit I also needed to escriptize and I’m not too familiar with rebar3/escriptize so others might to a better job than me.

From my perspective for use cases such as benchee (i.e. you don’t need something at runtime but rather just a tool) it’s easiest/most reliable to just install elixir use the library and then call to erlang code from there. I’d be happy for a better solution though.

At Lambdadays I talked quickly with Michał and José and it sounded like it’d be nice to do something to make it easier and that it shouldn’t be that hard but also not a top priority right now.

Crowdhailer

Crowdhailer

Creator of Raxx

Thanks for the test repo. I hope it is something that can be prioritised soon. I would really like to see the beam community as people like to call it become more cohesive

josevalim

josevalim

Creator of Elixir

Thanks @PragTob! I want to clarify that people should not expect the Elixir team to solve this. We already do all of the work to ensure you can use rebar/rebar3/make from Mix, the other direction is not quite up to us but rather to people using those other build tools. We will be glad to answer any questions or improve Elixir if necessary but it is up to the community to tackle the problem together.

tristan

tristan

Rebar3 Core Team

Where Next?

Trending in Questions Top

jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
silverdr
Using Phoenix.LiveView.TagEngine as an EEx.Engine is deprecated! To compile HEEx, use Phoenix.LiveView.TagEngine.compile/2 instead. Sta...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
michallepicki
I am using Oban and occasionally, shortly after a deployment, a handful of jobs can fail because of dependency on other parts of the syst...
New
rahultumpala
Hello, I have an Elixir backend that implements a custom protocol over TCP. I want to load test the backend and assess the performance o...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New

We're in Beta

About us Mission Statement