State of the Beam survey

Where there’s a will there is a way :slight_smile: Worse comes to worst, you can always write a tiny wrapper module in Elixir which would use the macro functionality and serve as a proxy between the Erlang module and the macros. So again, I’m not even pretending this is nice or perfect, but it’s definitely not “outright impossible”.

I used the same approach in Elixir once to use QLC (and IIRC also for lager). So the problem in fact cuts both ways but is definitely solvable, even if the solution is not very pretty.

1 Like

That’s not what I meant. I meant something simple so we can do:

?EX_MOD(Enum):each(...)

?EX_STRUCT(SomeMod, #{foo => bar, ...})

Not sure if this can be done with macros, but I think it should be at least possible with pars transes.

With parse transforms it could be, alias’s and all. :slight_smile:

With defines I don’t think so?

But isn’t that because OCaml’s typesystem is weaker? OCaml’s HM, and Rust and C++ have turing-complete typesystems (Ruby is horrid because it’s horrid, ahah). Isn’t it a little unfair?

That also makes the turing complete type systems unsound, thus meaning that you have to type most things, which makes them far larger syntactically (and/or you get error messages like C++ template horrors ;-)). I’d prefer a sound type system with macro’s (although the closest OCaml has are compiler transforms, I.E. PPX’s, which are like parse transforms in erlang in their scope, though significantly more powerful than parse transforms due to being able to introduce new constructs, and PP’s which are able to introduce new syntax (look at ReasonML for example)) then a turing complete type system any day.

You took the characters right off my keyboard. :+1:

2 Likes

More of what I think here. :+1:

2 Likes

Firstly apologies for the delay - I’ve had a busy day :lol:

I think the best example to emphasise this is referencing Ruby on Rails. Rails came up with a lot of new ideas and packaged them with some existing ones in to what one might expect of a ‘current’, extremely handy and delightful web framework.

For example, prior to then, we needed to put together html forms by hand - then along came Rails and generated the code for you. Rails also allowed us to interact with our databases without knowing a single line of SQL. It generated boilerplate code with its generators, built in security measures and gave us sane convention over configuration defaults. It also allowed us to create DB migrations that not only updated our databases …but allowed rollbacks as well. Then there were areas, whether tech or methodologies, that it spearheaded or brought into the mainstream, TDD, REST, DRY. And much much more. It’s also worth mentioning that Rails itself was a DSL which made it extremely intuitive to work with (arguably Ruby and its syntax was responsible for this - and one of the things that made Ruby ‘modern’).

It set the standard that other web frameworks began to follow. If they lacked much of those features, they weren’t considered modern.

That’s what modern means in the context of languages and frameworks and that’s why I consider Elixir and Phoenix ‘modern’. They’re not just in line with what’s expected of a language or framework to deal with the challenges and expectations of today, but are arguably, leaders in the field, too :slight_smile:

Remember these anyone? :lol:

1 Like

Actually I’d argue that other frameworks were doing such things well before Rails. Everything from I’m pretty sure Nitrogen is older to Wt to a variety of even PHP and Perl frameworks. ^.^

Perhaps some were doing some of them, but I think Rails was the first super successful ‘complete package’.

Personal like/dislike aside, there’s no denying it took the web framework world by storm - and for about a decade most of the other successful frameworks were probably, in some way or another, inspired by it :slight_smile:

1 Like

Hmm, the first complete mega-build like that for open source use might actually be Django, which predecesses ruby on rails by about a half year. Although there were many others before it, Django reached a monstrous level of use and is still in heavy use today. It was built very modular so you built things as kind of plugins, it was an interesting style but not very performant, like rails. Not really any reason for them to not be performant, either of them, just bad designs I think?

1 Like

My comment about the difficulty of “calling” Elixir from Erlang had nothing to do with calling functions which of course is (relatively) easy as at the call level they are basically the same language. However, you care to look at it Elixir is a “skin” on top of Erlang/OTP, sometimes thick and sometimes thin.

The problem is more about using packages designed for Elixir from Erlang, or the other way around. Many of the better Erlang applications are behaviour based, you need to provide a callback module which will be called at certain times and whether this is written in Erlang or Elixir is irrelevant just as long as it does what it is supposed. It does not look like many of the more interesting Elixir packages follow this design.

An example I mentioned is cowboy and phoenix. Cowboy works through callbacks which makes it easy to use from any language. I have tried with Erlang, Elixir and LFE and all work without problems. The interface to phoenix however seems very Elixir specific, the fact that you mention writing a wrapper reinforces this. The first question that comes to my mind is whether this is a generic wrapper for all uses or must be specific for each special case?

This all started around a discussion of the Erlang and Elixir communities and whether they are drifting apart, whether it is a good or bad thing, and what we can do about it. I think that the ease in which “one side” can use things developed for/by the “other side” very much influences how interesting the “other side” is. The easier it is to use things from either side the more there will be a positive interaction between the communities. which I think would be a good thing.

6 Likes

And my point was that since you can invoke functions, you can in fact use many things from the Elixir ecosystem in Erlang. A couple of examples which come to mind and could be useful for erlangers:

  • Enum module from stdlib for rich iteration and manipulation of enumerables (e.g. lists, maps, streams)
  • Stream module from stdlib for lazy consumation of enumerables
  • Task module for starting OTP compliant one-off jobs
  • GenStage behaviour for producer-consumer pipelines with backpressure
  • Connection behaviour for managing remote connections

The interfaces for these things are not based on macros, so they can be easily invoked from Erlang. Thus, I don’t agree that anything created in the Elixir space is useless for Erlangers.

Phoenix is indeed more difficult to use from Erlang, since its interface is heavily based on Elixir metaprogramming features which are obviously not available in Erlang. That said, where there’s a will, there’s always a way :slight_smile: Let’s try to serve some requests from Phoenix using an Erlang module.

First, we need to create a Phoenix project:

$ mix phx.new --no-ecto --no-html --no-brunch my_site

Then, we need a small generic Elixir-Erlang plug bridge:

defmodule ErlangPlug do
  def init(opts), do:
    Keyword.fetch!(opts, :mod).init(opts)

  def call(conn, opts), do:
    Keyword.fetch!(opts, :mod).call(conn, opts)
end

Now, we can create a plug module in Erlang:

-module(my_controller).

-export([init/1, call/2]).

init(Opts) -> Opts.

call(Conn, _Opts) ->
  'Elixir.Phoenix.Controller':text(Conn, <<"Hello from Erlang!">>).

And finally, in the endpoint module we can replace the call to Elixir router with the call to our plug:

defmodule MySiteWeb.Endpoint do
  # ...
 
  plug ErlangPlug, mod: :my_controller
end

Granted, the experience could be better. There is some room for improvements in some libraries, most notably in plug, which currently makes it impossible to use Erlang modules as plugs. But with a little bit of trickery we can still work around that problem, and get to the point where we can write most of our Phoenix code in Erlang.

I absolutely agree with this. I also agree that some Elixir libraries are harder to use from Erlang. If a library provides its interface via macros, you obviously can’t use it directly from an Erlang module, so you need to have some adapter.

However, if a library is based on plain modules and functions, then the usage is straightforward. I don’t have any data, but my impression is that many libraries don’t provide a macro interface, which means that there should be quite enough of interesting Elixir things that could be easily used by Erlangers.

Of course, as has been repeatedly mentioned in this thread, in order to simplify the integration, Erlang build tools such as rebar3 and relx should support Elixir dependencies.

So tl;dr:

  • It is possible to invoke Elixir from Erlang.
  • Many libraries have plain functional interface, and can therefore be easily invoked from Erlang.
  • If a library provides a macro-based interface, then you need some bridge/proxy Elixir code.
  • The support for using Elixir deps in Erlang projects is AFAIK currently missing, which is IMO the biggest obstacle to using Elixir from Erlang.
5 Likes

I think what he’s trying to demonstrate is that using erlang libraries from elixir do not require such shims (sometimes you have to drop into quotes in elixir due to limitations in elixir’s syntax, like using the ASN1 lib generated files from Elixir is an utter pain and have to use unquote everywhere), but elixir libraries are hard as heck to use in erlang without shims.

I can of course see reasons why, erlang lacks good macro’s, and since they do not exist elixir was not able to plug into them, and without macros the interface becomes much harder or less efficient or significantly more wordy, so there are reasons. But what he’s getting it as the biggest libraries so require macro’s that it is about impossible to use them without shims or writing a crap-ton of boilerplate.

This is why I am very much for macro’s becoming a, oh, Core Erlang feature. ^.^

Also, as for your erlang plug, you should not need the shim for that, plugs are basically just an init/1 and call/2 functions, not even macro’s, so it should be useable straight? Now ‘using’ a plug in erlang is more trouble as there is no way to call at compile-time the init data and cache it but instead would have to carry it around in the state with the rest of the plugs, which you could pretty easily do too. Plug overall is pretty easily useable from Erlang, Phoenix and Ecto not so much.

Agreed, though there are exceptions (e.g. qlc). But that shouldn’t be surprising since Elixir is an extension of Erlang in the sense that for most constructs in Erlang you can find an equivalent in Elixir. OTOH, macros are not available in Erlang, so you clearly can’t use them directly in the Erlang source code.

However, and this is one of the points I’m trying to drive through my example, you can integrate a macros based interface somehow, even if it’s less than perfect. In particular, you can implement most of your system in plain Erlang and put a Phoenix layer on top of it, with a small fraction (which is mostly going to be declarative) of Elixir code. So Phoenix (and e.g. Ecto) can definitely be used from Erlang.

Moreover, we seem to be constantly revolving around Phoenix here, as if there are no tons of other libraries (and many modules from Elixir stdlib) which don’t have macro-based interface. And those libraries can be used with no shimming at all.

Unfortunately, this is an issue of the PlugBuilder.plug macro, where plug :foo is interpreted as the local foo/2 function, so it can’t be used to specify the Erlang module. See here for details.

1 Like

You could also do plug {:some_erlang_module, [], true} too as I recall. You should be able to pass in any set of options that the pipeline option in compile/3 accepts as I recall?

We are actually in agreement here. It is the last point which is the most critical as it is generally this what you would like to do from the Erlang side and many of the potentially interesting deps have macro based interfaces.

It should be possible to write a macro based LFE interface to these deps. You still end up having to write an interface package but it would at least be generic. :grin:

3 Likes

Looks like the survey may have sparked some interest/a few conversations, such as:

https://twitter.com/joeerl/status/911646823248523265

https://twitter.com/joeerl/status/911691185042989059

https://twitter.com/3xplus1/status/911727762406363137

I actually quite like Brian’s idea. Wonder if the Elixir core team have thought about adding that… (So in the absence of a ^, it will pipe to the first argument as normal (so shouldn’t break existing code)).


Also nice to see Joe playing with Elixir again. Robert has you beat tho Joe, he’s been here for time :003: (Where’s Mike these days? C’mon Mike jump in :lol:)

https://twitter.com/joeerl/status/911533445179899904

https://twitter.com/joeerl/status/911645923838742528

Please don’t… This requires you to mentally count arguments in your head.

This is the way to go. It should be _ instead of ^, though (also, x |> f(y, ^, z) is not valid Elixir syntax). There are two a libraries named pipe_to and pipe_here that already make it possible if you want.

EDIT: Just noticed that the comment was talking about Erlang, I don’t know enough erlang to say what is or isn’t valid there… What I’ve said applies to Elixir, though.

2 Likes

I’m not keen on numbered arguments either :023:

Not sure I like the underscore version tbh - we use that when we want to say we don’t care about the value.

I like ^ because it points upwards, so in a way visually indicates what came before (appreciate it’s not currently valid syntax but maybe it be worked into core somehow?) (I’m not a language designer so have no idea myself.)