Getting things done with Elixir - tips?

I finished working thru “Programming Elixir 1.3”. It was theoretically interesting, but I’m left wondering how to use it to accomplish real world tasks.

I’ve been a developer for a long time. My background is in imperative procedural languages, focusing on back end apps that are highly DB, and moderately computationally, intensive.

Am I just overlooking the obvious?

1 Like

Programming Elixir teaches you the language - how you apply it is either down to you (based on your experience, perhaps specifically, in designing concurrent/parallel systems) or, based on what you learn next. Luckily, there’s a lot of material on the latter!

I have currently read/done:

And loved every single one of them (check out my reviews in their respective threads). Dave’s course shows you how to apply Elixir - and it is one of the best programming courses I have ever done!

There are other books and courses that show you how to ‘think’ in or get the most out of Elixir too, have a look through our Books and Courses sections :023:

7 Likes

You’re saying that you have imperative/procedural background. I can recommend reading this article on What’s Functional All About?.

My two cents: Not every problem should be solved in a functional way. Sometimes it is not convenient. But maybe we can locate the parts of the solution that need to store state, produce side-effects or do other non-functional stuff, isolate these parts and write functional code around them? If the answer is yes, then I’d say we can have much more readable, maintainable and testable code. And, often, more parallelizable.

To add to the resources @AstonJ mentioned, I’d like to suggest trying out Phoenix if you want to build a web thing, or Nerves if you want to build an embedded thing.

5 Likes

I think we’ve all been there with different languages and technologies. I’m sorry to hear that “Programming Elixir” didn’t teach you how to approach problem solving in Elixir. When I read it I had gone through the “Getting Started” + “Mix and OTP” series on elixir-lang.org and after those two I felt like I was properly prepared to put together solutions in Elixir. Obviously, your mileage may vary and we’re not all the same.

Do you feel comfortable with working with OTP and do you recognize what it is about and why it’s a good thing to have? Does the Zen of Erlang resonate with you yet? I think those two questions are central to solving problems on the BEAM.

On a practical note, have you (already done or) considered solving a toy problem like writing a (distributed?) logging server using Elixir? You could write a daemon that takes logging requests for different files/topics that it then stores in-memory and periodically persists to disk, and can be queried for the logs of a certain file/topic. This can start super small (only storing the given log for a topic in-memory and not persisting it at all) and not even using UDP/TCP from the beginning, maybe ending up doing a whole high-performance UDP server that uses ETS to store logs, etc…

It’s obviously hard to appreciate someone’s arbitrary suggestion for a project, but I think this is at least a problem that fits the platform and language.

Luckily, your user profile of “back end apps that are highly DB and moderately computationally intensive” is exactly what Erlang/Elixir is about, so once your thinking fits the platform you’re exactly the type of user that will love it and get the most out of it.

3 Likes

Eixir in action is really good book. Maybe best in elixir world

3 Likes

Try to develop something very simple, like an Encryptor.

Give a read to this tutorial which teaches you, how to develop an Encryptor in Ruby. Then do the same in Elixir. That’s how I practice Elxir.
Try to use IEx helper h() as much as possible.

BTW, what do you wanna use Elixir for, primarily?

1 Like

One issue with this model of learning Elixir is that you won’t necessarily learn about working on the BEAM. Ruby doesn’t allow for the same kind of development and architecture that you use on the BEAM, so there is a whole world of things out there that you might not learn about when doing this.

For sequential code that has a one to one mapping to other libraries on other platforms, I agree that it’s good to pick something you know from elsewhere and implement it, but it won’t introduce the concepts and thinking that are specific to platforms like the BEAM.

If you have something made in Smalltalk and want to implement the architecture from there that might actually transfer much better, but I doubt that applies to a lot of people.

1 Like

Yes, you’re right.
I suggested this method only because the title says “Getting things done with Elixir?” and he said that his “background is in imperative procedural languages”.
He has to learn how the OTP and it’s actor model work and other stuff, once he learns how to implement the basic stuff he learnt by using “Programming Elixir”.
And BTW he can also develop that Encrypter in a distributed way. :slight_smile:

Anyway Thank You for reminding me not to forget I’m in an environment and paradigm which is stronger than Ruby’s, and develop things that way.

1 Like

I’m hearing, “hang in there”. :smile:

I got “Programming Phoenix” at the same time as “Programming Elixir”, and I just started Phoenix. Most of my “web” work I’ve done is limited to wrapping webservices around existing methods.

Elixir isn’t supported, so I can’t even dabble with it professionally. I’m still trying to conceive how it would handle my real work.

Thanks

I think this pretty much describes where Elixir starts to make sense: http://bholley.net/images/posts/thistall.jpg

Try to write something in your current language that will use threads to use all the cores available on a typical developer laptop. Now imagine attempting to port that app to something with 128 or 256 cores. With the BEAM that scaling comes almost for free, you can just throw hardware at your code and it will work.

2 Likes

Not entirely free, you still have to make enough processes and split up your calculations enough that concurrent work can be done. That is pretty trivially easy to emulate in C++ even, you just don’t have the same reliability there. :slight_smile:

Insert semi made up quote about “Every sufficiently concurrent program includes a buggy half-implemented version of the BEAM”.

6 Likes

My concern was not the theory or benefits of Elixir. My question was about where the rubber meets the road.

In a simple real world example, you have a task to query and join a half dozen different data sources, perform a bunch of arbitrary logic and output something. In Elixir, would that simply be a set that you pipe thru a ton of functions?

1 Like

That is the essence of functional programming in the rough, yes. You have functions which take data as input and return some value as a result. So you could describe any / all functional programs in the way you have.

Where it gets more interesting in this case is in the details of the “how” with Elixir. For instance, querying a handful of data sources: this is trivial to do in parallel with Elixir by putting each query into its own process. You may choose to do this with one-off usage of Tasks, or you might create GenServers that are re-used between queries. You may even create a worker pool to limit the number of queries being made in parallel. (The latter is what Ecto does behind the scenes.)

Those processes would then preferably be monitored by a Supervisor (or by just manually linking, via spawn_link e.g.), so that if something goes wrong in one of them they can be restarted / retried, or even abort the parent process that is doing the top-level task.

Depending on the shape of the data being processed, the “arbitrary logic” you mention can be written elegantly using Elixir’s facilities for pattern matching, pipelining using the |> operator, etc.

You can then very easily turn that into a distributed application where a bunch of computers (high end servers or clusters of little RPi’s even) work on that task together.

So … while you could describe it as a “ton of functions”, and you could literally write it that way, there is a lot more in the toolbox. Personally, that toolbox added on top of FP, great tooling, etc. is what makes Elixir so exciting.

6 Likes

Pretty much - for example:

defmodule Summary  do
  defstruct order_id: 0, items: [], customer_name: "", date: Date.utc_today()

  defp cons_description(order_id),
    do:
      fn
        (%Item{order_id: id, item: description}, items) when id === order_id ->
          [description | items]
        (_, items) ->
          items
      end

  defp gather_descriptions(items, order_id),
    do: List.foldl(items, [], cons_description(order_id))

  defp find_customer(customers, customer_id),
    do: Enum.find(customers, fn(%Customer{id: id}) -> id === customer_id end)

  defp lookup_name(customers, id) do
    with %Customer{name: name} <- find_customer(customers, id) do
      name
    else
      _ -> "N.A."
    end
  end

  def summarize_order(items, customers) do
    fn(%Order{id: order_id, customer_id: customer_id, date: date}) ->
      %Summary{
        order_id: order_id,
        items: gather_descriptions(items, order_id),
        customer_name: lookup_name(customers, customer_id),
        date: date
      }
    end
end

  def from(orders, items, customers) do
    orders
    |> Enum.map(summarize_order(items, customers))
  end
end
$ elixir demo.exs
Customers:
[%Customer{id: 1, name: "Samson Bowman"},
 %Customer{id: 2, name: "Zelda Graves"}, 
 %Customer{id: 3, name: "Noah Hensley"},
 %Customer{id: 4, name: "Noelle Haynes"},
 %Customer{id: 5, name: "Paloma Deleon"}]
Orders:
[%Order{customer_id: 3, date: ~D[2014-03-20], id: 1},
 %Order{customer_id: 4, date: ~D[2014-04-25], id: 2},
 %Order{customer_id: 5, date: ~D[2014-07-17], id: 3},
 %Order{customer_id: 2, date: ~D[2014-01-05], id: 4},
 %Order{customer_id: 5, date: ~D[2014-06-09], id: 5}]
Items:
[%Item{item: "gum", order_id: 2}, %Item{item: "sandals", order_id: 4},
 %Item{item: "pen", order_id: 3}, %Item{item: "gum", order_id: 1},
 %Item{item: "pen", order_id: 2}, %Item{item: "chips", order_id: 3},
 %Item{item: "pop", order_id: 1}, %Item{item: "chips", order_id: 5}]
Summaries:
[%Summary{customer_name: "Noah Hensley", order_id: 1, date: "2014-03-20", items:
 ["pop","gum"]},
 %Summary{customer_name: "Noelle Haynes", order_id: 2, date: "2014-04-25", items:
 ["pen","gum"]},
 %Summary{customer_name: "Paloma Deleon", order_id: 3, date: "2014-07-17", items:
 ["chips","pen"]},
 %Summary{customer_name: "Zelda Graves", order_id: 4, date: "2014-01-05", items:
 ["sandals"]},
 %Summary{customer_name: "Paloma Deleon", order_id: 5, date: "2014-06-09", items:
 ["chips"]}]
$

Full code (with alternate Summary module):

# https://stackoverflow.com/questions/25438893/haskell-how-to-implement-sql-like-operations#answer-25438952

defmodule Customer do
  defstruct id: 0, name: "First Last"

  def new(id, name),
    do:
      %Customer{
        id: id,
        name: name
      }
end

defmodule Order do
  defstruct id: 0, customer_id: 0, date: Date.utc_today()

  def new(id, customer_id, date),
    do:
      %Order{
        id: id,
        customer_id: customer_id,
        date: date
      }
end

defmodule Item do
  defstruct order_id: 0, item: "Description"

  def new(order_id, item),
    do:
      %Item{
        order_id: order_id,
        item: item
      }
end

defmodule Summary  do
  defstruct order_id: 0, items: [], customer_name: "", date: Date.utc_today()

  defp cons_description(%Item{order_id: id, item: description}, m),
    do: Map.update(m, id, [description], &([description|&1]))

  defp make_descriptions_map(items),
    do: List.foldl(items, Map.new(), &cons_description/2)

  defp put_name(%Customer{id: id, name: name}, m),
    do: Map.put(m, id, name)

  defp make_names_map(names),
    do: List.foldl(names, Map.new(), &put_name/2)

  def summarize_order(items, customers) do
    descriptions = make_descriptions_map(items)
    names = make_names_map(customers)

    fn(%Order{id: order_id, customer_id: customer_id, date: date}) ->
      %Summary{
        order_id: order_id,
        items: Map.get(descriptions, order_id, []),
        customer_name: Map.get(names, customer_id, "N.A."),
        date: date
      }
    end
  end

  def from(orders, items, customers),
    do: Enum.map(orders, summarize_order(items, customers))

end

defimpl Inspect, for: Summary do
  import Inspect.Algebra

  def inspect(summary, opts) do
    concat [
      "%Summary{customer_name: ",
      to_doc(summary.customer_name, opts),
      ", order_id: ",
      to_doc(summary.order_id, opts),
      ", date: ",
      to_doc(Date.to_iso8601(summary.date), opts),
      ", items: ",
      to_doc(summary.items, opts),
      "}"
    ]
  end
end

defmodule Demo do

  defp make_customers,
    do: [
      Customer.new(1, "Samson Bowman"),
      Customer.new(2, "Zelda Graves"),
      Customer.new(3, "Noah Hensley"),
      Customer.new(4, "Noelle Haynes"),
      Customer.new(5, "Paloma Deleon")
    ]

  def to_date(year,month,day) do
    with {:ok, date} <- Date.new(year, month, day) do
      date
    else
      _ -> Date.utc_today()
    end
  end

  def make_order(id, customer_id, year, month, day),
    do: Order.new(id, customer_id, to_date(year, month, day))

  defp make_orders,
    do: [
      make_order(1, 3, 2014, 3, 20),
      make_order(2, 4, 2014, 4, 25),
      make_order(3, 5, 2014, 7, 17),
      make_order(4, 2, 2014, 1, 5),
      make_order(5, 5, 2014, 6, 9)
    ]

  defp make_items,
    do: [
      Item.new(2, "gum"),
      Item.new(4, "sandals"),
      Item.new(3, "pen"),
      Item.new(1, "gum"),
      Item.new(2, "pen"),
      Item.new(3, "chips"),
      Item.new(1, "pop"),
      Item.new(5, "chips")
    ]

  def run() do
    customers = make_customers()
    orders = make_orders()
    items = make_items()

    IO.puts("Customers:")
    IO.inspect(customers)
    IO.puts("Orders:")
    IO.inspect(orders)
    IO.puts("Items:")
    IO.inspect(items)

    IO.puts("Summaries:")
    IO.inspect(Summary.from(orders, items, customers))
  end

end

Demo.run()

One thing to keep in mind is that functional programming is “value-oriented programming” while imperative programming is largely PLace-Oriented Programming (PLOP) due to the fact that it heavily relies on mutable state (locations). See Rich Hickey’s Value of Values talk.

Ultimately value based computations compose better (again Rich Hickey Simple made Easy).

The preceding demonstration code is entirely sequential. As already mentioned with BEAM languages exploiting concurrency relentlessly is always an option. For example for the lifetime of the script:

  • one process could steward the customer list and be responsible for looking up the customer name by customer_id.
  • another process could steward the items list and be responsible for serving the list of item descriptions for a particular order_id.
  • each order record could be serviced by it’s own process, returning the completed summary upon termination.
6 Likes

I wouldn’t buy anymore books. You are a kindred spirit as practical is how I really learn and 90% of the books on programming SUCK for that. I found Google better for finding practical tutorials and Udemy had a slightly helpful video series.

Google is great when you just want to get something done. Books have their use too. They can help to answer questions you didn’t even know to ask.

6 Likes

Who said that quote? Looks like I will say this quote in some of my Quora answers in near future!

After reading the blogs about how good Erlang’s concurrency model is and how we just just made a super implementation of it in XXX I have been led to formulate Virding’s First Rule of Programming:

Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.

With all respect to Greenspun.

The quote is an adaptation of Greenspun’s Tenth Rule that says:

Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

6 Likes

Apologies for necroing to the mods but this discussion is interesting. :023:

The way I educated myself with Elixir professionally pretty quickly was to try and replicate parts of, or the entirety of, my recent past projects in other languages (in my case Ruby and Go). I still sometimes write and rewrite functionality for a small e-commerce app I was allowed to keep the source of (but forbidden to distribute) and I keep teaching myself various aspects of Elixir’s more heavy-duty tooling through it (like Flow and GenStage). I managed to make report generation there 17x - 25x faster than the Ruby counterpart, to create a small macro-based DSL for the CMS section, rewrote some of the REST APIs in GraphQL (through Elixir’s excellent Absinthe library) and made them tens of times faster without caching… examples abound.

You come off like a practical and somewhat skeptical person (being skeptical is a healthy habit). So IMO for you the best way would be to make direct analogies: if you have the free time, try to rewrite some of your professional apps or libraries in Elixir and, of course, feel free to ask a lot of questions along the way. We are here to help.