Elixir In Action Book Club!

EDIT: References to Chapter 9 emended to Chapter 10
Wanted to post this for those working on Chapter 10’s exercise.

I did not like my initial answer to Chapter 10’s exercise with respect to the :ets implementation for simple registries.

I did not understand why using Process.flag in the register/1 function prior to :ets.new_entry would result in exits being captured in a way that could be handled by the server process as per @sasajuric delineation (I’m still not sure on that facet btw). However, my inital alternative to that was to keep the Process.flag in init and have the module call a GenServer.cast @mod, {:register, pid}, having the resultant handle_cast call Process.link pid.

I felt like this may have created a bottleneck, though an ostensibly wide one, out of the registry process, since it was once again burdened with some degree of registration processing.

I could not think of how else to get the process to handle :EXIT messages, I wanted to link the Processes from the client end using Process.link(Process.whereis(@mod)) within the ETSR.register/1 function, but this was ineffective at first, which was not surprising as this just linked the registry process to the caller…and I was calling this from the shell!

Handles exits by ets registered process, but requires some handling of registrations by GenServer:

defmodule ETSR do # (ETSR is short for ets registry)
  use GenServer
  @mod __MODULE__

  def start() do
    GenServer.start(@mod, nil, name: @mod)
  end

  def register({pid, name}) do
    case Process.alive?(pid) do
      true ->
        GenServer.cast(@mod, {:register, pid})
        :ets.insert_new(@mod, {name, pid})
      _ ->
        IO.puts("Process not active")
    end
  end

  def whereis(name) do
    :ets.lookup(@mod, name)
  end

  @impl GenServer
  def init(_) do
    :ets.new(@mod, [:named_table, :public])
    Process.flag(:trap_exit, true)
    {:ok, %{}}
  end

  @impl GenServer
  def handle_cast({:register, pid}, _) do
    Process.link(pid)
    {:noreply, nil}
  end

  @impl GenServer
  def handle_info({:EXIT, pid, _reason}, state) do
    :ets.match_delete(@mod, {:_, pid})
    {:noreply, state}
  end
end

Then, finally and thankfully, the truth of the exercise dawned on me.

The process itself is to call the ETSR.register({pid, :name}), rather than the call being made from the shell. I’m not sure why I thought that these calls were to be made from the shell, but I’m thankful to have understood the question properly now.

Working implementation without bottleneck:

defmodule ETSR do
  use GenServer
  @mod __MODULE__

  def start() do
    GenServer.start(@mod, nil, name: @mod)
  end

  def register({pid, name}) do
    case Process.alive?(pid) do
      true ->
        Process.whereis(@mod)
        |> Process.link
        :ets.insert_new(@mod, {name, pid})
      _ ->
        IO.puts("Process not active")
    end
  end

  def whereis(name) do
    case :ets.lookup(@mod, name) do
      [] -> nil
      result -> result
    end
  end

  @impl GenServer
  def init(_) do
    :ets.new(@mod, [:named_table, :public])
    Process.flag(:trap_exit, true)
    {:ok, %{}}
  end

  @impl GenServer
  def handle_info({:EXIT, pid, _reason}, state) do
    :ets.match_delete(@mod, {:_, pid})
    {:noreply, state}
  end
end

Still not entirely sure why we are to place and Process.flag(:trap_exit, true) in the register/1 function, when I tried that approach (probably incorrectly) the exits were not being handled.

This is the test I performed on the working version:

iex(1)> ETSR.start
{:ok, #PID<0.116.0>}
iex(2)> pid1 = spawn(fn -> ETSR.register({self(), :proc1})
...(2)> Process.sleep 10_000
...(2)> end)
#PID<0.117.0>
iex(3)> ETSR.whereis :proc1
[proc1: #PID<0.117.0>]
iex(4)> ETSR.whereis :proc1
nil  #After 10 seconds

Just wanted to post this in case anyone is pulling their hair out over that exercise due to a silly misunderstanding.

I take pid and match to pid1 just so I can check Process.alive? pid1, however, I did not need to. I may try the benchmarks later BGG.

1 Like

I am back with Chapter 6.

Chapter 6

Straight forward one. It teaches about GenServers. And instead of directly jumping down to behaviour, it lets us appreciate it by making one of our own from scratch. 6.1 makes use of our sequential Elixir, and the newly acquired concurrency primitive skillset(s) and makes us build something that does roughly the same thing 6.2 (aka GenServer) will automate.

I think this is a good way to teach most OTP concepts, I recall having a fear of GenServer-s when I first encountered it, it spawn and friends were easy to understand but how do I go from the knowledge of spawning a process to making a whole serving system out of it? This approach gave me aha moment when I first encountered this chapter years ago, it gave me an aha moment now when I read it again.

Chapter 6.2 talks about the “opinion behaviour” (Yes I prefer the -our, I’m Canadian). If 6.1 received (no pun intended) justive, then 6.2 will reward you more.

One thing that helped me grok the parameter/return contract of GenServer is this cheatsheet … it becomes second nature eventually (perhaps?) but this could help out if you’re new and trying to keep the function signatures and returns in your short-term memory.

All in all, a great chapter that deals with the heart of OTP.

1 Like

Quick question, how are y’all reading this book? PDF? Dead tree Edition? Livebook?

So I am on a rampant learning path and was reading a bundle of Manning books and since I didn’t own all of them, so I was using Live Book feature of Manning’s.

The highlight feature had been heavily used and it was quite useful for me in the journey. If reading from a browser is your cup of tea, you might try the live-book feature and see if it is of any use? (If you bought the book, you’d get the live book edition).

Also, this live-book != Elixir Live Book, and I don’t work for Manning (i.e. not advertising).

I’m a bit behind everyone else as I started a couple months afterward, but I’ve completed chapter 4 now. From what I’ve read thus far, it’s no wonder why this book is so highly recommended. Sasa is a wonderful teacher. I’m excited to continue with the rest of the book.

I do have one question regarding chapter 4 Collectable protocol. With our protocol implemented and the ability to use Enum.into/2, would it still be common or expected that TodoList.new accepted an enumerable in which to initially populate our list?

For example, my original CSV importer needed a version of new that accepted an enumerable:

  def import(filename) do
    File.stream!(filename)
    |> Stream.map(&String.trim_trailing(&1, "\n"))
    |> Stream.map(&create_entry/1)
    |> TodoList.new()
  end

But, with Collectable defined, I changed it to:

  def import(filename) do
    File.stream!(filename)
    |> Stream.map(&String.trim_trailing(&1, "\n"))
    |> Stream.map(&create_entry/1)
    |> Enum.into(TodoList.new())
  end

Does the Elixir community lean heavily into this Collectable protocol? I.e., should I not bother accepting the enumerable to TodoList.new as we’ve implemented Collectable? I also changed my implementation of new to use Enum.into instead of Enum.reduce.

Thanks!

1 Like

Regarding changing from Enum.reduce to Enum.into, I think you have more flexibility using reduce than using Enum.into for situation where you want to do some more processing; however if using the Enum.into is an option then that means you simply wish to take an enumerable and use it to populate the empty struct, so it is better to use Enum.into as you have already done the work to streamline such a process by implementing the protocol.

I would also not pass TodoList.new() to the Enum.into in the pipe chain, because what’s happening here is that you are relying on TodoList to output an empty %TodoList{} struct and then do a further operation to put the enumerable into the empty struct.

I may get disagreement here, but I think it’s better to just show what your are passing the enum into, and empty %TodoList{} struct.

Just my thoughts.

I hope I understood your question.

Welcome to the forum.

Thanks.

I conflated my questions. The main question I’m trying to understand: was adding support for passing an enum into our ‘new’ function superfluous since we later implemented Collectable (into)?

Not really, because you may not want to do…

enum
|> Enum.into %TodoList{}

…everywhere that it’s needed. You might want to kick start the struct with some entries, in which case you could wrap the snippet above in a TodoList.new func. Indeed passing a map to the TodoList.new is touched upon in one of the chapters IIRC.

Setting a default value for that parameter automatically creates two functions with arity n and n+1. (n being the arity without the additional arg)

I actually keep the feature of passing a set of entries to the TodoList.new/x in my own implementation as I have been writing along with the book.

1 Like

Concerning Chapter 11

:observer.start/0 error
Anyone having trouble getting : observer.start() to work due to an error

** (UndefinedFunctionError) function :observer.start/0 is undefined (module :observer not available) :observer.start

Please refer to this post by @benstepp.

:poolboy.child_spec/3 warning

I got a contract break warning from my :poolboy.child_spec/3 call. Apparently that call should have failed, I tried to resolve the warning, but ended up proceeding with no issues in the shell.

1 Like

Chapters 12 and 13 were bittersweet.

I aim to give a more detailed breakdown of chapters 9 - 13 a little later, BGG.

(Which means I’ll likely leisurely reread them for the nth time.)

The reason 12 and 13 are bittersweet is because they are very well structured, esp true for 12, and they’re good at what they are ostensibly intended to do.

12 gives you an entry into building distributed systems and directs you to where you need to go in order to make them more robust, sophisticated and pragmatic.

13 is more of a tour of tools, and really let’s you now that (in I’d say around two instances I’ve highlighted a line of code and just asked “what is this??”). The chapter gets away with it because it is replete with directions for where to find more information to expound on the material covered in the chapter.

Despite the fact that it is a tour-of-a-chapter, the first half very much puts you in a position to at the very least set-up local and remote releases, as well as interact with them.

That being said 12 and 13 drove home the fact that there is much more to building and maintaining distributed systems, which I expected. I understood, from prior cursory reading, that these were large topics, but I had hope for some unreasonably out-of-the-box quick fixes.

You can, and do, build a web server from the ground up using this book.

(Should you feel a little uncomfortable with the basic aspects of programming in Elixir, you can look at the book Programming Elixir, as it contains many exercises, and discusses much of fundamentals at length and from another perspective.)

Nothing wrong with even building a distributed application from this (which you do tbh), and encountering the stumbling blocks alluded to in these chapters, as it will serve as an opportunity to abstract away a lot of the context of the book and solidify the knowledge gained from reading.

I plan on doing the very same BGG.

I am not in a hurry to build a distributed system right now, but this is one of my main goals in coming to elixir; another being mastery and effective use of its Macro system. for which, @sasajuric has a brilliant guide. I will be reading Metaprogramming Elixir, BGG, to see if knowledge from Sasa’s guides can be augmented there.

How I read the book:

Speed read.
Comprehensive read (just understanding what I read with notes on what is unclear) Intention is of course to use the book as a reference when building other projects, so as to gain an abstract (applied) understanding of the topics covered in the book.

This is a very valuable book, and I don’t think my praise of it’s contents really does it justice.

You don’t just get elixir info in this book, you get some very nice programming practices and deep appreciation for what is actually happening behind the scenes which makes problem solving and strategising much less awkward and daunting.

Happy Sunday.

Got sidetracked with work and life in general but catching up again now.

Finished Chapter 6 - overall a good straightforward chapter which nicely follows on from Chapter 5.

Thanks - very handy!

I’m reading as a PDF which works well as I can have a split screen with my code editor on one side and the PDF on the other. But I’ve recently been considering buying a bigish ereader - haven’t figured out which one just yet.

Hello, I’m getting a lot out of this book as I learn and start to use elixir. I have a question about a statement in 11.1.6 “Mix Environments.” The book says “you’re advised to support running a prod-compiled version locally. Commit to this goal from the start of the project because doing this later, after the codebase grows significantly, is typically much more difficult.” I’d appreciate more information about what it means to support a prod-compiled version locally, what typically may need to be done to achieve this goal, and what makes it more difficult down the line. Are we talking about defining local environment variables, spinning up a local “prod” database? General tips welcome. Thank you!

1 Like

Here’s an incomplete list of some problems I’ve experienced in various projects:

  1. No comprehensive list of env vars that must be set.

  2. The system fails to boot if some remote service is not available.

  3. Obtaining the credentials to remote service accounts may require some bureaucracy. In some cases it might not even be possible, depending on the company policy and the availability (or lack) of a staging or per-dev accounts.

  4. It’s unclear which other steps need to be done (e.g. generate ssl keys, start other supporting local services) for the system to start and function properly.

  5. Conflation of prod-compiled with running on prod machine. E.g. a prod-compiled version should not hardcode the site URL nor the URL of remote services, because it means that I can’t easily run it locally.

These aren’t difficult problems, but once the codebase grows, they are trickier to solve, because there will be a bunch of such small problems scattered around. Furthermore, addressing some of them will require some coordination with other teams (e.g. ops).

As a result, folks will take the path of least resistance, and not try to make it work. Then, when some problems arise which can’t be reproduced on a dev version, folks will try to debug it remotely (e.g. on staging or prod). This works,but it’s much more cumbersome than trying things out locally, and experimenting with potential fixes.

OTOH, if you commit to this goal from the day 1, it should be easier to set things up, because you’re dealing with a smaller codebase, a smaller team, and a smaller set of problems. This property becomes a part of the team culture and philosophy, so it should be much easier to keep it as the team and the codebase grow.

In summary, what I like to see from a project:

  1. Starting a prod-compiled version locally should be dead simple. Give me 2-3 commands I need to run, and it should work out of the box, using my local database.

  2. The system should work even if there is no network connection. Obviously, the features that depend on external services, such as AWS, can’t work. But e.g. serving local content, interacting with the local db & such should work.

  3. It should be clear how to get the credentials to remote services. Ideally, each dev should get their own sandbox account, if possible. If the system depends on other in-house services it should be easy to start these locally.

  4. If the prod is containerized, building and starting a local container should also be possible and straightforward.

6 Likes

This is very helpful, thank you so much!

I have recently started learning Elixir… Like I am just 3 weeks in… Also India still doesnt have the 3rd edition available & I didnt want to pay almost $40 shipping so got a friend to carry it from US… And Im reading the digital version now…

As I am learning, coding & reading all in parallel lines are blurry about what I learnt from book & stuff from anywhere else… but no doubt the reading is effortless & very natural… Shows the clarity of thought which was already visibile in the Soul of Erlang video by Sasa Juric (which actually made it a no-brainer to get this book)…

I want to learn more about Phoenix & also how I can tweak things for a chat app we are building… sadly other books are atleast 5 year old… From what I understand Elixir & Phoenix have evolved rapidly in the last 5 years… so thanks for working on a new edition…

Will take better notes & update chapter wise questions & notes for discussion here…

The way Chapter 1 starts with Macros had me a little confused but researching more on the topic helped me acheive a lot of clarity… so as a begineer, the book is nudging me towards the right questions & expanding on stuff from there… Couldnt ask for more :slight_smile:

Im on Chapter2 now & its interesting to know that most of the stuff in Elixir is just common conventions. like modules naming with dot notation format or variable names with ? or !.

Presently on the section about Typespecs. I am a big fan of Typescript, I hope it will be the same for Typespecs :slight_smile:

Learnt about Tail vs non-tail recurssion & refernceto using GOTO just took me back to the days of programming in BASIC. Nice stuff :ok_hand:

In section 9.2.3, the way of checking if to-do server process exists is to attempt to start it with registering using via tuple. If registering fails with “already registered” reason we know we can use existing process. Section mentions 2 inefficiencies:

  1. “Every time you want to work with a to-do list, you issue a request to the supervisor, so the supervisor process can become a bottleneck. Even if the to-do server is already running, the supervisor will briefly start a new child, which will immediately stop.”
  2. notion of distributed registration in chapter 12, which itself in section 12.2.2 states that “unconditional registration attempt can become a serious bottleneck. Every time you want to work with a to-do list, even if the server process is already running, you attempt a :global registration, which will in turn grab a cluster-wide lock and will then chat with all other nodes in the system.”

So even in a non-distributed scenario, supervisor bottleneck is possible? E.g. in a chat application with many rooms, where each message in each room will trigger supervisor request and unconditional registration attempt

Does Registry.lookup solves this? We could make a lookup and return existing process, only starting the new one if it doesn’t exist