Looking for clarity around using agent

Questions / Help
1

In the following code I use agent to “save state” of a list and then I update the list.
(I know elixir does not have objects but naming my functions as such helps me reason about the code )

defmodule M do
	def create_object() do
		{:ok, pid} = Agent.start_link(fn -> [1, 2, 3] end)
		pid
	end

	def update_object(pid, new_data) do
		Agent.update(pid, fn (state) -> state ++ new_data end)
		pid
	end

	def get_object(pid) do
		IO.inspect 	Agent.get(pid, &(&1))
	end
end


M.create_object() |> M.update_object([4, 5]) |> M.get_object()

The following code has the same result but does not use agent. What can the code that does use agent do that the following code can not? Asking this question is my attempt to see what I am not understanding.

defmodule M do
	def create_object() do
		obj  = [1, 2, 3] 
		obj 
	end

	def update_object(obj, new_data) do
		obj = obj ++ new_data
		obj
	end

	def get_object(obj) do
		IO.inspect 	obj
		obj
	end
end

M.create_object() |> M.update_object([4, 5]) |> M.get_object()
1 Like
2

The main difference is that Agent (or GenServer) runs in it’s own process.

This means that knowing the pid of Agent (or it’s registered name) you can retrieve/save state from different concurrent processes.

You can use Agent to say implement a dynamic configuration for the application. All HTTP requests are handled in their own processses, but they would be able to access this shared configuration. Then, you could update the configuration say from iex shell and it would be picked up by all other processes now on.

7 Likes
3

Hubert’s reply is correct

I’d like to explore some aspects in more depth and detail.

First of all many of us are still scratching our head in terms of finding valid use cases for Agents as evidenced by this topic:
Discussion about uses for Agent Processes

Hubert’s use case of changing the configuration state of a running system through the shell is a viable one - though I would imagine that sending a function to update the state is inherently more risky than simply replacing the old state with a new, known to be consistent, state.

Then there is the choice of your example - it is inherently sequential. I recommend that you watch:

Erlang Master Class 2: Video 1 - Turning sequential code into concurrent code

Your example forces a particular ordering on the sequence of operations which doesn’t take advantage of the capabilities of the Agent - there are no independent, concurrent parts in your computation.

Your use of an Agent can be roughly expressed as a GenServer like this

defmodule M do
  use GenServer

  # instead of fn -> [1, 2, 3] end
  def create_object() do
    {:ok, pid} = GenServer.start_link(__MODULE__, [1,2,3])
    pid
  end
  
  def update_object(pid, new_data) do
    GenServer.call(pid,{:update, new_data})
    pid
  end
  
  def get_object(pid) do
    IO.inspect GenServer.call(pid,:get)
    pid
  end 
  
  # GenServer callbacks
  def init(state)  do
    # TODO initialization logic
    {:ok, state}
  end
  
  # instead of `fn (state) -> state ++ new_data end`
  def handle_call({:update, new_data}, _from, state) do
    {:reply, :ok, state ++ new_data}
  end

  # instead of `&(&1)`
  def handle_call(:get, _from, state) do
    {:reply, state, state}
  end
  
end
M.create_object() |> M.update_object([4,5]) |> M.get_object()

Agents are often used to introduce the concept of processes because the code looks initially much less arcane than GenServer code. In your example the client code provides the function that is updating the Agent’s state. In my GenServer based code I fixed the “meaning” of “update” in a module function to appending new_data to the GenServer state.

The point I’m trying to make is that Agent can be viewed as a GenServer turned-inside-out. With a GenServer the functionality to change process state is fixed within the callback module - with an Agent the computations (functionality) to change process state are provided from outside of the process.

The whole point for an Agent:

Agent.update(pid, fn (state) -> state ++ new_data end)

or a GenServer

GenServer.call(pid,{:update, new_data})

is that numerous (tens, hundreds, thousands, … of) other processes can concurrently append data to the list managed by process pid without sharing state.

In a typical conventional multi-threaded program that list would often be shared so that each thread could append its own data to the shared list - so the list would have to be explicitly protected by locks, mutexes, etc.

Now Agent.update and GenServer.call are synchronous calls - so they will block until the target process receives the message and sends a reply. For asynchronous processing there is Agent.cast and GenServer.cast

  def update_object(pid, new_data) do
    GenServer.cast(pid,{:update, new_data})
    pid
  end
  
  ...
  
  # instead of `fn (state) -> state ++ new_data end`
  def handle_cast({:update, new_data}, state) do
    {:noreply, state ++ new_data}
  end

Now when update_object/2 returns there is no guarantee that the list in pidhas been updated yet. But the code still works - because get_object/1 is still synchronous and because here messages are processed in the order of arrival, i.e. {:update,new_data} before :get, the list will be updated before the state is returned.

4 Likes
4

I will need to read your response a bunch to understand it but…

I think a perfect example to help someone like me to understand agents would be a minimalist terminal chat application that uses them. In my head I have an image of firing up the script in one terminal with a user name and then firing up the same script in a different terminal with a different user name and then they communicate with each other. I’ve seen this kind of thing as a big bulky “Chat server” and using Phoenix but not as a slimmed down “light” example with Agents that can fit on a page or two.

5

Something like that is here. But it is implemented with the concurrency primitives spawn, send and receive. Now it would possible to use GenServer instead.

Using Agent would not be a good fit.

6

Is it bad practice to mix/match spawn and agent. I have this image in my head of spawn recursing to create the “main process” and an agent is being used as the entry to join the chat as a user. If this sounds stupid just ignore me - I’m new :slight_smile:

7

Both Agent and GenServer are part of the Elixir OTP library and are built on top of the concurrency primitives - that is why you don’t mix them.

That being said it is helpful to know how processes work on the primitive level before using OTP. So looking at Processes is a good start.

1 Like
8

I think you generally don’t use bare Erlang/Elixir processes without relying on GenServer (or Agent) in real life. The only cases where I ever did that, was when I simply wanted to start a background job that I did not really care or interacted with afterwards. Such as sending e-mail.

And even then, it usually evolved to a GenServer(s) since this gives some sort of ability to limit the concurrency, instll some monitoring etc.

Spawning bare bones processes is quickly becoming inconvenient.

9

Like the other answers say, Agents are uses to manage state across different processes. For example, in my Chat app, I use an Agent to store who’s tying state. When someone starts typing, their typing state is send to the agent. When someone else starts typing that process can query the agent for others typing in that room.

They can also be used for saving state across different web requests. For example, my authentication plug stores the user’s session key in an agent when they login. On each subsequent web request, the plug checks for their session key in the agent and sets current_user in the conn assigns to the user data (stored as the value in the agent).

In a nut shell, I use Agents to save state that I don’t need persisted across server restarts. They are faster then using a database. If I need more than just simple put, update, get access I will use a GenServer.

There are other approaches that can use as an alternative to Agents; ETS, Process dictionary, GenServer, and database.

1 Like
10

The chat example doesn’t work. It’s outdated.

11

A lot can be learned by updating outdated code.

defmodule Chat.Server do
  defp reply(pid, reply) do
     send pid, {self(), reply}
  end

  defp cast(pid, message) do
    send pid, {self(), message}
    # noreply expected
  end

  # Send 'message' to all clients except 'sender'
  defp broadcast(room, message, sender \\ :undefined) do
    targets =
      case is_pid sender do
        false ->
          room
        _ ->
          List.delete(room, sender)
      end
    Enum.each targets, fn(pid) -> cast(pid, {:message, message}) end
  end

  def loop(room) do
    receive do
      {pid, :join} ->
        broadcast room, "Some user with pid #{inspect pid} joined"
        reply(pid, :ok)
        loop([pid|room])

      {pid, {:say, message}} ->
        broadcast room, "#{inspect pid}" <> message, pid
        reply(pid, :ok)
        loop(room)

      {pid, :leave} ->
        reply(pid, :ok)
        new_room = List.delete(room, pid)
        broadcast new_room, "User with pid #{inspect pid} left"
        loop(new_room)

      {_pid, :stop} ->
        IO.puts "#{inspect self()} Server terminating"
        :ok
    end
  end
end

defmodule Chat.Interface do

  # Send 'message' to 'server' and wait for a reply
  # Notice how this function is declared using 'defp' meaning
  # it's private and can only be called inside this module
  defp call(server, message) do
    send server, {self(), message}
    receive do
      {^server, reply} ->
         reply
      after
        1000 ->
          IO.puts "Connection to room timed out"
          :timeout
    end
  end

  defp cast(server, message) do
    send server, {self(), message}
    # noreply expected
  end

  # Receive a pending message from 'server' and print it
  def flush(server) do
    receive do
      # The caret '^' is used to match against the value of 'server',
      # it is a basic filtering based on the sender
      { ^server, {:message, message} } ->
        IO.puts message
        flush(server)

    # no more messages to flush
    after
      0 ->
        :ok
    end
  end

  # In all of the following functions 'server' stands for the server's pid
  def join(server) do
    call(server, :join)
  end

  def say(server, message) do
    call(server, {:say, message} )
  end

  def leave(server) do
    call(server, :leave)
  end

  # Part of the interface - though not the "Chat" interface
  def stop(server) do
    cast(server, :stop)
  end
end

server_pid = spawn fn() -> Chat.Server.loop [] end

# register shell as a client
Chat.Interface.join server_pid

# Spawn another process as client
spawn fn() ->
  Chat.Interface.join server_pid
  Chat.Interface.say server_pid, "Hi!"
  Chat.Interface.leave server_pid
end

# And another one
spawn fn() ->
  Chat.Interface.join server_pid
  Chat.Interface.say server_pid, "What's up?"
  Chat.Interface.leave server_pid
end

:timer.sleep 500
# messages accumulated for the shell
Chat.Interface.flush server_pid

Chat.Interface.leave server_pid
Chat.Interface.stop server_pid

:timer.sleep 500
IO.puts"Done"

works even on http://elixirplayground.com/

Some user with pid #PID<0.58.0> joined
Some user with pid #PID<0.59.0> joined
#PID<0.58.0>Hi!
User with pid #PID<0.58.0> left
#PID<0.59.0>What's up?
User with pid #PID<0.59.0> left
#PID<0.57.0> Server terminating
Done

GenServer version

defmodule Chat.Server do
  use GenServer

  # Send 'message' to all clients except 'sender'
  defp broadcast(room, message, sender \\ :undefined) do
    targets =
      case is_pid sender do
        false ->
          room
        _ ->
          List.delete(room, sender)
      end
    Enum.each targets, fn(pid) -> GenServer.cast(pid, {:message, message}) end
  end

  ## GenServer callbacks
  def terminate(_reason, _room) do
    IO.puts "#{inspect self()} Server terminating"
    :ok
  end

  # Note:
  # Always use "from" as an opaque data type;
  # don’t assume it is a tuple, as its representation might change in future releases.
  # p.89 Designing for Scalability with Erlang/OTP (2016)
  #
  def handle_call({:join, pid}, _from, room) do
    broadcast room, "Some user with pid #{inspect pid} joined"
    {:reply, :ok, [pid|room]}
  end

  def handle_call({:say, pid, message}, _from, room) do
    broadcast room, "#{inspect pid}" <> message, pid
    {:reply, :ok, room}
  end

  def handle_call({:leave, pid}, _from, room) do
    new_room = List.delete(room, pid)
    broadcast new_room, "User with pid #{inspect pid} left"
    {:reply, :ok, new_room}
  end
end

defmodule Chat.Interface do

  # In all of the following functions 'server' stands for the server's pid
  def join(server) do
    GenServer.call(server, {:join, self()})
  end

  def say(server, message) do
    GenServer.call(server, {:say, self(), message})
  end

  def leave(server) do
    GenServer.call(server, {:leave, self()})
  end

  # Part of the interface - though not the "Chat" interface
  def stop(server) do
    GenServer.stop(server)
  end

  # Receive a pending message from 'server' and print it
  def flush(server) do
    receive do
      {:"$gen_cast", {:message, message }} ->
        IO.inspect message
        flush(server)

    # no more messages to flush
    after
      0 ->
        :ok
    end
  end
end

{:ok, server_pid} = GenServer.start Chat.Server, []

# register shell as a client
Chat.Interface.join server_pid

# Spawn another process as client
spawn fn() ->
  Chat.Interface.join server_pid
  Chat.Interface.say server_pid, "Hi!"
  Chat.Interface.leave server_pid
end

# And another one
spawn fn() ->
  Chat.Interface.join server_pid
  Chat.Interface.say server_pid, "What's up?"
  Chat.Interface.leave server_pid
end

:timer.sleep 500
# messages accumulated for the shell
Chat.Interface.flush server_pid

Chat.Interface.leave server_pid
Chat.Interface.stop server_pid

:timer.sleep 500
IO.puts"Done"

Elixir playground doesn’t support GenServer.stop.

And finally the Agent version … it’s wrong in so many ways.

defmodule Chat.App do
  ## Agent.update runs the following functions in the Agent's process
  ## via the anonymous functions returned by the "handle_x" functions
  
  # Send 'message' to all clients except 'sender'
  defp broadcast(room, message, sender \\ :undefined) do
    targets =
      case is_pid sender do
        false ->
          room
        _ ->
          List.delete(room, sender)
      end
    Enum.each targets,
      fn(pid) -> send pid, {self(), {:message, message}} end
  end

  defp handle_join(pid) do
    fn(room) ->
      broadcast room, "Some user with pid #{inspect pid} joined"
      [pid|room]
    end
  end

  defp handle_say(pid, message) do
    fn(room) ->
      broadcast room, "#{inspect pid}" <> message, pid
      room
    end
  end

  defp handle_leave(pid) do
    fn(room) ->
      new_room = List.delete(room, pid)
      broadcast new_room, "User with pid #{inspect pid} left"
      new_room
    end
  end

  ## the following functions run in the client's process

  def join(agent) do
    Agent.update(agent, handle_join(self()))
  end

  def say(agent, message) do
    Agent.update(agent, handle_say(self(),message))
  end

  def leave(agent) do
    Agent.update(agent, handle_leave(self()))
  end

  # Receive pending messages from 'agent' and print them
  def flush(agent) do
    receive do
      # The caret '^' is used to match against the value of 'server',
      # it is a basic filtering based on the sender
      { ^agent, {:message, message} } ->
        IO.puts message
        flush(agent)

    # no more messages to flush
    after
      0 ->
        :ok
    end
  end

end

{:ok, agent_pid} = Agent.start fn -> [] end

# register shell as a client
Chat.App.join agent_pid

# Spawn another process as client
spawn fn() ->
  Chat.App.join agent_pid
  Chat.App.say agent_pid, "Hi!"
  Chat.App.leave agent_pid
end

# And another one
spawn fn() ->
  Chat.App.join agent_pid
  Chat.App.say agent_pid, "What's up?"
  Chat.App.leave agent_pid
end

:timer.sleep 500
# messages accumulated for the shell
Chat.App.flush agent_pid

Chat.App.leave agent_pid
Agent.stop agent_pid

:timer.sleep 500
IO.puts"Done"

The following is more in keeping with how an Agent is actually supposed to be used - as a state container. As a result the chat is now peer-to-peer - there is no “server” as such. However the clients do not keep the “room” as part of their own state - that is farmed out to the agent and manipulated through the anonymous functions created by the various handle_x functions.

However it still remains a strange way of using an agent.

defmodule Chat.App do

  #"handle_x" return anonymous functions
  # that are sent to the agent to run in the agent process      
  defp handle_join(pid) do
    fn(room) ->
      {room, [pid|room]}
    end
  end

  defp handle_say() do
    fn(room) -> room end
  end

  defp handle_leave(pid) do
    fn(room) ->
      new_room = List.delete(room, pid)
      {new_room, new_room}
    end
  end

  ## the following functions run in the client's process
  ## now only the clients are sending messages
  
  def join(agent) do
    pid = self()
    old_room = Agent.get_and_update(agent, handle_join(pid))
    broadcast old_room, "Some user with pid #{inspect pid} joined"
  end

  def say(agent, message) do
    pid = self()
    room = Agent.get(agent, handle_say())
    broadcast room, "#{inspect pid}" <> message, pid
  end

  def leave(agent) do
    pid = self()
    new_room = Agent.get_and_update(agent, handle_leave(pid))
    broadcast new_room, "User with pid #{inspect pid} left"
  end

  # Send 'message' to all clients except 'sender'
  defp broadcast(room, message, sender \\ :undefined) do
    targets =
      case is_pid sender do
        false ->
          room
        _ ->
          List.delete(room, sender)
      end
    Enum.each targets,
      fn(pid) -> send pid, {self(), {:message, message}} end
  end

  # Receive pending messages from 'agent' and print them
  def flush() do
    receive do
      # The caret '^' is used to match against the value of 'server',
      # it is a basic filtering based on the sender
      { _pid, {:message, message} } ->
        IO.puts message
        flush()

    # no more messages to flush
    after
      0 ->
        :ok
    end
  end

end

{:ok, agent_pid} = Agent.start fn -> [] end

# register shell as a client
Chat.App.join agent_pid

# Spawn another process as client
spawn fn() ->
  Chat.App.join agent_pid
  Chat.App.say agent_pid, "Hi!"
  Chat.App.leave agent_pid
end

# And another one
spawn fn() ->
  Chat.App.join agent_pid
  Chat.App.say agent_pid, "What's up?"
  Chat.App.leave agent_pid
end

:timer.sleep 500
# messages accumulated for the shell
Chat.App.flush

Chat.App.leave agent_pid
Agent.stop agent_pid

:timer.sleep 500
IO.puts"Done"
3 Likes
12

I’m not sure I get the example. Why would you not just use Phoenix Channels?

13

We’re talking about barebones Elixir here. The topic started about Agents - but they really are best understood in the context of the concurrency primitives they are based on and the (more commonly used) counterpart GenServer.

The primary objective is Agents (and it’s relatives); the chat application was merely an example that was asked for in this context (admittedly far from a great one for Agents).

1 Like
14

Got it. It does seem like a misuse of agents IMHO.

1 Like
15

I’m going to clarify how I use Agents based on some of the other posts that came after mine. I’ve been hearing a lot of noise about Agents being misused or that they provide not value. At least that’s my take away from various source I can’t remember.

I use Agents mostly as a key/value storage solution. In most cases I could have used ETS to accomplish the same. And from what I’ve read, the performance would probably be better. However, I probably use mostly out of laziness. I just find it hard to remember Erlang’s (IMHO cryptic) syntax. I did find that using an Agent came in handy once when I had to extend an existing Agent based module for a port reuse strategy.

In this case, I had a general Agent handling long lived state (but not requiring vm restart persistence) in a pure Elixir application (no database). I needed to later add a port reuse strategy. This means, reading the data store, checking to see if a port was available for reuse. So this required reads and updates that that were concurrency safe.

I could have moved the whole design over to a GenServer, but instead, I just wrapped the code in an Agent.get_and_update. That was a few years ago. The implementation has been reused on another project since then. I have not has an issue with the design on hundreds of the products sold.

Personally, I would not use an Agent for logic any more complicated then that. If you need behaviour, I much prefer using either a GenServer or a GenFSM.

16

My personal issue with Agents is that they depend on “functions as the message payload” - I’m all for “functions being first class citizens” within a process but I get a bit queasy when functions start moving between processes especially as those processes could be on two different nodes.

  • By accepting functions in messages the process is essentially giving up the autonomy to manage its own state - it’s completely at the mercy of whatever decides to send it a function, potentially risking its state that other processes are depending on.
  • If the sent function references any application modules - those dependencies better be well in sync on the nodes that the message/function is travelling between.

I feel that the functions that manipulate process state should be collocated with the process (and the state), so I’d go with a GenServer even if it is a bit more work.

As “cool” as it is to be able to execute functions in an atomic fashion on a state that is potentially “conceptually shared” between multiple processes, I’d much rather have those functions be an integral part of the process. I guess that makes me a process-autonomy advocate.
:icon_biggrin:

2 Likes
17

Some interesting points @peerreynders. To be honest, all my Elixir projects over the last 3 years have all been single node solutions. Busy ones, but all single node, mainly enterprise telephony relates. I have played around with multi node, but nothing commercially. With that said, I have not given much thought to the distribution aspect of Agents.

Also, when I design a module with an Agent, I typically add the api to the module. So the consumers are calling an API with data. In only one instance did I expose an interface where the consumer passes their own fn. And I wasn’t very comfortable with the design.

I am, however, staring to see the pain points around using Agents, especially for the inexperienced. Its been an informative discussion for me.

As for Agent vs GenServer, I guess I save a few lines of code. Basically its 1 function / API call for Agent vs 2 for GenServer. And not even that if I used something like exactor which I’ve pretty well abandoned these days.

I’m almost convinced to abandon Agents in favour of GenServers :wink:

2 Likes