Discussion about uses for Agent Processes

For me it is essentially a separation of concerns. I realise that an Agent is effectively a specialised GenServer, and I could put all the code in one Module, but I find this approach helpful.

1 Like

Iā€™m actually quite curious as to how it is used. A GenServer passes a single state argument to all callbacks, which seems no different from a pid for an agent, you can pass both sets to functions to mutate the data within and return it, it just seems like a lot of overhead and extra code so I am curious as to the benefit?

1 Like

I think the main benefit is to meā€¦ :wink:

Specifically, I start my GenServer with a unique id, which in turn starts the Agent with an initial state, perhaps a record loaded via Ecto. The Agent contains a number of functions that transform its state via get_and_update/3, returning the changes and the new state. The GenServer manages the logic that invokes those transformation. It checks that the command is valid, etc.

Iā€™ve adopted this approach for a couple of reasons:

  1. The Agent is only concerned with what is needed to be done to transform the state of the struct it wraps. It could be a record from Ecto, but could also be a remote service. Nothing needs to know how it works, only that it does.

  2. The GenServer is only concerned with how the Agent should be transformed.

  3. A complex command might chain several Agent transformations together, and this reduces code duplication.

2 Likes

Hmm, Iā€™d probably have stuffed that record into the state (or ā€˜asā€™ the state if nothing else).

Wait, how do you do this? Adding functions to an Agent basically just means a GenServer, so why not just have it all in a GenServer?

Yeah, that should be a GenServer, you only add things to that GenServer that relate to messing with its own state.

That could also be another GenServer, but you would not need to keep the transformation for the agent in ā€˜thisā€™ genserver but could keep it with the data itself in the other genserver directly.

Hmm, how does this work? Code examples?

1 Like

I started with a GenServer having the entity as part of the state, but it soon got complex. I donā€™t mind writing more code in order to reduce complexity. I then migrated the entity to an Agent, and added some helpers. See this gist - caveat, I have macros for handling parts of this, and have extracted this simple case as best as I can. Mea cupla if it doesnā€™t serve the purpose! :slight_smile:

1 Like

Just because the GenServer holds the state does not mean all the functionality needs to be within it. Like on your User.Entity.activate/1 function, why can it not just take the user instead of the pid, then get rid of lines 13 and 17 then the part at User.handle_call/3 on line 18 can just store it back as the new state, then it becomes fully functional, code flow is well defined, no weird magical calls that return nothing are sitting around, and so forthā€¦ Iā€™m not sure what an Agent here is for other than maybe obfuscating what is happening. It does not seem to be very functional at all as well as being quite a bit slower than just a direct method call to the other module would be since it has to pass a message, copy data, set new data in the other process, then send a message back, etcā€¦?

1 Like

Iā€™m glad you said that, because I have been thinking about that recently. Iā€™ve been coding Ruby (on and off Rails) for the past 10 years, and have brought some of that baggage with me, no doubt. I started my current project with a prototype using DRb, and so naturally gravitated to Elixir when I came across it (Iā€™m actually from a Telecoms background, and was well are of Erlang/OTP, but never got around to using it).

So, back to the issue in hand - youā€™re saying that my I should drop the Agent, stick with just a GenServer, but keep the transformation functionality in a separate Module, passing user as a parameter. I can see the sense in this. :wink:

3 Likes

Yes absolutely! That is how my GenServerā€™s often are. The code that operates over the data and returns new data is elsewhere, the GenServer itself just handles the synchronization and messages to call the right commands and hold the state. :slight_smile:

EDIT: Just keep this in mind, the fundamental unit of Code Organization is not a Process on the BEAM, it is a Module. You should always jump to a new Module first and pass data around instead of a process until you really need a process, like for concurrency or so. :slight_smile:

9 Likes

1000x this!

Iā€™ve seen multiple cases where people reach for Agents to organize their code, where plain functional module would do the job. I even remember seeing somewhere someone stating that such approach is functional (because presumably Elixir is functional), which is definitely wrong.

My feeling is that since Agents are easy to use, they are also easy to abuse. Moreover, I believe that Agents solve a fairly trivial problem, and are basically just reducing LOC compared to GenServer. Which is why I only use Agents in tests (where they really come in handy), and avoid them otherwise. YMMV of course :slight_smile:

6 Likes

With all respect to JosĆ©, even when they were first introduced I found Agents to be uninteresting. I learned GenServer first and thatā€™s still what I tend toward. I even took a detour into Clojure at a prior job, and I learned agents there, but GenServer is so much more.

3 Likes

I will take that as a compliment. :smiley:

I also donā€™t tend to use agents much but, in the few cases I do, I prefer them to a GenServer. A GenServer, as the name says, is generic so when you only need to keep state around, the intent gets lost. Other than that, a GenServer is likely the way to go and is definitely much more.

Answering the original question: Mix has two or three examples of using agents.

2 Likes

We also use it in Gettext to store translations being extracted during compilation.

1 Like

I use it as a simple KV store when parsing things into DAGs, which is notoriously difficult to do in a pure functional style (especially without more hardcore tools like Monads et al.)

I think in general it seems that most of the ā€œgoodā€ use cases are short-lived accumulator-like processes, for times when insisting on passing accumulators around in a pure functional style gets too complex. For long-lived computation or processing or state storage, I definitely reach for a GenServer.

A monad is not hardcore, heck if you pass state from function to function (like what |> does already) that is just explicit monad handling instead of implicit, the monad in this case is just a raw value. ^.^

For anything in-process I still think passing the state around is better, after all you have to pass the PID around anyway. ^.^
For out of process I usually hit ETS, why spin up an agent when ETS can do it better and faster?
For out of node, well a GenServer, but an agent might have use here, or Mnesia dependingā€¦

I donā€™t think these are good use cases. If you really want that (and I believe in most cases itā€™s not a good idea), then consider process dictionary. At least with that, you wonā€™t need to pay the price for the separate process, copy data, and depend on the scheduler. An example of that in practice is :rand.uniform which uses procdict to implicitly manage the state of the RNG.

2 Likes

You might consider :digraph for this. The fact that itā€™s powered by ETS might confirm that this is indeed hard to do with FP. Never tried implementing a DAG myself, so canā€™t say. I had good experience with :digraph though :slight_smile:

I have actually looked at :digraph, what Iā€™m doing isnā€™t exactly processing arbitrary DAGs and the module isnā€™t a good fitā€”but yes, I suppose Iā€™m using an Agent as a lightweight alternative to an ETS table (all it does it keep two maps around) as the structures Iā€™m working with are quite small (couple hundred entries at most).

Here is another of my uses of Agent: Iā€™m using it as a lightweight cache to the database while doing an import of a dataset. There are a bunch of references to things and also some things which I am denormalising, and keeping things in memory instead of going to the database speeds things up when importing thousands of records.

defmodule PFServer.Finder.ImportMeta do
  @moduledoc """
  Keeps track of information during an import, and asynchronously imports program types and focus areas.
  """
  alias PFServer.Finder.{Organization, ProgramFocusArea, ProgramType}
  alias PFServer.Repo
  alias PFServer.TransactionManager, as: TM
  require Logger

  defstruct importing?: false, organization_map: %{}, ptypes: %{}, fareas: %{}

  def start_link do
    Agent.start_link(fn -> %__MODULE__{} end, name: __MODULE__)
  end

  def start_import do
    Agent.get_and_update(__MODULE__, fn state ->
      cond do
        state.importing? ->
          {{:error, :already_importing}, state}
        true ->
          {:ok, put_in(state.importing?, true)}
      end
     end)
  end

  def end_import do
    Agent.update(__MODULE__, fn _ -> %__MODULE__{} end)
  end

  def register_organization(%Organization{id: db_id, airtable_recid: recid}) do
    Agent.update(__MODULE__, fn state -> put_in(state.organization_map[recid], db_id) end)
  end

  def get_organization_id!(recid) do
    case Agent.get(__MODULE__, fn state -> state.organization_map[recid] end) do
      nil -> raise "Organization with record ID #{recid} has not been registered."
      id -> id
    end
  end

  def get_program_type(name) do
    Agent.get_and_update(__MODULE__, fn state ->
      case state.ptypes[name] do
        nil ->
          ptype = TM.execute fn -> Repo.insert!(%ProgramType{name: name}) end
          {ptype, put_in(state.ptypes[name], ptype)}
        ptype -> {ptype, state}
      end
     end)
  end

  def get_focus_area(name) do
    Agent.get_and_update(__MODULE__, fn state ->
      case state.fareas[name] do
        nil ->
          farea = TM.execute fn -> Repo.insert!(%ProgramFocusArea{name: name}) end
          {farea, put_in(state.fareas[name], farea)}
        farea -> {farea, state}
      end
     end)
  end

end

However even looking at it now, a case could be made that it should actually be a GenServer, given that itā€™s doing a little more than just keeping state. But then, all Agent is is a wrapper for a GenServer with a different API.

Looking at it more, even though usually we have all the logic to manipulate an Agentā€™s state inside a single module, an Agent is more like a ā€œpromiscuousā€ GenServer, in that in a typical GenServer setup, the logic to manipulate GenServer state is inside the GenServer itself and we have a functional interface to the GenServer. An Agent, on the other hand, allows us to provide it with arbitrary lambdas to modify its state.

This breaks a lot of conventions in OTP/Elixir in general, and I think thatā€™s why thereā€™s such an instinctive dislike of them in this thread.

It begs the questionā€”should they even be included in Elixir by default at all?

1 Like

Just judging by your Agent code here it seems like you could just as easily have a cache that was just a map and do it all in a purely functional style.

I donā€™t understand what this means. They arenā€™t arbitrary, they take the state and they return very specific values. This is exactly like a GenServer which has particular callbacks that take the state, arbitrary arguments, and return specific values.

1 Like

The issue you mention is indeed one drawback of Agent, but my main concern is about something else. I have an increasing feeling that agents are frequently misused as objects, where in-process structure would work just fine. I share Benā€™s impression of your code:

Looking at the interface of your module, it looks like it serves as a dumping ground used by a sinle process. If that is indeed the case, I think that plain data structure would work better. We had other examples in this thread where people used agents to organize the code, or maintain some accumulator in a loop.

While I agree Agents offer some benefits over GenServer when properly used (as demonstrated in mix and gettext examples), Iā€™m not sure these benefits are worth the downsides:

  • Agent is the additional abstraction people need to learn.
  • People (especially newcomers) seem to be confused with Agent vs GenServer. Over years, Iā€™ve repeatedly seen people asking which one should be used.
  • As you mentioned, by default agents break encapsulation.
  • It seems that agents are easily misused to work around FP and simulate OO. Admittedly thatā€™s also possible with GenServer (I know for a fact because I used to do that myself :slight_smile:), but it requires more overhead compared to agents. Wrapping e.g an integer in an agent is trivial. Doing the same with GenServer will require a dedicated module, so thatā€™s a hint that perhaps this is not the best approach :slight_smile:

Given these downsides, and the fact that Iā€™m personally not superimpressed with supposed benefits of Agents, looking at your question

I personally feel that weā€™d be better off without agents.

2 Likes