Command Pattern via TCP

fabioticconi

Hi all, I come from a relatively brief Erlang background from many years ago, and I’m trying now to think (again) in that distributed way - while learning Elixir, which I prefer at a glance.

In short, I’d like to know what (if any) is the Elixir-way to deal with a pattern I encounter often my side projects: the command pattern.

Assume I have a little TCP server setup (via ranch, in fact) and I want to have a clean way of adding commands to manipulate “global state” (currently, I’m using Mnesia.. but I already feel like this is not well supported in Elixir. A question for another day).

I went the polymorphism way, so each command is a module with behaviour Command which, in itself, defines callbacks. A series of processes run these commands.

However, clearly each string coming from the socket needs to be processed and validated as a Command. This is my attempt:

  defp parse!(_, ["quit" | _]) do {:ok, :quit} end
  defp parse!(_, ["shutdown" | _]) do {:ok, :shutdown} end
  defp parse!(_, ["echo" | opts]) do {:ok, {:echo, Enum.join(opts, " ")}} end
  defp parse!(_, cmd) when cmd == [] do {:ok, {:echo, ""}} end
  defp parse!(_state, [cmd | opts]) do
    module_name = Macro.camelize(cmd)
    try do
      module = String.to_existing_atom("Elixir.Commands.#{module_name}")
      {:ok, {module, opts}}
    rescue
      _ -> {:ok, {:echo, "#{cmd}: UNKNOWN_COMMAND"}}
    end
  end

It works, but I think it’s very brittle. So I’m trying meta-magic but I’m not sure I’m going in the right/sensible/idiomatic direction:

defmacro __using__(_opts) do
    quote do
      @behaviour Command

      @on_load :register_command

      def register_command() do
        Command.register_command(__MODULE__)
      end
    end
  end

This essentially allows me to register a command implementation, when it’s loaded. Again, it works, but I’m not sure it’s the right way.

If you can advise or point me in the right direction, I’d be very grateful Elixir is a very interesting language and I’d love to continue working with it.

13 comments

#rpc #command-pattern

1 1461 13

2021-10-10 20:19:28 UTC

First 10 of 13 Posts

smanza

I implementated a concept like this with tcp but I took another approach (without the polymerphism) .

By simply using the pattern matching , I have a module Message where there is a function encode , decode, and process.

The encode takes a message (struct) and serializes it by prepending a message which is then used to pattern match on the decode function.
And then the process take message struct as parameter to perform the task for the command.

Example:

defmodule Message do

  def encode(%MyMessage{}) do 
     <<0::8>>
  end

  def decode(<<0::8>>), do: %MyMessage{}

   def process(%MyMessage{}) do
       # Doing stuff
    end
end

And on the tcp handler:

data_to_send = data_received
|> Message.decode()
|> Message.process()
|> Message.encode() # If you want to send a response

If it can help you

Post #1

ityonemo

For the level of dynamicity you seek, the on_load idea is correct.

For idiomacity:. I recommend not doing shenanigans with camelize and string.to_atom. instead, I recommend registering your modules by updating an application env value (this is backed by an ets table, so it is blazing fast). The keys should be stringified final term of Module.split and the value should be the module itself.

Nitpicky: parse should not be parse!

Post #2

fabioticconi

I’m not sure I understand this well - in my use case, I may have hundreds of commands of varying complexity, how would you organise this? Everything in the Message module? This cannot work for me

Post #3

fabioticconi

Thanks! Essentially with the on_load, I’m not using string.to_atom anymore. I’m actually using an Agent as a key-value store (string → module as you also suggested). Eventually I’ll need a bit of a smarter key-value store (a treemap for example, if I want to support partial matching of command names, eg “rem” instead of “removefile”).

Nitpicky: parse should not be parse!

Can you explain why? In the “dynamic dispatch” tutorials they used it so I used it too, but it isn’t clear to me what it is or when it should be used instead..

Post #4

smanza

In my use case I have few ten of messages. Each message has it own struct but the serialization, deserialization is perform on the message module as its define the message ID for a given command.

Post #5

ityonemo

Use application.put_env and application.get_env instead of an agent, it doesn’t need to be supervised, you don’t have to worry about it going down, etc.

Functions that end in ! by convention signify that they are a raising equivalent of a function that emits ok/error tuples

Post #6

ityonemo

Looks like I was subtly wrong about ! convention:

But I would also say “don’t put a ! just because something can error”; I would say non-bang functions can raise on “programmer fault” (something analogous to :badarg); but should not raise on “user fault”.

Post #7

al2o3cr

Consider the simplest thing that could work: listing the mapping from command to handler atom explicitly.

  @handlers %{
    "foo" => Commands.Foo,
    "bar" => Commands.Bar,
    # etc
  }
  defp parse!(_state, [cmd | opts]) do
    case Map.fetch(@handlers, cmd) do
      {:ok, mod} -> {:ok, {mod, opts}}
      :error -> {:ok, {:echo, "#{cmd}: UNKNOWN_COMMAND"}}
    end
  end

This approach also has logical extension points for useful things:

broadening the possible keys of the map to things like Regexes would allow for “partial match” commands
broadening the possible values of the map to {module, baked_in_opts} lets one “command module” serve multiple external commands

One downside is that the command → module mapping can get quite long; consider extracting parts of it to functions and combining them at compile-time to reduce clutter.

Worth looking into persistent_term for storing the map - an Agent still forces every access through a single thread.

Post #8

fabioticconi

I’m having trouble with the set_env, as it doesn’t have an update method so I have race conditions (modules are loaded in parallel with on_load executions, it seems; maybe I can find some way to have on_load block the whole loading until it’s done).

So I’m thinking about alternatives. Ets tables etc, would work. But your idea is interesting @al2o3cr - an “attribute map” of the module? I need to try this

Essentially I still prefer to keep commands as separate files, and to register them automatically on load; it just seems cleaner (also works for hot code loading I think, as then the module is re-loaded and overwrites its entry in the command map). But your suggestion might fit nicely in my architecture.

Post #9