Is it possible to use `use/2` macro inside `defprotocol` block?

Questions / Help
1

Hi!

I’m building a system which is intended to run inside an embedded device and should work with “peripherals”: hardware modules. Each device represented as a struct with a common core and specific device configuration options. So I create a module for each device type and use the common core module like this (it isn’t a real code, just an example I wrote for this question):

defmodule Tool do
  @callback check_config(tool :: term()) ::
              :ok
              | {:error, term()}

  defmacro __using__(_) do
    quote do
      @behaviour unquote(__MODULE__)

      defstruct [:id, :name, :config, :attached]
      # and other common core struct fields and functions
    end
  end
end

defmodule Tools.SimpleRelay do
  use Tool

  def check_config(%SimpleRelay{config: nil} = _tool), do: {:error, :config_required}
  def check_config(%SimpleRelay{} = _tool), do: :ok
end

So now I can instantiate SimpleRelay structs and use them.

The next abstraction is an Action - a specific action a Tool can do: like “turn on” for a relay or “do a measurement” for a sensor. Actions have common names but must have per-device type implementation. So the most obvious solution is Protocols: each Action is a protocol, and we can add an implementation for the specific device types:

defprotocol Actions.TurnOn do
  def run(tool)
end

defimpl Actions.TurnOn, for: Tools.SimpleRelay do
  def run(%Tools.SimpleRelay{attached: true, config: %{pin: pin}}) do
    {:ok, pin_ref} = Circuits.GPIO.open(pin, output)
    Circuits.GPIO.write(pin_ref, 1)
  end
  def run(%Tools.SimpleRelay{}), do: {:error, :device_should_be_attached}
end

And here is a question I’ve got. Actions should have some kind of meta information as well: for example, resulting_events/1 gives a list of returning events (which is also might be different for different devices: one sensor can measure temperature only but an another one does temperature, humidity and happiness on Alpha Centauri), or even different options list: some actions can have a config, so it would be run/1 for no config actions and run/2 for the configurable ones. And so on.

So as a result it is necessary to have a lot of different protocol definitions with the same bodies:

defprotocol Actions.TurnOn do
  def run(tool)
  def resulting_events(tool)
  def other_meta(tool)
end

defprotocol Actions.TurnOff do
  def run(tool)
  def resulting_events(tool)
  def other_meta(tool)
end

defprotocol Actions.DoMeasurement do
  def run(tool)
  def resulting_events(tool)
  def other_meta(tool)
end

For the general module I would use use/2 macro. But I can’t get it to work with defprotocol/2:

iex(1)> defmodule Action do
...(1)>   defmacro __using__(_) do
...(1)>     quote do
...(1)>       def run(tool)
...(1)>     end
...(1)>   end
...(1)> end
{:module, Action,
 <<70, 79, 82, 49, 0, 0, 5, 68, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 136,
   0, 0, 0, 13, 13, 69, 108, 105, 120, 105, 114, 46, 65, 99, 116, 105, 111, 110,
   8, 95, 95, 105, 110, 102, 111, 95, 95, ...>>, {:__using__, 1}}
iex(2)>
nil
iex(3)> defprotocol Actions.ShowId do
...(3)>   use Action
...(3)> end
{:module, Actions.ShowId,
 <<70, 79, 82, 49, 0, 0, 17, 100, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 2, 173,
   0, 0, 0, 45, 21, 69, 108, 105, 120, 105, 114, 46, 65, 99, 116, 105, 111, 110,
   115, 46, 83, 104, 111, 119, 73, 100, 8, ...>>, {:__protocol__, 1}}
iex(4)>
nil
iex(5)> defmodule Tools.SimpleRelay do
...(5)>   defstruct [:id]
...(5)>
...(5)>   defimpl Actions.ShowId do
...(5)>     def run(%Tools.SimpleRelay{id: id}), do: IO.puts(inspect(id))
...(5)>   end
...(5)> end
warning: module Actions.ShowId is not a behaviour (in module Actions.ShowId.Tools.SimpleRelay)
  iex:8: Actions.ShowId.Tools.SimpleRelay (module)
iex(6)> relay = struct(Tools.SimpleRelay, id: "ALongStringId")
%Tools.SimpleRelay{id: "ALongStringId"}
iex(7)> Actions.ShowId.run(relay)
** (UndefinedFunctionError) function Actions.ShowId.run/1 is undefined or private
    Actions.ShowId.run(%Tools.SimpleRelay{id: "ALongStringId"})

I’ve tried to grok Protocol.__protocol__/2 but without success.

So the questions are:

  1. Is it possible to reach what I want? Is it my lack of knowledge or this doesn’t work intentionally?
  2. Or do you think I’ve got a wrong path? Could give me an idea on how to do it in another way (I mean, not “in general” but anything as effective and efficient from the development perspective as protocols)?

Thanks!

2

I can see that Actions.ShowId does not define behaviour_info/1 hence protocol is not properly built when protocol functions are defined by quote block. IMO it’s a bug in Protocol

2 Likes
3

This is what I’m trying to get: is it desired behaviour or a bug.
Thanks, I’ll try to open an issue.

4

I have provided a detailed answer at SO, FWIW.

1 Like
5

Thanks!

I’ve tried to import Protocol, only: [def: 1] inside __using__ macro like this:

defmodule Action do
  defmacro __using__(_) do
    quote do
      import Protocol, only: [def: 1]
      def run(tool)
    end
  end
end

but it doesn’t work, so I believe I didn’t even try direct macro call via Protocol.def/1

6

I wonder how it could ever compile. ATM there is Kernel.def/2 already imported. Also, even if it does, this AST gets injected into all the modules using it, basically ruining Kernel.def/2 behaviour afterward.

When people say “use macros with caution,” they mean exactly that: one should clearly understand drawbacks and caveats that direct AST modification brings on the table. If you thought that import Protocol inside a quoted block in __using__/1 would be scoped, I’d strongly suggest you to read “Metaprogramming Elixir” by Chris McCord.

2 Likes
7

Could you explain what is the difference with Protocol.__protocol__/2 then?

Thanks, this book is on my list.

8

Protocol.__protocol__/2 starts with unexporting Kernel.def/2.

1 Like
9

This is pretty interesting to see such accents. I know it starts with unexporting, I can read sources, but doing the same doesn’t change nothing. This is why I asked if you can explain the difference in mechanics, not in obvious things.
Anyway, thanks for your time.

10

Sure it does, but you are not doing the same. There is no way to get into the scope of implementation module that gets opened here from your client code. You might alter the calling scope, though (see below.)

When you do

defmodule Action do
  defmacro __using__(_) do
    quote do
      import Kernel, except: [def: 1, def: 2]
      import Protocol, only: [def: 1]
      def run(tool)
    end
  end
end

defprotocol Actions.ShowId, do: use Action

you effectively import the proper def/1 into unrelated context. You might easily use IO.inspect/2 to notice that:

defmodule Action do
  defmacro __using__(_) do
    quote do
      import Kernel, except: [def: 1, def: 2]
      import Protocol, only: [def: 1]
      def run(tool)
    end
    |> IO.inspect()
  end
end

...

While compiling you’ll see:

{:__block__, [],
 [
   ...,
   {:def, [context: Action, import: Kernel],
    [{:run, [context: Action], [{:tool, [], Action}]}]}
 ]}

[context: Action, import: Kernel] is how this block gets quoted, because at the moment of quoting Kernel.def/2 is imported. You might do instead:

defmodule Action do
  defmacro __using__(_) do
    import Kernel, except: [def: 1, def: 2]
    import Protocol, only: [def: 1]

    quote do
      def run(tool)
    end
    |> IO.inspect()
  end
end

And now everything would work as expected. Output:

{:def, [context: Action, import: Protocol],
 [{:run, [context: Action], [{:tool, [], Action}]}]}

The proper macro is now imported (see [context: Action, import: Protocol],) but I did not suggest that way because it damages the scope afterward.

2 Likes
11

Thank you very much for the explanation, now it has more sense!

I initially was under impression that quoted content should expand in the current context, so here the block and all its inherited parts should get the current context.

Sadly I didn’t spend enough time for metaprogramming, just occasionally reviewed a few pieces of code, played command line games a little and read very sketchy articles. I’ve postponed “Metaprogramming Elixir” for a pretty long time, now I see I should find time to finally read it.

2 Likes
12

This is indeed true. What broke your expectation is that Kernel.SpecialForms.quote/2 is a macro, that does its job in the context where is was called.

It needs to produce AST out of literally anything, so it grabs the context to allow not FQN calls within. Then it sees def and looks it up in the current context, returning {:def, [context: Action, import: Kernel], ...} because that is the def that is known here.

Hope it made things even more vivid.

2 Likes