Macro defining function, but overriding function block

Questions / Help
1

Given the following:

defmodule Builder do

  defmacro define_builder( name ) do

    block = quote do
      IO.puts unquote({:passed_in_context, [], Elixir})
    end

    # defines a new function called 'name'
    {
      {:., [], [{:__aliases__, [alias: false], [:Kernel]}, :def]},
      [],
      [
        {name,[],[ {:passed_in_context, [], Elixir}, {:passed_in_block, [], Elixir} ]},
        [do: block]
      ]
    }
  end
end

defmodule BuildFunctions do
  import Builder
  define_builder :build_house
end

BuildFunctions.build_house( "this should be printed" ) do
  IO.puts "this should not be printed"
end

What I am trying to do is override the evaluation of the BuildFunctions.build_house do block and only execute the block defined within the define_builder macro. But the outcome is that both blocks are executed.

Following on from that, how would I get the AST of the passed in block, manipulate it, and reinsert so that it is executed?

2

Depends on your needs sometimes is better to write just:

defmodule BuildFunctions do
  use Builder, name: :build_house
end

Only macros works with AST like that. Of course you can write helper functions, but they works only when do_block is passed from macro argument:

defmodule Example do
  defmacro sample(do: {:__block__, _, block) do
    helper_function(block)
  end

  defp helper_function(block), do: # …
end

The problem is that you are writing macro which generates function (not macro), so passed do_block is evaluated and return value from it is passed to BuildFunctions.build_house/2 as passed_in_block argument. In your case passed value here is :ok (from IO.puts/1).

Also I don’t like idea to modify what developer passed in do_block, because it’s too more magic. Depends on use case you could look at Ecto.Schema.schema/2 macro (which imports other macros) or exactor library.

Anyway please say what you are trying to do (when you say that you want modify do_block). I’m sure that someone could find better solution for such use case.

4 Likes
3

Ultimately I’m after doing something like this:

defmodule BuildFunctions do
  import Builder
  define_builder :build_house, function_map: %{lay_bricks: :house_lay_bricks}
  define_builder :build_shop, function_map: %{lay_bricks: :shop_lay_bricks}
  
  def house_lay_bricks(context,inventory), do: %{ context, inc_bricks(inventory) }
  def house_build_door(context, inventory), do: %{context, inc_door(inventory)}
  def shop_lay_bricks(context,inventory), do: %{ context, inc_bricks(inventory) }
  
  defp inc_bricks( %{brick_count: brick_count} ), do: %{ context, %{ brick_count: brick_count+1 } }
  defp inc_door ...
end

Build_Functions.build_house( context ) do
  lay_bricks(2)
  build_door(1)
end

Build_Functions.build_shop( context ) do
  lay_bricks(10)
end

I already have the block ast rewriting working well with the use statement as you suggested:

defmodule BuildFunctions do
  use Builder, function_map: %{lay_bricks: :house_lay_bricks}, initialise: fn ctx -> ...
end

but that creates only a single predefined function, and what I am after is the ability to define many, each with their own function mapping/ initialisation etc.

My sense is that this is a ‘macro defining a macro’ problem - it is certainly hurting my head at least twice as much as normal. It is possibly too complicated, but interesting nonetheless.

Thanks for the Ecto and exactor tips - I’ll examine those for leads.

4

Nothing stops you from writing:

defmodule Example do
  use Sample, [first_name: %{}, second_name: %{}]
  # or
  use Sample, data: %{}, name: :first_name
  use Sample, data: %{}, name: :second_name
end

anyway, now I see what you are going to do

I would suggest much different code …

defmodule Example.Builder do
  @callback build_door(pid, integer) :: any
  @callback lay_bricks(pid, integer) :: any

  defmacro __using__(_opts \\ []) do
    quote do
      use ExActor.GenServer
      
      defstart start_link, do: initial_state(0)

      @behaviour unquote(__MODULE__)

      defoverridable [init: 1, start_link: 0]
    end
  end
end

defmodule Example.House do
  use Example.Builder

  # …
end

# …

alias Example.House

{:ok, pid} = House.start_link()

House.lay_bricks(pid, 2)
House.build_door(pid, 1)

This should be enough at start. Of course you need to properly write Example.House and Example.Shop modules + change callbacks in Example.Builder.

Look that in same way you can write __using__/1 macro in House, so other modules could use it as well:

defmodule Example.House do
  use Example.Builder

  defstart start_link, do: initial_state(1)

  defmacro __using__(_opts \\ []) do
    quote do
      use Example.Builder
      
      defstart start_link, do: initial_state(1)

      defoverridable [init: 1, start_link: 0]
    end
  end

  # …
end

# …

defmodule Example.House.SemiDetached do
  use Example.House

  # …
end
1 Like
5

After further twiddling, I solved the original question - the key was, as you pointed out @Eiji, that my macro should have been generating another macro:

defmodule Builder do

  defmacro define_builder( name ) do
    caller_module = __CALLER__.module

    block = quote do
      caller_module = unquote(caller_module)
      IO.puts unquote({:passed_in_context, [], caller_module})
    end

    # defines a new function called 'name'
    {
      {:., [], [{:__aliases__, [alias: false], [:Kernel]}, :defmacro]},
      [],
      [
        {name,[],[ {:passed_in_context, [], caller_module}, {:passed_in_block, [], caller_module} ]},
        [do: block]
      ]
    }
  end
end

defmodule BuildFunctions do
  import Builder
  define_builder :build_house
end

defmodule Runner do
  require BuildFunctions

  BuildFunctions.build_house( "this should be printed" ) do
    IO.puts "this should not be printed"
  end
end
1 Like
6

This should probably be a blog post, but just to conclude this topic here is the full solution to my use case:

defmodule Builder do
  defmacro define_builder(name, opts \\ []) do
    function_map = Keyword.get(opts, :function_map)
    initialise_fn = Macro.escape(Keyword.get(opts, :initialise))
    caller_module = __CALLER__.module

    # define the block which gets called in the builder function
    block =
      quote do
        caller_module = unquote(caller_module)

        fn_map = unquote(function_map)
        block_args = unquote({:passed_in_block, [], caller_module})

        block_args =
          case block_args do
            [do: {:__block__, [], block_args}] -> block_args
            _ -> block_args
          end

        context = unquote({:passed_in_context, [], caller_module})

        context =
          case unquote(initialise_fn) do
            nil -> context
            _ -> {{:., [], [unquote(initialise_fn)]}, [], [context]}
          end

        # go through each statement of the block, mapping function calls,
        # and ensuring that the context is passed to
        Enum.reduce(block_args, context, fn fn_arg, acc ->
          Builder.remap_functions(caller_module, fn_map, acc, fn_arg)
        end)
      end

    # defines a new function called 'name'
    {
      {:., [], [{:__aliases__, [alias: false], [:Kernel]}, :defmacro]},
      [],
      [
        {name, [],
         [{:passed_in_context, [], caller_module}, {:passed_in_block, [], caller_module}]},
        [do: block]
      ]
    }
  end

  # maps known function calls in the incoming quoted expression
  def remap_functions(caller_module, function_map, context, {fn_call, fn_ctx, fn_args}) do
    mapped_fn = Map.get(function_map, fn_call)
    # remove nil fn_args - we just send the context in this case
    fn_args = clean_fn_args(fn_args, context)

    if mapped_fn do
      {{:., fn_ctx, [{:__aliases__, fn_ctx, [caller_module]}, mapped_fn]}, fn_ctx, fn_args}
    else
      {fn_call, fn_ctx, fn_args}
    end
  end

  defp clean_fn_args(args, context) do
    case args do
      args when is_nil(args) -> [context]
      args -> [context] ++ args
    end
  end
end

defmodule BuildFunctions do
  import Builder

  define_builder(:build_house,
    initialise: fn ctx -> {ctx, %{:brick_count => 0, :door_count => 0}} end,
    function_map: %{lay_bricks: :house_lay_bricks, build_door: :house_build_door}
  )

  def house_lay_bricks({context, inventory}, count),
    do: {inc_ops(context), inc_bricks(inventory, count)}

  def house_build_door({context, inventory}, count),
    do: {inc_ops(context), inc_door(inventory, count)}

  defp inc_bricks(inventory, count), do: add_inventory(inventory, :brick_count, count)
  defp inc_door(inventory, count), do: add_inventory(inventory, :door_count, count)
  defp inc_ops(ops), do: %{ops | op_count: Map.get(ops, :op_count, 0) + 1}

  defp add_inventory(inv, inv_type, count),
    do: Map.put(inv, inv_type, Map.get(inv, inv_type) + count)

  define_builder(:build_garden,
    initialise: fn ctx -> {ctx, %{:flowers => %MapSet{}, :trees => %MapSet{}}} end,
    function_map: %{tree: :garden_plant_tree, flower: :garden_plant_flower}
  )

  def garden_plant_tree({context, inventory}, type),
    do: {inc_ops(context), add_type(inventory, :trees, type)}

  def garden_plant_flower({context, inventory}, type),
    do: {inc_ops(context), add_type(inventory, :flowers, type)}

  defp add_type(inv, inv_type, type),
    do: Map.put(inv, inv_type, Map.get(inv, inv_type) |> MapSet.put(type))
end

which is demonstrated by:

defmodule Runner do
  require BuildFunctions

  result =
    BuildFunctions.build_house %{op_count: 0} do
      lay_bricks(3)
      lay_bricks(2)
      build_door(1)
    end

  IO.puts(result == {%{op_count: 3}, %{brick_count: 5, door_count: 1}})

  result =
    BuildFunctions.build_garden %{op_count: 0} do
      tree("oak")
      flower("roses")
      tree("willow")
      flower("azalea")
    end

  IO.puts(
    result ==
      {%{op_count: 4},
       %{flowers: MapSet.new(["azalea", "roses"]), trees: MapSet.new(["oak", "willow"])}}
  )
end

What the builder function does is to rewrite the incoming block so that context is passed into and back from each statement (effectively piping) - thus avoiding the need to have GenServer provided state.

1 Like
7

Yeah, but it’s really unclear until you would look at source code. Just think: Are you looking at source code of every library in your every project? Will you remember how your every application is working for months/years? Nobody would have time to investigate what kind of magic you are doing. There are lots of ways to handle it in different way. Such code is not maintainable for me.

1 Like
8

I take your point, but it doesn’t appear to me to be that different to how Ecto.Schema works. And in such cases, good documentation and tests do a good job of demonstrating how such code works.

1 Like
9

Ecto.Schema definitely works much more differently. Look that ecto core team is not doing any do_block modifications.

1 Like
10

Food for thought! Thank you for your help.

1 Like