Hi!
Sorry for a long descrition but the problem context should be fully defined.
I’m developing a lib to process a lot of messages thats are the maps with binary name field (%{name: "some:very:very:long:name", other: :field}). I want to optimize a pattern matching on such messages (for some reasons I can’t use atoms for names - messages are delivered through different message queues such as RabbitMQ and Kafka). I decide to change names to their hashes at compile time. I define following macroses:
defmodule MyLib.Message do
defmacro __using__(otp_app) do
quote do
defmacro msg(name, fields) do
unquote(__MODULE__).msg(name, fields, unquote(otp_app))
end
end
end
def msg(name, {:%{}, ctx, fields}, app) do
{:%{}, ctx, Keyword.merge(fields, name_hash: name_hash(name, app))}
end
defp name_hash(name, app) do
config = with nil <- :elixir_config.get(MyLib, nil), do: MyLib.load_config(app)
with nil <- get_in(config, [:message_names, name]) do
names = Map.get(config, :message_names, %{})
hashes = Map.get(config, :message_hashes, %{})
hash = ensure_uniq_hash(Helpers.hash_name(name), hashes)
names = Map.put(names, name, hash)
hashes = Map.put(hashes, hash, name)
config = Map.merge(config, %{message_names: names, message_hashes: hashes})
:elixir_config.put(MyLib, config)
hash
end
end
end
inside name_hash hashes accumulated in the map that is stored with :elixir_config.put/2 and we need a compiler task after elixir compilation to save hashes on disk:
defmodule Mix.Tasks.Compile.MyLib do
alias Mix.Task.Compiler
use Compiler
@impl Compiler
def run(_) do
MyLib |> :elixir_config.get(:undefined) |> save_config()
end
@impl Compiler
def clean do
Mix.Project.config()[:app] |> MyLib.config_file() |> File.rm()
end
@persistent_fields ~w[message_names]a
defp save_config(:undefined), do: {:noop, []}
defp save_config(%{} = config) do
bin = config |> Map.take(@persistent_fields) |> :erlang.term_to_binary()
Mix.Project.config()[:app] |> MyLib.config_file() |> File.write(bin)
end
defp save_config(_invalid), do: {:noop, []}
end
At this moment all works fine - I’m loading generated file at startup into ets and quickly retrieve name hashes while receiving messages and than get a fast pattern-matching on it:
defmodule MyApp do
use MyLib.Message, :my_app
def handle_message(msg("my:long:name", %{type: type})) do
# do something with type
end
end
The problem arrives when I try to use this lib in umbrella app (suppose I use my_lib only in my_app1 and my_app3):
apps
-- my_app1
-- my_app2
-- my_app3
-- my_lib
The main problem - my custom compiler task runs only once and not after all applications compiled but only after my_app1. And the second problem - Mix.Project.config()[:app] returns nil. But this problem can be worked around with Mix.Project.umbrella? and adding cutom option to root mix project.
Is it possible to run compiler task after compiling all applications in umbrella project?
Thanks!
