yolo007wizard

Need help with app architecture - how to dynamically run code?

Problem Statement:

I am trying to build an ‘action runner’ for judge json rules. In a nut shell, say a json defined rule matches with action string:

{
...rule
    "action": "collect_signature"
}

I’m trying to think of best architecture to run that ‘collect_signature’ code. It could be a module name or a script file. The intention is for rules to trigger actions in a dynamic way so actions are easy to maintain separately.

Desired Requirements:

the action code has its own dependencies/hex packages
adding new actions does not require app reload/recompile/redeploy of the ‘action runner’ i.e. the actions are dynamically loaded and run somehow.

Idea 1:

standalone exs scripts - Actions are defined in standalone exs files with their own Mix.installs. I tried this already by using Code.require_file/1 to dynamically run an exs file however ran into dependency errors. The ‘action runner’ runtime dependencies had conflicts with the loaded exs dependencies. Need a way to get around that. This seems the most promising however.

Idea 2:

Umbrella project where each action is a separate application. I don’t have experience with umbrella projects but this might be a feasible path. It seems a bit overkill however and I think the action runner would have to be reloaded every time a new action is created (which is a big con).

Idea 3:

Similar to umbrella, load actions as private local packages via a dependency path. The ‘action runner’ would then dynamically load dependencies in its mix.exs. Sudo code like:

defp deps do
    for each action_name in actions folder return:
        {:action_name, ">= 0.0.0", path: "/actions/action_name"}
end

Then maybe add a Mix task ‘new action’ that loads up the boiler plate for a new action in the correct folder. The main con with this is same as #2 - the action runner would have to be recompiled with every new action.

17 comments

#umbrella #script #dependencies #exs #dynamic

0 1319 17

2023-09-09 17:53:12 UTC

Most Liked

axelson

Scenic Core Team

When I tried running multiple notes before I used the Erlang :peer module.

Here’s a few snippets from a semi-defunct unreleased project I have:

    {:ok, node} =
      :peer.start_link(%{
        host: :localhost,
        name: 'peer_node',
        args: peer_args()
      })

    IO.puts("DONE Starting peer node!")
    add_code_paths(node)
    ensure_applications_started(node)

  defp add_code_paths(node) do
    rpc(node, :code, :add_paths, [:code.get_path()])
  end

  @apps_to_start [
    # :iex,
    :logger,
    # :file_system,
    # :jason,
    :erlexec,
    :runtime_tools,
    # :inets,
    :stdlib,
    :crypto,
    # :hex,
    :elixir,
    # :public_key,
    # :gviz,
    # :mix,
    # :gettext,
    :kernel,
    :ssl,
    :compiler
    # :asn1,
  ]

  defp ensure_applications_started(node) do
    rpc(node, Application, :ensure_all_started, [:mix])
    rpc(node, Mix, :env, [Mix.env()])

    # for {app_name, _, _} <- Application.loaded_applications() do
    #   rpc(node, Application, :ensure_all_started, [app_name])
    # end
    for app_name <- @apps_to_start do
      IO.inspect(app_name, label: "starting app_name")

      rpc(node, Application, :ensure_all_started, [app_name])
      |> IO.inspect(label: "started #{inspect(app_name)}")
    end
  end

  defp rpc(node, module, function, args) do
    :rpc.block_call(node, module, function, args)
  end

  defp peer_args do
    Enum.join(
      [
        "-loader inet -hosts 127.0.0.1",
        "-setcookie #{:erlang.get_cookie()}",
        # "-env MIX_BUILD_ROOT /tmp/gviz_build"
      ],
      " "
    )
    |> to_charlist()
  end

Post #10

yolo007wizard

@LostKobrakai All the debugging above is for the :peer approach in PeerNode
@D4no0 instead of ‘best’ perhaps I should have said ‘easiest’ so far

I’m learning and trying to understand more. If both :peer and CLI approaches start a new beam instance, why is one considered better/worse?

Post #17

derek-zhou

Or you can do what Livebook does. Use a separate BEAM node.

Post #2

Last Post!

yolo007wizard

Architecture ideas so far:

R&D Notes:

The CLI approach has big benefit of caching mix dependencies locally on the container for consecutive action runs
Was good to read over the initial proposal for Mix.install plus looking over the source code.

Why all this?

I’m working on an open source app idea for event handling. It will be an ‘if-this-then-that’ application where ‘if-this’ logic is no-code gui frontend and the ‘then-that’ portion is Elixir code (probably exs mix install scripts). The essence of the idea is to find the right balance between code and no-code / low-code tools.

Post #18