How to take some data which contains valid Elixir code format and execute it as Elixir code?

zwpdbh

I am doing a POC for implementing an E2E workflow test application. And I think Elixir is perfect for such use case.

There is a feature I want to implement is: Given a set of existing steps(functions), the user should generate new workflow by themselves.
For example:

defmodule St.Workflow do
  def get_random_str(params) do
    IO.puts("generate random_str")
    # assign random string into _params

    params
  end

  def create_resource_group(params) do
    IO.puts("create resource group")

    params
  end

  def run_helm_exec_01(params) do
    IO.puts("do helm 01")
    params
  end

  def run_helm_exec_02(params) do
    IO.puts("do helm 02")
    params
  end

  def run_helm_exec_03(params) do
    IO.puts("do helm 03")
    params
  end

  # St.Workflow.workflow_01(%{})
  def workflow_01(params) do
    get_random_str(params)
    |> create_resource_group
    |> run_helm_exec_01
    |> run_helm_exec_02
    |> run_helm_exec_03
  end
  
  def install_helm(params) do
    run_helm_exec_01(params)
    |> run_helm_exec_02
    |> run_helm_exec_03
  end

  # St.Workflow.workflow_02(%{})
  def workflow_02(params) do
    get_random_str(params)
    |> create_resource_group
    |> install_helm
  end

  def eval_workflow(workflow_definition) do
    # A workflow_definition could be combination of existing functions in this module.
    # workflow_definition = "run_helm_exec_01(params)|> run_helm_exec_03"

    # how to run this? 
  end
end

4 comments

#learning-elixir

0 552 4

2022-03-28 11:55:47 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

cmo

You can collect the functions and args to call and then apply them.

Post #1

csadewa

Do you mean you want to implement Plugin/Pipeline pattern? in which a set of Plugin/Pipeline can be arranged customly by client to produce a particular workflow? in Elixir there’s a lot of inspiration for this pattern, like Plug v1.20.2 — Documentation or internals of Overview — absinthe v1.11.0.

At it’s core, most plugin pattern in elixir usually:

Define individual plugin as module
arrange steps of plugin in a list
Have a set of helper function for handling context of execution

Then you could execute the list of plugin by repeatedly calling apply on them.

Example of simple plugin pattern

defmodule GenerateRandomString do
  def call(execution_context, opts) do
    put_execution_context(execution_context, :random_string, some_random_string)
  end
end

defmodule CreateResourceGroup do
  def call(execution_context, opts) do
    ... do something here
  end
end

def execute_workflow(workflow) do
  context = new_execution_context()
  Enum.reduce(workflow, context, fn {module, opts}, context ->
    apply(module, :call, [context, opts])
  end)
end

workflow = [
  {GenerateRandomString, []},
  {CreateResourceGroup, [namespace: "dev"]}
]

execute_workflow(workflow)

# Unwrapped of execute_workflow, for step by step visualisation
# context = new_execution_context()
# context = GenerateRandomString.call(context, [])
# context = CreateResourceGroup.call(context, [namespace: dev])

Post #2

mpope

Executing arbitrary code is dangerous. See this example of what can be done: GitHub - koudelka/annelid: Unwelcome, Replicating, Evasive, Self-Healing Infrastructure for Elixir 💪🐛☣︎ · GitHub You are much better off either coming up with exposing a subset of ‘functions’ to the users and calling them yourself manually based on pattern matching, or creating some sort of DSL that you can evaluate using leex and yecc. With the parser-generator based approach, it is reasonable to chain valid predefined functions. It’d be much easier to have the user submit {"params": "xyz", "funs": ["fun1", "fun2", "fun3"]} and control what is called.

Post #3