I am doing a POC for implementing an E2E workflow test application. And I think Elixir is perfect for such use case.
There is a feature I want to implement is: Given a set of existing steps(functions), the user should generate new workflow by themselves.
For example:
defmodule St.Workflow do
def get_random_str(params) do
IO.puts("generate random_str")
# assign random string into _params
params
end
def create_resource_group(params) do
IO.puts("create resource group")
params
end
def run_helm_exec_01(params) do
IO.puts("do helm 01")
params
end
def run_helm_exec_02(params) do
IO.puts("do helm 02")
params
end
def run_helm_exec_03(params) do
IO.puts("do helm 03")
params
end
# St.Workflow.workflow_01(%{})
def workflow_01(params) do
get_random_str(params)
|> create_resource_group
|> run_helm_exec_01
|> run_helm_exec_02
|> run_helm_exec_03
end
def install_helm(params) do
run_helm_exec_01(params)
|> run_helm_exec_02
|> run_helm_exec_03
end
# St.Workflow.workflow_02(%{})
def workflow_02(params) do
get_random_str(params)
|> create_resource_group
|> install_helm
end
def eval_workflow(workflow_definition) do
# A workflow_definition could be combination of existing functions in this module.
# workflow_definition = "run_helm_exec_01(params)|> run_helm_exec_03"
# how to run this?
end
end
Do you mean you want to implement Plugin/Pipeline pattern? in which a set of Plugin/Pipeline can be arranged customly by client to produce a particular workflow? in Elixir there’s a lot of inspiration for this pattern, like Plug v1.13.4 — Documentation or internals of Overview — absinthe v1.7.0.
At it’s core, most plugin pattern in elixir usually:
Define individual plugin as module
arrange steps of plugin in a list
Have a set of helper function for handling context of execution
Then you could execute the list of plugin by repeatedly calling apply on them.
Example of simple plugin pattern
defmodule GenerateRandomString do
def call(execution_context, opts) do
put_execution_context(execution_context, :random_string, some_random_string)
end
end
defmodule CreateResourceGroup do
def call(execution_context, opts) do
... do something here
end
end
def execute_workflow(workflow) do
context = new_execution_context()
Enum.reduce(workflow, context, fn {module, opts}, context ->
apply(module, :call, [context, opts])
end)
end
workflow = [
{GenerateRandomString, []},
{CreateResourceGroup, [namespace: "dev"]}
]
execute_workflow(workflow)
# Unwrapped of execute_workflow, for step by step visualisation
# context = new_execution_context()
# context = GenerateRandomString.call(context, [])
# context = CreateResourceGroup.call(context, [namespace: dev])
Executing arbitrary code is dangerous. See this example of what can be done: GitHub - koudelka/annelid: Unwelcome, Replicating, Evasive, Self-Healing Infrastructure for Elixir 💪🐛☣︎ You are much better off either coming up with exposing a subset of ‘functions’ to the users and calling them yourself manually based on pattern matching, or creating some sort of DSL that you can evaluate using leex and yecc. With the parser-generator based approach, it is reasonable to chain valid predefined functions. It’d be much easier to have the user submit {"params": "xyz", "funs": ["fun1", "fun2", "fun3"]} and control what is called.
