Modify Data based on dynamically loaded script/code?


I want to load scripts/code and call it with a data structure (for example a map) to receive some modified result.

  1. Transformer loads script/code
  2. Transformer calls loaded code with data
  3. Transformer receives modified data
  4. Transformer unloads script/code

I solved something like this in Java with groovy scripts that were located in a folder and automatically loaded. But i do not know how i could solve this with elixir.

Is there anything widely used like Groovy scripts to solve this? Or is there a different and/or better way to solve this with elixir?

Very easily done in about every immediate way that I can think of. What precisely are you wanting to do?

I think an example should clarify what i want to do:

I have a map like this:

%{table1: [{1,1}, {2,2}, {3,3}, ...], table2: [{1,1}, {2,2}, {3,3}, ...]}

I want to do something like this:

pie_chart :table1,
  as: 'Chart 1'
  x: fn x -> x end,
  y: fn y -> y*2 end

and i want this piece to be changeable at runtime.
(It should be loaded from a defined folder, from a database or something like that)

The expected result would be something like this:

%{pie_charts: [{"Chart 1", [1,2,3,...], [2,4,6,...]}]}

That then begs the question, what is pie_chart in your example since you say you want it changeable? A function call? Where does it get the map/data from? Etc…? :slight_smile:

Yes, i thought of pie_chart in this case as a predefined function, supplied with the user defined arguments as showed above. The complete file should have mutlple possible functions like (line_chart, etc.) and multiples of each.

The map data should be passed to it alongside the customizable arguments.

Are the arguments of x and y (the anonymous functions) baked in at compile time? If so then this will do:

def process_charts(input_map) do
  charts =, fn(name, data) -.
      pie_chart(name, as: 'Chart 1', x: fn x -> x end, y: fn y -> y*2 end, data)
  %{pie_charts: charts}

Or so. You were not feeding in your data into the pie_chart anywhere so I put it at the end for this example. Need more details so I can be more detailed. :slight_smile:

The way you call that function all depends on where that user data is being stored. Can you give a more complete psuedo-code example of what you want done?

I’ll give a more thorough explanation and code example tomorrow. Thanks for your time so far!

1 Like

Okay, here is the basic structure where i want to do what i described:

defmodule Example.Transformer do
  use GenServer

  # Api
  def start_link do
    GenServer.start_link(__MODULE__, :ok, [])

  def run(pid, map) do, {:run, map})

  # Callbacks

  def init(:ok) do
    {:ok, %{}}

  def handle_call({:run, map}, _from, _state) do
    new_state = transform(map)
    {:reply, :ok, new_state}

  defp transform(map) do
    tranformation_rules = load_rules()
    do_tranformation(map, tranformation_rules)

  defp load_rules do

  defp do_transformation(map, rules) do


I thought of it like this: In the best case the rules are just an module which uses some predefined module which has the functions like pie_chart, line_chart, etc. all the “do_transformation” method would have to do in that case would be calling a “run” method on the rule module.

I am not sure if this would be the right way to do this or how i could achieve this. The reason i want the “rules” to be a module is that i want to be able to define almost anything in that code and then put it in the predefined functions like pie_chart, line_chart, etc.

If you don’t know what the code will be at compile time, you’ll have to use Code.eval_quoted to load the code you want, but I don’t believe that anyone will recommend actually doing that in a production system. Elixir sips did a rules engine a while back that showed two approaches of doing this at compile time vs runtime and the performance impacts of it. I can’t think honestly of any requirement that would warrant new code without a recompile/deploy as you’d most certainly want to test your new code and make sure it does as expected.

I would recommend building with macros an easy way to add new transformations, but not to be able to do it at runtime.

This is probably the one you are talking about?

I want to expose the “rule” part to be changed by a customer, without the customer having to deploy to our systems (i do not think that would be a great idea at all).

I’ll take a look at the elixir sips episode too see if this would be applicable to my problem. Thank you for that hint.

I don’t know the specifics of your situation, but I agree with @mgwidmann that dynamically loading executable code from untrustworthy sources is not an idea to take lightly. Some things that I might consider are the language luerl (lua on the BEAM) or ruby/python/javascript code that my app sends to a port process for evaluation. Or even write a parser and interpreter myself for a custom mini-language. eval_quoted would be the very last resort.

Okay, watching the Elixir drip helped my understanding of the part in which i would apply the rules. I still have issues with the process of generating the rules based on some input from the customer. The Elixir drips says at the end:

Now all that would remain to build this out would be to provide a DSL for building the rules, […]

How would i go about building a DSL for this?

I´ve read

But based on that, the customer would still end up writing a module.

From what I understand of your specs, which is very little, maybe a better approach is to have code components that they can in turn combine together to produce the result they desire. If you look at how things like Ecto.Multi or Stream work, they produce a struct which is basically a large set of pure data that describes the instructions that need to be performed. This is where functional programming really shines, your customer creates that data describing what needs to be done and you write the part that will execute those “instructions”.

If your customer can’t handle deployment of the codebase they can’t be trusted with writing Elixir modules either (nor will they be particularly thrilled with having to do so). Sounds like they want a powerful configuration interface, as long as you deliver that then everything will be great.

As an aside, I don’t think this pertains just to Elixir, I wouldn’t suggest the strategy you described in groovy because of the potential security holes being so large it isn’t worth the functionality, no matter how powerful it may be.

1 Like