Dynamic runtime code or scripts

Working on an application that will receive messages whether API or other formats
so I should parse it based on each external message format then move this data to internal data format like a message mapper then based on some business logic build the external message format or API based on destination.

the issue is that business logic changes frequently and hence enabling GUI configuration to do mapping from internal data to external message fields is required but I wonder if we can have also a way to make it more flexible by enabling admin users to do dynamic scripting or code to manipulate certain data structures or fields , any advice how we can do this with elixir ?

This highly depends on how capable are your admin users and what kind of mapping we are talking about.

There are few notable options:

  1. Use a safe templating DSL like mustache. This is the most limited option in terms of flexibility as you can’t execute custom code, however is the safest one from the point of security;
  2. Use elixir EEx. This option combines complete flexibility and a declarative approach, but at the same time it is the most dangerous one as the executed code is not sandboxed. You can use this in combination with something like dune to limit the allowed functionality, but it still has some security holes;
  3. Use a fully sandboxed solution like luerl. This combines both full flexibility with a great sandbox and doesn’t require messing with external processes as the stack machine used to run lua is implemented in erlang as pure data. The only downside is that you will have to write the scripts in lua, even though I don’t think this is a problem if the admins know how to write code;
  4. Start a separate runtime on demand. I’ve seen some commercial elixir projects using javascript sandbox runtimes to run their custom business logic in a safe manner. This option I would recommend the least simply because of performance penalty and potential risk of leaking processes and memory.
1 Like

4 = DenoEx v0.5.0 — Documentation?

Here is the project: GitHub - OpenFn/lightning: OpenFn/Lightning ⚡️ is the newest version of the OpenFn DPG and provides a web UI to visually manage complex workflow automation projects.

Now looking at it again, I think this is more of a plugin system where the plugins are written in JS.

1 Like

Interesting , actually when I was in my early years I worked in the implementation team for a software company who have switching system " The system responsible about processing cards/POS/ATM transactions " and one of its main selling points was its message mapper besides its flexible configuration and then its scripting engine.
I am not C++ expert but the application allowed users to write scripts that is included in the authorization logic or any other flow of handling a transaction manipulating the internal data structure created from parsing the incoming message plus new fields created by the engine with simple conditional logic and updates for certain fields then simple compile for the script and something called warmboot to load it in memory and make it in use.
Just as an example for the need , let us assume you find out you need to reject certain transactions based on new scenario that you did not think about when building the GUI configuration then you can use quickly scripts .

Thanks for you valuable feedback