We are looking to build an enterprise application for processing online messages and part of the validation/checks of the business logic to be applied to the message preferred to be executed using configurable scripts with simple syntax for admin UI users and with access to internal messages structures , those ones are checked/validated when entered on UI and hopefully complied in order to have maximum performance.
Any resources you recommend to achieve this using Elixir?
We will have messages coming in different formats like ISO20022 or ISO8583 or REST API , hence we are going to map it first to internal data structure “internal message” , this will have all the fields that should be accessible to the admin UI user to build scripts to handle it in case the parameters we included in the configuration of business logic did not cover the business use case so they can add custom handling or validation.
I hope it is clearer now.
Not just be careful, but this is effectively not the way to go. Evaling code by users means giving people full access to wherever elixir runs.
This usecase really needs a sandboxed language. You can e.g. look at luerl. TVLabs has recently been vocal about using it in their system for people to build workflows with.
Yes , it is in payments domain.
Access to fields usually restricted to read and in minor cases to set certain defined fields values besides triggering an action to reject message through setting some result code field to non approved.
The users who have access are not the regular customers but the admin users who will enter those scripts through defined UI access and permissions
I’m neither that strict nor that eval-friendly. I think that with great care and security measures, and depending on the context, it can be useful. Otherwise, they wouldn’t include it in the language. If it can be avoided, all the better, but if not, and in a very controlled environment, it can be useful.
I would try first what @LostKobrakai commented… I didn’t know about it and seems good though
What about if it is very simple phrases limited to certain functions?
as an example , let us say we pass the message to the service or system responsible about checking fraud but somehow a case reported about new scenario that can be detected through checking the value of a field not passed to this service , in this case all we need to do in the script is something like :
if int_message.abc_field like '%Crypto%' then int_message.response_code = rejected
This is entered by the UI admin user , parsed and compiled without any file or config deployment to the production environment hence it is more flexible to solve such scenarios
I found this lib that might be useful (never tried and I don’t know how safe it is)
Other approach to have a “sandbox” could be prepare a isolated docker instance with an API, send the message struct as erlang term term_to_binary and the code to execute in base 64 or something… then execute it inside the docker and get the result.
Enqueue the jobs in Oban or your own GenServer with retries. Destroy and restart the docker every X time… If the job is sent meanwhile the docker is re-building Oban will try again in a while.
The Oban worker can broadcast a message to pubsub when success, and your users are notified wherever they are.
You might need to ask yourself:
Is response speed really important, or can I create an enqueued async job?
Does the code execution require the context of my current app in addition to the message struct?
If simple phrases are good enough then something like lua should be good enough and someone doesn’t need to get accustomed to a homebrewed subset of what once was elixir, but they can use lua like any other lua code, while you don’t need to go sanitize the input and worry you forgot something allowing people to do whatever they like to do with your machine.
Actually the response speed is really important as those are online messages with timeout limits usually in few seconds for the whole response and hence this script execution should take few moments.
By the way thanks for finding out about the library as its use in Beacon CMS is interesting since it has a similar concept where you want to do some changes to a live application without going through the release cycle and the Beacon CMS admin user will do the change.
And then they still can use Kernel.apply and other ways you do not image. Blacklisting is a rabithole and users will work around it because “we need the feature now, we’ll refactor later” where later is never
That is one reason for using Luerl. You can setup Luerl so that the Lua code it runs is completely sandboxed, only allowing the external calls you choose. It gives you a controlled full power Lua.
100% agree here @LostKobrakai! This sounds like a recipe for exploits and vulnerabilities. Unless you absolutely need the ability to evaluate code, you’d be much safer (and in finance safety is paramount isn’t it?) to find another approach.
May I suggest then that if there’s no other way to deal with this than embedding a language take a look at Racket. Racket is designed for building languages (DSL’s too.)
But I’d really try to find another approach. Embedding languages is dangerous and error prone and there are a lot more ways that some clever black hat can exploit your system than you can think of to protect it. I mean it sounds a lot like you’re doing direct SQL and that leads to SQL injection.
Actually what we are looking for is not to offer full coding capability but more of a way to make the application flexible to accept/reject messages and set some optional fields on the fly without the need for new code deployment “only script through the UI used by Admin”.Here the object that will be accessed is the internal data structure that represent the message fields and all related data added through configuration or processing no database or files access. To reiterate the example I mentioned above , let us assume we find out about new fraud transactions pattern that our current fraud service cannot stop “field not being passed to fraud service” , we can use this script to check related field and if match the pattern set the reject reason code field.
