Performance considerations when using Code.eval_string/2

Hi,

I’m considering using a dynamically configured “filter expression” and Code.eval_string/2 to filter events through a GenStage pipeline. I’m concerned with possible issues regarding the Erlang VM resource usage when calling this potentially many times a second. Will this end up overflowing memory? Should I instead manage a cache of pre-compiled expressions and avoid calling eval_string too often?

I appreciate any input from someone with better understanding of how code compilation works.

Paulo

1 Like

I would avoid calling Code.eval_string/2 at runtime entirely. Can you elaborate a bit more on what you’re evaling, and in what context, so that we can suggest some alternate approaches?

3 Likes

I have a message (SMS/Email) delivery engine that can be configured to send on certain events passing through the application event stream, by configuring a list of triggers. A trigger is a Struct specifying the matching event type, delivery method and delivery options.

I now have the need to actually look into the event attribute values, not only the event type, to decide if a message should be sent for a specific event.

My initial solution is to add a filter_expr to the trigger struct. This would allow defining something like “e.name == “Paulo”” and by testing the value of `Code.eval_string(“e.name == “Paulo””, e: event), decide if a message is sent.

The filter_expr is not set by an external user, so I’m not really concerned about unsafe code. My concern is that by calling eval_string I’m growing the VM memory reserved for code to the point of eventually triggering an OOM.

1 Like

Code.eval_string runs an interpreter, not the regular compiler (similar to iex), so it’s safe in the regard of “growing the code space” - no code is produced.

It can be potentially quite slow, though.

2 Likes

This doesn’t seem to be a complex enough usecase to warrant it, but this kind of embedded logic is a great fit for lua via https://github.com/rvirding/luerl

2 Likes

Thanks for the replies.

I went ahead with Code.eval_string/2 and while it’s at least an order of magnitude slower than running compiled code, it’s good enough for the current event throughput. The expressions I’ll use are quite small. Basically what would go into the condition of an if control statement.

1 Like

I’d consider just having a small datastructure to give you the options you want.

1 Like

But it will also allow ~S'System.cmd("rm", ["-rf", "/"])'.

Do not do eval in any language, you can’t be sure that the input can be trusted.

But by whom? Even “internal” users may be hostile to your application, just look at trumps twitter downtime a couple of weeks ago…

4 Likes