ExPression - evaluate user input expressions

mtvch

I was looking for a library for safely evaluating expressions written by users but didn’t find one that would match my needs. I wanted it to be extensible, support all JSON data types, and have its syntax and semantics intuitive to people who are used to Python.

That’s why I created ex_pression:

Safe evaluation without access to other Elixir modules.

iex> ExPression.eval("exit(self())")
{:error, %ExPression.Error{name: "UndefinedFunctionError", message: "Function 'self/0' was referenced, but was not defined", data: %{function: :self}}}

Support for JSON syntax and data types.

iex> ExPression.eval("""
{
  "name": "ex_pression",
  "deps": ["xpeg"]
}
""")
{:ok, %{"name" => "ex_pression", "deps" => ["xpeg"]}}

Familiar python-like operators and standard functions.

iex> ExPression.eval(~s/{"1": "en", "2": "fr"}[str(int_code)]/, bindings: %{"int_code" => 1})
{:ok, "en"}

Extend expressions by providing Elixir module with functions that you want to use.

defmodule MyFunctions do
  # use $ special symbol in expressions
  def handle_special("$", date_str), do: Date.from_iso8601!(date_str)
  # Use diff function in expressions
  def diff(date_1, date_2), do: Date.diff(date_1, date_2)
end
iex> ExPression.eval(~s/diff($"2023-02-02", $"2022-02-02")/, functions_module: MyFunctions)
{:ok, 365}

Full language description can be found here.

4 comments

#library #script #language-implementation #language #input

2 918 4

2024-01-23 20:42:18 UTC

First 4 of 4 Posts

mayel

Looks interesting! I’m gonna be needing such a library in the future, so would be curious to hear opinions on the pros/cons of approaches taken by similar libs:

which seem to take a safer approach than these that use elixir more directly:

Post #1

mtvch

Hello!

Thank you for your advice, I will work on adding a comparison table to README.md. You can track progress in the issue.

Here is a quick comparison with the libraries you mentioned:

Abacus and Formulae do not support composite types: objects and arrays. Also, they are not extensible: you can only use functions and syntax that library provides.
In this category I would mention evalex which works on top of rust’s evalexpr. But it’s lacking the same features.

Regarding Formula and formula2, I can’t even say much since there is no proper documentation.

Overall, first libraries you mentioned are designed mostly for math calculations, while ExPression is more general purpose. You can do various data transformations or other operations depending on your domain.

I really liked the dune project: it has sessions, support for composite Elixir data types and seems to be extensible. But it expects users to know Elixir, which is not the case for many systems. Most people know Python, that’s why I target this language.

Now I’d like to come to some of ExPression cons:

No sessions: expressions are stateless. Sequences of statements like “x = 1; x + 2” are not supported by design.
The standard library does not have many mathematical functions yet. But it’s not a problem to add them; contributions are appreciated.
There is no control over the execution of expressions like in dune: you can’t specify a timeout, memory or reductions limit. This can also be added if the need arises.
The library is not very stable yet. It has its own parser and interpreter which means there are many cases to handle. But it’s only a matter of time to clear bugs out. The project is important for my company, and we plan to support it.

If you need a library for math calculations - try Formulae. If you need sessions and your users are not afraid of Elixir - try Dune. If your users are likely to know Python or if you want to let users define transformations on JSON data or if you want to extend expressions language for your specific domain - try ExPression.

Post #2

mayel

Thank you, I appreciate you having taken the time

Post #3

Eiji

I believe it would be worth to describe it a bit more. I believe that answering for example below questions may help a common readers to decide if and when to use it.

What are the target users (for example developers)?
What is a use case for evaluating user input (for example evaluating math string)?
In what kind of projects it makes the most sense (for example a complex calculator)?

Why did you choose python-like operators? Because it’s popular or sees as easy for beginners? How does it work with JSON support?

For example here is a first example of search for python and json:

>>> import json
>>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
'["foo", {"bar": ["baz", null, 1.0, 2]}]'

Please pay attention to single and double quotes. In Elixir it’s a common gotcha and that’s why a new sigil for creating charlists as well as it’s inspect behaviour have been introduced in Elixir’s core, so now … Do you support single quotes (there is no mention about it)?

Oh, that’s interesting example!

The link looks empty i.e. [](here). Here is a working one Full ExPressions language description.

master

Not that branch names are so important. Simply saying that it’s more common to use main name.

(…) Arrays (…) Objects (…)

Just for sure I guess that you are using such naming because of python-like operators support, right? Just to let common readers know such naming is used in OOP languages. Elixir’s using List and Map instead.

Special symbols

Do you support or have any plans for supporting custom operators? In your example there is a custom date type using $ symbol. Also is special supposed to be 1-letter long or it’s just an example?

Post #4

— All posts loaded —