EtsSelect - an ETS match spec builder from a simple query language.

Your Libraries & OS Mentoring Libraries
1

I needed to store some structs/maps in ETS and wanted to use a query language that felt simple and approachable.

After looking at several alternatives, I noticed that all the wrappers around ETS match specifications were doing a bit too much for my taste. This is how EtsSelect came to life. :slightly_smiling_face:

You can dump your maps/structs into an ETS table and select them using the following query syntax:

# "OR" query
%{or: [[:=, :field1, "value1"], [:=, :field2, "value2"]]}

# "AND" query
%{and: [[:=, :field1, "value1"], [:=, :field2, "value2"]]}

# OR query with nested AND
%{or: [[:=, :status, :new], %{and: [[:=, :status, :old], [:=, :age, 50]]}]}

That’s basically it. Check out the ExUnit tests to see if this could be useful to you.

Cheers!

6 Likes
5

Oooh, another match spec library! I definitely welcome alternative DSLs in the space.

Are you interested in adding more support for keys that are tuples? For example in my data_tracer library I insert data that looks roughly like:

{{1723901292213, 0, "age"}, "100"}
{{1723901292214, 0, "age"}, "101"}
{{1723901292214, 0, "name"}, "bob"}

And I want to do a query to get all the values where the third element in the tuple key is "age", i.e. something like :ets.select(:my_table, match_spec) == ["100", "101"]. From what I can tell this isn’t currently possible to query via ets_select, is that right?

I also submitted a quick PR to remove the test files from the published package: Exclude test files from published package by axelson · Pull Request #1 · maxohq/ets_select · GitHub

3 Likes
6

@axelson Thanks for the encouragement!

While it could be possible, it would change the original goal of the package. It seems you are storing arbitrary values for debugging in data_tracer, yet my use case is selecting based on field values in maps / structs.

BTW, GitHub - axelson/data_tracer: Elixir library to facilitate debugging data flow, helps capture terms for later inspection looks awesome, gonna try it out soon. If you want to have a query language for data_tracer, please give some example queries. IF your only query type is selecting on the 3rd element of the key tuple, there is probably no need for a 3rd-party library :slight_smile:

The spec to match all elements with “age” OR “name” would look like this:

match_spec = [
    {{{:_, :_, :"$3"}, :_}, [{:orelse, {:==, :"$3", "age"}, {:==, :"$3", "name"}}], [:"$_"]}
]

So, please share some examples, maybe I could contribute a query language just for data_tracer, that would be most simple and direct approach.

1 Like
8

It seems the only place where you execute a query on ETS tables is this

Exactly this query should be covered by this spec:

match_spec = [
    {{{:_, :_, :"$3"}, :"$4"}, [{:==, :"$3", key}], [:"$4"]}
]

Example:

:ets.new(:my_table, [:set, :public, :named_table])

:ets.insert(:my_table, {{1723901292213, 0, "age"}, "100"})
:ets.insert(:my_table, {{1723901292214, 0, "age"}, "101"})
:ets.insert(:my_table, {{1723901292214, 0, "name"}, "bob"})

key = "age"
match_spec = [
    {{{:_, :_, :"$3"}, :"$4"}, [{:==, :"$3", key}], [:"$4"]}
]
result = :ets.select(:my_table, match_spec)
#=> ["101", "100"]

So… In theory you could drop your dependency on Matcha.

1 Like
9

@axelson I went ahead and dropped matcha in this PR: Feat: drop `matcha` package by mindreframer · Pull Request #5 · axelson/data_tracer · GitHub

It should be 100% exact behaviour, just without all the non-trivial abstractions provided by GitHub - christhekeele/matcha: 🍵 First-class Elixir match specifications..

If you want a proper query language, this should be done separately.

Have a good Saturday!

2 Likes
13

Fair enough! All libraries need to define the scope that they want to tackle :+1:

Thank you for the kind words! I don’t have a plan to add a query language for data_tracer, the use case is mainly getting stored values out ordered by insertion and only for specific keys. For my usage of it I probably don’t even need to worry about efficient ETS lookups and could instead dump the whole table and filter it, but I wanted to take it as an opportunity to work with ETS in a more efficient manner.

Thank you for the contribution! I’ve merged the PR and released it as version 0.1.2 :tada:

The reason matcha was even there was two fold: I wanted an excuse to try out Matcha, and I find match specs inscrutable to the point where I’m not super interested in learning them.

15

@axelson

Yeah, the specs are really painful to read / write / understand. But… Just after having implemented he EtsSelect package, it was bearable to come up this those 2 match-specs. And having less dependencies is always nice :slight_smile:

Thanks for merging the PR!

@dimitarvp Yeah, I’m quite familiar with the great, even outstanding quality of Elixir docs. I just dont need to lookup something, that is already working :slight_smile: Only when things dont behave as expected, I dig deeper. So 100% agree with you on this one.

Thanks for chiming in and good night (its about midnight in EU)!