Elixir with Erlang fun2ms -> ETS

Questions / Help
, ,
1

Hi, does anyone know how I can create a matchspec for this condition?

matchspec = :ets.fun2ms(fn({key, test}) when key != :counter -> test end)

[error] exited in: :ets.fun2ms(:function, :called, :with, :real, :fun, :should, :be, :transformed, :with, :parse_transform, :or, :called, :with, :a, :fun, :generated, :in, :the, :shell)
        ** (EXIT) :badarg

I can get the result I want with:

test = :ets.foldl(fn({key, obj}, acc) -> case key != :counter do
                                           true -> [obj | acc]
                                           _ -> acc
                                         end
 end, [], :queue_cache)

or

 test = :ets.foldl(fn({key, obj}, acc) when key != :counter -> [obj | acc]
                   (_, acc) -> acc
 end, [], :queue_cache)

But fun2ms seems a bit cleaner to write - I’m not sure I understand the restraints in fun2ms, perhaps is my when condition that’s not valid?

2

I had a similar question…

I think it’s safer to use ex2ms (Elixir to match spec) for the moment.

1 Like
3

thank you! I ended up going to the console and see what match specification it output and I got this:

[{{:"$1", :"$2"}, [{:"/=", :"$1", :counter}], [:"$2"]}]

Which seems to be correct to my very thin knowledge of Erlang,
So I tried:
test = :ets.match(:queue_cache, [{{:"$1", :"$2"}, [{:"/=", :"$1", :counter}], [:"$2"]}])

but it won’t match anything, I’ve tried with :"counter" as well

4

In your second example, /= has turned into \=?

5

ups, that was after trying a couple variations, it’s :"/="

6

The reason :ets.fun2ms/1 works like this is because it is basically a macro. To use this in Erlang you need to include a -include_lib("stdlib/include/ms_transform.hrl"). in your Erlang file. (it’s in the docs) This runs a parse transform which at compile-time transforms the fun into the match spec. If this transformation is not done at compile time then the :ets.fun2ms/1 function is called at run-time and it gives that rather clumsy error message.

This include also the :dbg.fun2ms/1 “macro” which is used to build match specs for tracing. They are very similar to the ones for tables but not quite the same.

Parse transforms are like macros on steroids. The parse transform’s module/2 function is called with the whole Erlang module AST as an argument with which it can do whatever it likes. It just has to return a new complete module AST. Quite fun once you get the hand of it.

6 Likes
7

Thank you for replying Robert - I did read the other thread and the docs as well. The error message was quite explicit (although presented in sort of a lispy fashion) I was just unsure how to include the matchspecs transform file when in elixir (actually I thought it would be somehow baked in).

I definitively need to dig a bit more into erlang.

Is that why you named these functions as “fun2…”? (j/k)

I still haven’t figured out why the match spec that I got from the terminal won’t work? I imagine that the one we retrieve in the shell will be equal to the one that is parse-transformed at compile-time right? Or am I missing something? I did also check the match specification in the erlangs docs

[{{:"$1", :"$2"}, [{:"/=", :"$1", :counter}], [:"$2"]}]

[{ { MatchHead }, [{ Guard Function }], [ MatchBody#$2_bound_in_matchhead ] }]

But it doesn’t return me any records and I haven’t figured yet what I’m doing wrong (I haven’t looked much more into it because :ets.foldl solves it right now)

8

Well it should only return the test value. I get it to work from the shell:

iex(8)> :ets.new(:qc, [:named_table,:bag])
:qc
iex(9)> :ets.insert(:qc, [counter: 1, foo: 2, counter: 3, bar: 4])
true
iex(10)> :ets.tab2list(:qc)                                        
[counter: 1, counter: 3, foo: 2, bar: 4]
iex(11)> ms = :ets.fun2ms(fn({k,t}) when k != :counter -> t end)   
[{{:"$1", :"$2"}, [{:"/=", :"$1", :counter}], [:"$2"]}]
iex(12)> :ets.select(:qc, ms)
[2, 4]

The name isn’t a joke, unfortunately. He who wrote the module like the shorter 2 instead of _to_. It does crop up in a few other places in the libraries as well. :anguished:

1 Like
9

Maybe the problem is that you’re giving the match spec to :ets.match, when it needs to be given to :ets.select? match accepts a pattern instead of match spec, if I’m not mistaken.

1 Like
10

Thank you @rvirding and @Nicd - indeed the problem was simply using a matchspec with match instead of with select (also known as "overlooking the obvious"TM)

for my defence though a match spec would seem like the right fit for a match function

1 Like
11

Maybe it should be called a select spec. :slight_smile:

2 Likes
12

Well, how tracing uses match specs is a bit different so calling them a select spec would not fit in there. And they are so close that giving them different names would be stupid. The main difference is what you can do in the body: in tables you can basically just return data while in tracing you can control how tracing works.

2 Likes
13

I definitively need to dig through the docs as I’ll be using quite a bit of ETS (btw I really like them, but until now I have only used them for unique key->value, with no need to do other stuff with them).

2 Likes
14

I’ve run into the same issue.

When I run:

iex(1)> :ets.fun2ms(fn x -> x end)
[{:"$1", [], [:"$1"]}]
iex(2)>

In the iex shell, it works. But when I try to use it in my module:

  def list() do
    :mnesia.transaction(fn ->
      :mnesia.select(mnesia_table_name(), :ets.fun2ms(fn x -> x end))
    end)
  end

I get the error

{:aborted,
 {:badarg,
  {:ets, :fun2ms,
   [:function, :called, :with, :real, :fun, :should, :be, :transformed, :with,
    :parse_transform, :or, :called, :with, :a, :fun, :generated, :in, :the,
    :shell]}}}

What’s strange and why in my case the solution proposed earlier doesn’t work is that when I replace the :ets.fun2ms with the MatchSpec which I get in the iex shell, like so:

  def list() do
    :mnesia.transaction(fn ->
      :mnesia.select(mnesia_table_name(), [{:"$1", [], [:"$1"]}])
    end)
  end

it works.

15

The :ets.fun2ms is a placeholder function. In erlang code a “parse transform” is run over your sourcecode, which replaces all calls to the :ets.fun2ms with the matchspec at compiletime. It comes close to macros in elixir, but not quite.

There is a package that can be used from within elixir code:

16

Thanks for the reply. All clear now.

I’m going to give ex2ms a shot, because I’d rather avoid more complex matchspecs in my code because I find them rather hard to read and maintain.

17

Added

      {:ex2ms, "~> 1.6"},

to my mix.exs but something isn’t working:

iex(1)> import Ex2ms
** (CompileError) iex:1: module Ex2ms is not loaded and could not be found
18

Have you done mix deps.get and started iex as iex -S mix?

19

Yes, I have

$ grep ex2ms mix.exs                                                                                                     ‹ruby-2.4.4›
      {:ex2ms, "~> 1.6"},


$ mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  amqp 1.4.2
  amqp_client 3.8.3
  cachex 3.2.0
  cowboy 2.7.0
  cowlib 2.8.0
  credentials_obfuscation 1.1.0
  eternal 1.2.1
  ex2ms 1.6.0
  goldrush 0.1.9
  jsx 2.9.0
  jumper 1.0.1
  lager 3.8.0
  mime 1.3.1
  plug 1.10.0
  plug_cowboy 2.2.1
  plug_crypto 1.1.2
  poison 3.1.0
  rabbit_common 3.8.3
  ranch 1.7.1
  recon 2.5.0
  sleeplocks 1.1.1
  telemetry 0.4.1
  unsafe 1.0.1
All dependencies are up to date

$ iex -S mix
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe] [dtrace]

iex(1)> import Ex2ms
** (CompileError) iex:1: module Ex2ms is not loaded and could not be found
20

Works for me:

$ nix run nixpkgs.elixir_1_10 -c mix new foo
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/foo.ex
* creating test
* creating test/test_helper.exs
* creating test/foo_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd foo
    mix test

Run "mix help" for more commands.
$ cd foo
$ emacs mix.exs
Waiting for Emacs...
$ nix run nixpkgs.elixir_1_10 -c mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
New:
  ex2ms 1.6.0
* Getting ex2ms (Hex package)
$ nix run nixpkgs.elixir_1_10 -c iex -S mix
Erlang/OTP 22 [erts-10.7] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [hipe]

==> ex2ms
Compiling 1 file (.ex)
Generated ex2ms app
==> foo
Compiling 1 file (.ex)
Generated foo app
Interactive Elixir (1.10.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> import Ex2ms
Ex2ms
1 Like