Ets.fun2ms needs ms_transform error

Hello everyone,

I have some strange error message when using :ets.fun2ms.

I am trying to filter an ets table based on some time.

def get_obsolete_check_in(time) do
  fun = :ets.fun2ms(fn {_, %CheckIn{start_time: start_time}} = check_in when start_time > time ->
    check_in
  end)
  :ets.select(:check_ins, fun)
end

Here is the error message.

** (exit) exited in: :ets.fun2ms(:function, :called, :with, :real, :fun, :should, :be, :transformed, :with, :parse_transform, :or, :called, :with, :a, :fun, :generated, :in, :the, :shell)
    ** (EXIT) :badarg
    (stdlib) ets.erl:580: :ets.fun2ms/1
    (next_engine) lib/next_engine/check_in_server.ex:30: NextEngine.CheckInServer.get_obsolete_check_in/0

This also happens with a simple function like

fun = :ets.fun2ms(fn check_in -> check_in end)
:ets.select(:check_ins, fun)

Looking for an answer I found similar problems in Erlang here. And it seems the Erlang solution would be to include…

-include_lib("stdlib/include/ms_transform.hrl").

How does this translate to Elixir?

Thanks for taking time.

3 Likes

The problem is that the fun must be converted to a match spec at compile time. In Erlang the “ms_transform.hrl” file causes a parse transform to be run which at compile time recognises the call to :ets.fun2ms/1 and converts the fun to a match spec. It basically behaves like a macro. If the function is called at run-time you get the error.

How this is handled in Elixir I don’t know.

EDIT: Funnily enough I have just been working fixing the handling of match specs in LFE (Lisp Flavoured Erlang), amongst other things writing and ets-ms and dbg-ms macros which do this conversion.

1 Like

Thank You for your response.

I tried this simple module, it fails with the same error. But if I type each line in the console, it’s working.

defmodule NextEngine.Koko do
  def do_work() do
    :ets.new(:checks, [:set, :protected, :named_table])
    fun = :ets.fun2ms(fn check_in -> check_in end)
    :ets.select(:checks, fun)
    :ets.delete(:checks)
  end
end

iex> NextEngine.Koko.do_work()
** (exit) exited in: :ets.fun2ms(:function, :called, :with, :real, :fun, :should, :be, :transformed, :with, :parse_transform, :or, :called, :with, :a, :fun, :generated, :in, :the, :shell)
    ** (EXIT) :badarg
    (stdlib) ets.erl:580: :ets.fun2ms/1
    lib/next_engine/koko.ex:4: NextEngine.Koko.do_work/0

I probably need to translate this line in Elixir, but I don’t know yet how to do it.

-include_lib("stdlib/include/ms_transform.hrl").

This is because the shell “knows” about the :ets.fun2ms/1 function and handles it specially, and :dbg.fun2ms/1 as well.

The :dbg version is for match spec for tracing which are a little different, the :ets version is for return data while th :dbg version do tracing commands.

1 Like

I’ve always used https://hex.pm/packages/ex2ms to achieve this in Elixir.

4 Likes

That lib seems to be exactly what I need.

Thank You very much.

I got it working, but it is a bit more complex than previously thought.

for future reference, if You happen to search into an ets table storing Elixir structure by key (here by date). You will need Ex2ms, and use correct syntax.

Here is pseudo code, with 3 functions f, the last one is working.

  import Ex2ms
  ...
  def get_obsolete_check_in(time) do
    # f = :ets.fun2ms(fn {_, %CheckIn{start_time: start_time}} = record when start_time > ^time -> record end)
    
    # f = fun do {_, %Checkin{start_time: start_time}} = record when start_time > ^time -> record end
    
    f = fun do {{_, _, _, start_time}, _} = record when start_time > ^time -> record end
    
    :ets.select(:check_ins, f)
  end

The first try with :ets.fun2ms works in iex, but fails when defined in a module with error.

** (exit) exited in: :ets.fun2ms(:function, :called, :with, :real, :fun, :should, :be, :transformed, :with, :parse_transform, :or, :called, :with, :a, :fun, :generated, :in, :the, :shell)

The second try with Ex2ms fails with error. You cannot extract start_time directly from the Elixir structure.

** (ArgumentError) parameters to matchspec has to be a single var or tuple

The solution is to store start_time in the key, so that Ex2ms can generate a match_spec from a single var, part of the key. Do not forget the little ^ when matching.

Thank You @rvirding and @pmonson711 for your kind help.

1 Like

Yes, while you can put structs in an ETS table you cannot put them directly in the table as the elements in an ETS table must be tuples so the struct itself must one of the elements of the tuple.

1 Like

For :ets.fun2ms to work, you should add @compile {:parse_transform, :ms_transform} into your module.

8 Likes

Thank You for the response. This will remove the need for ex2ms.

After getting some warnings with Elixir 1.6 and Erlang 20.1

warning: @compile {:parse_transform, _} is deprecated. Elixir will no longer support Erlang-based transforms in future versions

I found this issue

https://github.com/elixir-lang/elixir/issues/5762

The better way now seems to be using ex2ms, losing some supports for better compilation.

You can always write matchspecs by hand, that’s what I often do, it’s well documented if perhaps a bit ugly. ^.^;

Sad that parse transforms are going away (think I saw that before a while back…).

2 Likes

For we old lispers matchspecs feel pretty natural, just replace the curlies with parentheses and drop the commas and you are there. :wink:

I am guessing the parse transforms are going to try and get everything into elixir.

3 Likes

Heh, I find them pretty natural in general. ^.^

Rather elixir is compiled to erlang, then the parse transforms are run on the erlang to transform to different erlang. :slight_smile:

1 Like

Yes, I know but that means some of the transformations taking place are “outside” Elixir and not controlled/done in Elixir. Basically adding an underlying hidden macro handling. This would be a way to get everything except the basic implementation into Elixir. Now most of the Erlang parse transforms are very mild compared to Elixir macros although it is pretty easy to do some global changes. For example in our (Erlang Solutions) parse transform course the first, and easiest, exercise is to make all functions in a module affirmative and return ok. The hardest is to convert a record definition into a set function interfaces.

I don’t know if this is the reason.

2 Likes

Thanks a lot
That solve the error :stuck_out_tongue:

Thanks , that worked.