NIF returns incorrect results, reuses binaries?

I’m currently in the very early stages of implementing futlixir, a bridge that enables Elixir to call out to Futhark programs. It works by taking a Futhark library and generating a corresponding NIF, similar to how Rustler lets you use Rust programs.

Futlixir is still in a very early stage, but I can currently create arrays and call simple functions with those arrays. Here is an example of what the Elixir code looks like:

c("lib_map.ex") # Import Map.NIF
{:ok, cfg} = Map.NIF.futhark_context_config_new()
{:ok, ctx} = Map.NIF.futhark_context_new(cfg)

xs_binary = <<0, 1>>
{:ok, xs} = Map.NIF.futhark_new_u8_1d(ctx, xs_binary)
{:ok, ^xs_binary} = Map.NIF.futhark_u8_1d_to_binary(ctx, xs)
{:ok, ys} = Map.NIF.futhark_new_u8_1d(ctx, <<1, 4>>)
{:ok, zs} = Map.NIF.futhark_entry_add(ctx, xs, ys)
{:ok, <<1, 5>> = zs_binary} = Map.NIF.futhark_u8_1d_to_binary(ctx, zs)

xs_binary = <<1::integer-signed-64-little>>
{:ok, xs} = Map.NIF.futhark_new_i64_1d(ctx, xs_binary)
{:ok, ^xs_binary} = Map.NIF.futhark_i64_1d_to_binary(ctx, xs)
{:ok, ys} = Map.NIF.futhark_new_i64_1d(ctx, <<1279::integer-signed-64-little>>)
{:ok, zs} = Map.NIF.futhark_entry_add_i64(ctx, xs, ys)
{:ok, <<1280::integer-signed-64-little>> = zs_binary} = Map.NIF.futhark_i64_1d_to_binary(ctx, zs)

The program is very simple, first it creates two arrays of bytes (u8) and adds them together, yielding <<1,5>>, next it takes two arrays of i64 and adds them together, yielding <<1280::integer-signed-64-little>> (the array has size 1).

The problem arises because sometimes the call to Map.NIF.futhark_i64_1d_to_binary will return <<2, 0, 0, 0, 0, 0, 0, 0>>, or indeed whatever xs_binary is set to, instead of the correct result. For some reason, it seems like it’s not correctly allocating a new binary for zs_binary but reusing xs_binary? This only happens some times, and only if the preceding block of code is included as well (the one adding two u8 arrays).

The definition of futhark_i64_1d_to_binary looks like this:

static ERL_NIF_TERM futhark_i64_1d_to_binary_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
{
  struct futhark_context **ctx;
  struct futhark_i64_1d **xs;

  ErlNifBinary binary;
  ERL_NIF_TERM ret;

  if(argc != 2) {
    return enif_make_badarg(env);
  }

  if(!enif_get_resource(env, argv[0], CONTEXT_TYPE, (void**) &ctx)) {
    return enif_make_badarg(env);
  }

  if(!enif_get_resource(env, argv[1], I64_1D, (void**) &xs)) {
    return enif_make_badarg(env);
  }

  const int64_t *shape = futhark_shape_i64_1d(*ctx, *xs);

  enif_alloc_binary(shape[0] * sizeof(int64_t), &binary);

  if (futhark_values_i64_1d(*ctx, *xs, (int64_t *)(binary.data)) != 0) return enif_make_badarg(env);
  futhark_context_sync(*ctx);

  ret = enif_make_binary(env, &binary);

  return enif_make_tuple2(env, atom_ok, ret);
}

Does anyone have a clue what might be wrong?

I’ve uploaded all the files needed to reproduce the issue here: lib_map.c · GitHub

To compile and run it, run the following commands:

gcc -Wall -shared -o lib_map_nif.so -fPIC lib_map_nif.c -lOpenCL -lm
iex --dot-iex test.exs

You’ll probably need to run it a handful of times for the error to trigger.