The NIF resource is destroyed if you put it in the ETS table

Questions / Help
,
1

Hi,

I’am trying to make a NIF resource and put it in the ETS, but the resource is destroyed just immediate after I have put it in ETS. Maybe someone knows what’s the matter?

For example:

iex(43)> :ets.insert :test, {5, TupleNif.tuple_to_binary({1, 2, 3})}  
Destructing array: 0x14042ed0

TupleNif.tuple_to_binary({1, 2, 3} - create NIF resource.
I am added the debug output in the resource destroying NIF function, so as we can see the debug output “Destructing array: 0x14042ed0” indicates that the resource is destroyed.

iex(44)> [{k, v}] = :ets.lookup :test, 5
[{5, #Reference<0.2003688078.3595960321.242503>}]
iex(45)> TupleNif.binary_to_tuple v
Bug Bug ..!!** (ArgumentError) argument error

In the TupleNif.binary_to_tuple NIF function the ArgumentError returned if we can’t get resource from the input parameter.

OTP21

2

Are you able to post the relevant parts of the NIF code?

3

Hi, I think yes. The common things is not a secret…

Array resource struct

static ErlNifResourceType* array_type = NULL;

typedef struct {
  int length;
  int* items;
} Array;

Load the nif

static int on_load(ErlNifEnv* env, void** priv, ERL_NIF_TERM info) {
    ErlNifResourceType* rt = enif_open_resource_type(env, "tuple_nif", "array_type", destruct_array, ERL_NIF_RT_CREATE, NULL);
    if(!rt) return -1;
    array_type = rt;
    return 0;
}

Create resource from tuple

static ERL_NIF_TERM
tuple_to_binary(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) {
    int arity;
    const ERL_NIF_TERM* tuple;
    if (!enif_get_tuple(env, argv[0], &arity, &tuple)) {
        return enif_make_badarg(env);
    }

    Array* array = enif_alloc_resource(array_type, sizeof(Array));

    ... filling array with data...

    ERL_NIF_TERM term = enif_make_resource(env, array);

    // Make Erlang the owner of the resource.
    // If we comment this line than a hanging reference to resource will appear and
    // it will not be removed in case of inserting it to ETS but of course this is just experiment
    enif_release_resource(array); 

    return term;
}

Convert resource to tuple

static ERL_NIF_TERM
binary_to_tuple(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) {
    Array* array = NULL;
    if(!enif_get_resource(env, argv[0], array_type, (void **) &array)) {
        return enif_make_badarg(env);
    }
   ...        
}

Release array resource. Used in the ‘enif_open_resource_type’

static void destruct_array(ErlNifEnv *env, void *arg) {
    printf("\n\rDestructing array: %p", arg);
    Array* array = (Array*) arg;
    if (array && array->length && array->items) {
        free(array->items);
        array->items = 0;
    }
}
4

While not the root cause of your problem, I expect that it is not possible to store structures with pointers inside ETS (but only fully ‘flat’ data), because how would the system otherwise be able to keep track of the pointers and the memory it pointed towards?

2 Likes
5

It seems you are right.
When we pass the created resource to the ETS, probably Erlang take over ownership of the resource by copying resource memory block to new place and calling resource destructor we pass to the enif_open_resource_type for the old resource memory block.

It looks like, when inserting user data into ETS, all data is copied into a special block of memory allocated for ETS and memory of user data is deallocated. Probably, ETS has its own memory block to save ETS data to disk via tab2file quickly.

Thanks!

3 Likes
6

It should be perfectly fine to insert nif resources into ETS tables. Most likely you have done something wrong with the refc of the resources in the nif. What happens if you just call a garbage collection instead of inserting the resource into the etc table?

1 Like
7

If to save the resource to a variable, then everything is fine. We can call garbage collection any times but while the variable is somehow used the resource destructor not called. If we also insert the resource to ETS, everything is fine too. But if we do not assign the resource to a variable and insert it to ETS, then the resource will be destroyed.
In the NIF I have created the resource next way:

Array* array = enif_alloc_resource(array_type, sizeof(Array));
... filling array with data...
ERL_NIF_TERM term = enif_make_resource(env, array);
enif_release_resource(array); 
return term;

If to comment the enif_release_resource(array); than a hanging reference to resource will appear and it will not be removed in case of inserting it to ETS but of course this is just experiment.

1 Like