I’m using a C/C++ library that is not thread-safe and requires some initialization. I’m trying to understand if I can use it safely as a nif across multiple processes, but I’m not sure of how to confirm this.
I’m thinking that if the memory allocation of the nif happens on the stack/heap of the erlang process that is calling it, that it should be fine? However, I’m not sure how to confirm this. I suppose if the library does any memory initialization, it may be outside the erlang process also.
Anyone with advice / experience on this topic? My gut instinct at this point is to make an external process that forks on each request to handle getting results from this library.
There are two main reasons why C/C++ code might not be thread-safe:
It is relying on global state (global variables, static inline variables, etc.)
It is sharing data between (OS level) threads without proper checks that might prevent data races.
If (1) is the case, then there is very little you can do; it would require re-writing the original library. The best you might be able to do in this case is instead wrap the program in a separate executable and interact with it through a Port (or multiple ports, one per request or maybe a pool of them).
If (2) is the case, then you might be able to write your code in the way where data is not shared between threads, or where you add the proper checks on top.
Thanks for the confirmation, I’ve never written multi-threading code before, so that was very useful. Looks like the library definitely belongs outside of BEAM.
More information about nifs and memory
Looks like the shared library will allocate memory from the calling OS process. For it to allocate memory on the calling erlang process, there would also need to be a mechanism for garbage collection to work.
The nif interface provides mechanisms for memory and thread management:
