Hi,
I am about to use NIF to embed some C functions to my elixir project. Should I change all the C functions to their NIF version (with static ERL_NIF_TERM etc.)? I am confused where all these functions should be implemented? Are they replacing the original functions or I should append them somewhere in the C file?
Thanks
What exactly are you referencing? C functions where?
If you are talking about creating a NIF, just put at the NIF boundary the minimal API necessary to do the work. You want to cross the NIF call boundary is little as possible as it has non-trivial overhead.
I meant should I create new C files with NIF definitions in my elixir project? (Which is basically rewriting the C code with NIF). The C code is in its own library, I donât want to add NIF constructs in it.
I have another question, how can I define a callback parameter, better said, how do I convert something like this with NIF library:
my_C_func(int (*Callback)(int x, struct y *buffer))
Then donât, write some wrapper functions. The BEAM doesnât care what else your functions are calling, it just cares that some functions with the required signatures are exposed. Since your NIF functions are going to receive an argc / argv pair rather than actual parameters, itâs probably a good idea even if your C library was only written to be a NIF library.
Callback-style function calls probably arenât what you want in Elixir, especially when using NIFs, since you need a process to execute the function on, asynchronously. I would adapt a callback-based API to one which specifies some pid to which a message is sent when the operation is finished, and you could even write an actual callback-style API using some sort of GenServer on top of that, which keeps track of outstanding requests and calls the appropriate callback / function whenever one is done. Your NIF wrapper function would provide the C function call with a C callback which would place a message in the appropriate process mailbox on completing the operation (something which you can do even without a process context with the NIF API).
I am very new to both elixir and NIF so I need these to be dumbed down.
Then donât, write some wrapper functions
How do I write wrapper functions? Can you give me a pseudo example (I am not sure if this is more C related or NIF related).
Callback-style function calls probably arenât what you want in Elixir, especially when using NIFs, since you need a process to execute the function on, asynchronously. I would adapt a callback-based API to one which specifies some pid to which a message is sent when the operation is finished, and you could even write an actual callback-style API using some sort of GenServer on top of that, which keeps track of outstanding requests and calls the appropriate callback / function whenever one is done. Your NIF wrapper function would provide the C function call with a C callback which would place a message in the appropriate process mailbox on completing the operation (something which you can do even without a process context with the NIF API).
Again, a short example on this would helpâŠ
Write a C function which conforms to the NIF external API, whose content consists of using the erl_nif functions to extract and convert the arguments passed in (reading them from the argc / argv youâre given), calling the âactualâ C function with those converted arguments, then taking the result and converting it to an Elixir/BEAM term that you can actually return to the calling Elixir process.
Writing NIFs is something which is a pretty advanced topic, and writing ones that behave and work well requires at least a moderate knowledge of how the BEAM / Elixir work internally (not to mention that on the Elixir side, you want the functions to âseemâ like pure Elixir functions and follow Elixir conventions, for which familiarity with Elixir is helpful). I would advise you to perhaps develop your skills there first. What is it that youâre actually trying to achieve? As a side note, I like writing my NIFs in Rust, using the excellent rustler crateâespecially when youâre a little unsure of what youâre doing, having the compiler complain at you is a much nicer experience than the BEAM crashing when it comes to getting your code to work.
Actually, re-reading your original question about this, I initially assumed you were talking about a callback in the âperform asynchronous operation, callback when itâs done with the resultâ, rather than âhereâs a function pointer which you can use as part of your operationâ. What youâre asking for is technically possible, but most likely inefficient for something like an internally-called callback for some sort of complex operation. See here for an overview, but basically, there is no way to execute a lambda from inside a NIF, as BEAM funs (lambdas / fns / whatever) can only execute inside a running BEAM process (which the NIF is not inside). To do this, you would have to have the callback function that you pass into the C library send a message to a process which is running asking it for the result of the callback, along with the fun youâre passed in when starting the operation, have it compute the result, and call another NIF function which would somehow communicate (need some locking primitives) with the waiting callback function and give it the result.
And Iâd really not recommend using a NIF unless you need the speed. If you do even the slightest thing wrong it can bring down the entire VM pretty randomly. A Port is the way to go (and much easier to write too) 90% of the time.
I think I understand what you mean now, I just define a new function with NIF, that calls the original function (with a different name), and use that one in my elixir!
I am trying to replace some functionality in an elixir project with C. Wouldnât Rust add only more complexity to my situation which I have to learn that too?
The callback function is part of the C library.â I am not sure if you answer still applies? A process in elixir? Which waiting callback? I am quite confused.
The C code is not a process (or multiple processes running) but a library, unfortunately this is not my decision, but I have checked and see that the ports are much easierâŠ
I was referring to Elixir processes (so like coroutines, separate âthreadsâ of execution), not OS processes.
Maybe, but writing a thin layer on top of a C library in Rust isnât too too hard, and it lets you worry less about using the erl_nif APIs correctly 
If the callback is one of several options which you statically have in the C library itself, I would make the Elixir-facing function simply take an atom indicating which of the functions should be used as the callback. Then you choose the correct function pointer to pass into the C call.
I agree!
The callback function is function pointer which is an argument to another function, as you have mentioned earlier.
my_C_func(int (*Callback)(int x, struct y *buffer))
I am not sure what you mean by âoptionsâ? Isnât the whole point of having the callback to be able to feed any function with specified signature?
With what you suggest, I need to have something like âswitch caseâ in the Elixir-facing function, no? Doesnât that contradict the whole point of having the callback? I am confused againâŠ
Thanks for helping me with this!
So there are two levels here. There is your Elixir code, and there is your NIF code, which is written in C, and can do anything normal C code can do, it just happens to have a macro which notifies the BEAM that there are a bunch of functions youâd like to expose as NIFs, here are their arities and the function pointers of the C functions which back them. You shouldnât think of the NIF as some âsimpleâ way of attaching C functions directly to Elixirâyou generally shouldnât do that, as youâll run into problems, and even if you manage to get that working, itâll feel âweirdâ on the Elixir side.
What the NIF layer should be instead is something which appears to your Elixir code to be just another Elixir module with functions which behave ânormallyâ, and which acts as a layer to convert this into something which whatever C library youâre using can consume.
It would be helpful if you detailed what problem youâre actually trying to solve here, but hereâs another way of explaining what I suggested. You want to somehow expose a C library which has a function, presumably some larger operation, which you can âcustomiseâ by passing in a function which performs some sub-step of the operation. Now, if you were writing a NIF to be published and used by anyone who wants to use this library in C, you would want users to be able to specify this callback function in Elixir directly, but as I wrote above, this is really awkward to get working and would likely be not very performant, since your NIF code cannot directly execute a BEAM functionâyou would need to message some BEAM process to actually perform the callback.
However, what is more likely is that you are writing this NIF for your own application, and so you will have a defined, finite set of these callbacks that you will actually pass in. When you drop the assumption that the callback function must be defined in Elixir, the problem becomes much easierâin your NIF library, you can define all the callbacks you need, and on the Elixir side, simply use an atom or something to select between them. The C library has a callback parameter because they must assume that the user could pass anything in, but if the NIF can be considered part of your application, rather than a generic library, you do not have to expose that assumption to the Elixir code.
Correct, a proper Port is a wrapper around such a library, it is accessed as a distinct process for safety reasons, but you still write an API there and it is quite a bit easier to do so as well.
To do a âcallbackâ on the BEAM, you either hold a 2/3-tuple of {Module, Func, Args} and call that via apply, an anon-function (which does not marshal very well so you generally avoid that in NIFâs), or you send a message, whichever is more appropriate is dependent on the API (messages have built-in rate limiting and such useful things for example).
Thank, I understand this now, thanks for dumbing down your answer!
Is this inline with the other answer (using Atom for accessing the callback)?
Send message from where/what to where/what?
Correct, the Module and Func are both atoms, the Args is a (usually empty) list of context arguments to append to yours, in erlang terms itâs generally done like apply(Module, Func, MyArgs ++ Args)
Either send the message back to the parent process via just returning it and let it dispatch it, or pass a PID in to the NIF to send the message to. 
Still though, unless absolute speed is needed, a Port is definitely better for this (though if you need absolute speed you have no choice but to use a NIF).
For note, even making it in Rust might be better, the Rustler NIF-making plugin is fantastic and Rust can call C APIâs with ease.
Iâm pretty sure that both a MFA tuple and an anon fun can be passed in and out of NIFs fine (I think you can even (modulo module/code loading) send anon funs across the wire), the bigger issue I was pointing out is that you canât (as far as I know) actually cause some Erlang / Elixir / BEAM code to be executed directly from a NIFâthe only interaction you have from NIF to the BEAM is sending messages.
If you had a callback which is called multiple times per operation (think of a comparison function in a sort algorithm, for instance), and the sort operation was in the NIF, but the callback was a BEAM function (either MFA or anon fun), what you would need to do is to set up some shared memory inside the callback you pass to the inner C function, send a message to a BEAM process that you set up earlier to be a callback handler, including the function (MFA or fun) and the arguments, and have that other process execute the BEAM callback and call another NIF function, which would then use the shared memory set up in the original callback (all the while that original thread is still waiting, so probably need to use a dirty scheduler) in order to pass it the result of the callback.
I think you can âapplyâ from a NIF, but I think it essentially exits the current nif callback too, unsure (I always minimize NIFâs and have the erlang side do the integration work).
And returning data.
Ooo yeah this would not work. If possible Iâd create a DSEL via tuples/lists/atoms for this if the C side really had to do the work very fast, otherwise perhaps to bake in the functionalityâŠ
Are you sure? I looked and couldnât find anything like this. Searching the docs for apply yields nothing.
Yes, what I meant to say was âthe only interaction possible where the NIF can initiate it asynchronously to being called as a functionâ.
Oh it definitely wasnât called that, it might have been an internal function I used now that I think of that⊠>.>
© Copyright Elixir Forum | Terms | Privacy & Cookies
