solanav

Dirty CPU NIF blocking scheduler

Hello everyone!

I’m currently working on a web spider/metadata extractor and because writing parsers for many file types would be too much work, I’m using libextractor.

I wrote a NIF which I learned only allows for functions that return in less than ~1ms. So I just changed this:

static ErlNifFunc funcs[] = {
  { "extract", 2, extract }
};

into this:

static ErlNifFunc funcs[] = {
  { "extract", 2, extract, ERL_NIF_DIRTY_JOB_CPU_BOUND }
};

The problem: it seems like the scheduler gets to 100% utilization which is bad because the spider downloads a lot of files in parallel so it becomes impossible to do things like launch the observer if I’m calling the NIF. Any idea if this is normal? Is there a way I can prevent the NIF from blocking the rest of the system?

Here is a picture of the observer while I test the calls to my main function:

Thanks!

15 comments

#nif #dirty-scheduler

1 3374 15

2020-11-17 10:04:02 UTC

Marked As Solved

sasajuric

Author of Elixir In Action

While a nif is running on a scheduler its utilization will be plotted as 100%. This is normal.

If the NIF is running on a dirty scheduler it should not block regular schedulers. Here’s how I tested it locally:

In the C code of the dirty NIF function I added sleep(10);
I started the shell with ELIXIR_ERL_OPTIONS="+S 1" iex -S mix (to make sure there’s just one scheduler and one dirty CPU scheduler)
From the shell I started the nif as spawn(&my_nif/0)
As soon as process was spawned the iex shell was responsive and I was able to start the observer.

If you invoke a lot of longer-running dirty nifs concurrently, you may end up occupying all the dirty scheduler threads, which might block some other operations. For example, length will switch to dirty schedulers for larger list, and if all schedulers are busy for a couple of seconds, length will be blocked. I was able to reproduce this by spawning a long-running NIF, and then invoking length(Enum.to_list(1..1_000_000)) which blocked until NIF finished.

Based on the description of the scenario this looks like a case where port might be a better fit.

Post #11

Also Liked

dch

For NIF related code I usually refer to Paul Davis, Jesper Andersen, & Steve Vinoski, in no particular order. These are all in erlang, but that’s not really important for NIFs anyway.

If people have other examples in Elixir – I expect Frank Hunleth has a bunch of amazing Nerves ones. I’d hope they can chime in with some examples.

Paul’s authored jiffy, probably the most widely used NIF of all, Steve was instrumental in documenting and authoring the first working dirty scheduler code to my knowledge, and Jesper’s enacl NIF comes with excellent explanatory notes in addition.

The first two are both used extensively and show IMO canonical examples of building cross-platform compilation solutions. The middle is really well documented, and the latter shows how you can ship C source as part of the code and have it compiled locally.

Post #15