What is the recommended way to invoke a library at host server?

Questions / Help
1

I am writing an app that will call an installed library at host server, wait for its execution and broadcast result output to client via channel.

At happy scenario, the execution time will finish in milli seconds, with little CPU consumption, however, considering that the called library might get stuck or consume high CPU power for threshold cycles, what’s the recommended way to kill its OS process and return some message to the user?

I am running phoenix as a service using systemd at my development laptop, I just restart the service as sudo systemctl restart phoenix and I can see that the library process will be killed, however, I can’t do so at production, as lots of users are connected and using the same library.

Any advice is appreciated, Thanks.

2

Is it a library that you call a function from via NIF or is it an executable that you want to orchestrate via Port?

3

What is NIF? I use System.cmd, here is how I call it:

def mylib_pass netlist do
    {:ok, fd, file_path} = Temp.open "file.cir"
    IO.write fd, netlist
    File.close fd
    {output, _} = System.cmd("mylib", ["-b", file_path], stderr_to_stdout: true)
    File.rm file_path
    output
  end
4

So you “library” is not a “library” but a regular application you run.

If you were using Port instead of System.cmd/3 (which is basically just a wrapper around the former), you could use Port.info/1,2 to get the OS pid and read from /proc/${os_pid} the necessary information to track CPU usage/Memory usage or whatever metric you need to kill it.

5

Thanks, so, do you recommend using an elixir worker to monitor the application os_pid? and decide to kill it? or using an external solution like: inspeqtor ? I think Elixir based solution would be better, as I can directly send a feedback to client about the monitored process…

Your thoughts please?

6

Personally I’d probably use a CPU time constraint via ulimit, wrapped in a shell script, and just run that script…

7

Is there a specific tool you recommend? or do you have some handy example? it seems that cpulimit is sufficient.

8

As I said already, I’d use ulimit.

#!/usr/bin/env sh
ulimit -t 60
mylib -b "$1"

This will kill the started mylib after it has consumed 60 seconds of CPU time. cpulimit does only seem to limit current usage, but is unable to kill on exhaustive use…

Though I maybe misunderstand what you are actually trying to achive.

1 Like
9

Though remember that CPUtime is not necessarily equal to Wall time

10

I found that cpulimit can kill the process as in this example:

cpulimit -l 20 -k firefox

Launch the Firefox program and kill it if the process goes over 20% CPU usage.

11

http://erlang.org/doc/tutorial/nif.html

NIF is basically a way to interact with native code. A good example is an Erlang sqlite library that maps Erlang function calls to C function calls that do the real work.

So in your scenario it would probably be an overkill to make a NIF module that delegates work to native code (C/C++ code, or in a shared library like a .dll or .so file). Because it seems you simply have to interact with a full-blown external program.

I think your scenario can be best served by a Port since you can monitor a “connection” to executing a program and just kill it whenever you like – but this lacks the ability to monitor CPU usage.

cpulimit can help you but you’ll have to monitor the exit code of the external program.

1 Like