fireproofsocks

fireproofsocks

Does Image Rendering "shell out"?

I’m hoping an Erlang/Elixir Jedi can help show me the ways of the force here. I got a bunch of images that need to be manipulated programmatically (stop me if you’ve heard this before). So I’m looking over the approaches written about here:
https://sergiotapia.me/generate-images-with-name-initials-using-elixir-and-imagemagick-374eca4d14ff
and here Elixir Native Interoperability – Ports vs. NIFs

Basically, the question comes down to this: is the image manipulation process “booting up” each time an image is sent to it for rendering? Or is there a way to boot up the image manipulating code (whatever it is) ONCE when the Elixir app starts, and then run each image through it?

In other languages, any time we “shell out” to an ImageMagick or similar command, the Image processing utility starts up each time the call is made. I’m imagining this like starting up Photoshop on my desktop: you launch the app, you see the banner image, the spinning wheel, the app loads, then the image editing occurs, and then it shuts down. Wash, rinse, repeat. However, we are crunching through millions of images, so that boot-up time ends up being prohibitively expensive (both in terms of rendering time and CPU).

Can someone help explain the differences between the different approaches available to Elixir? System.cmd() seems to represent “shelling out” – i.e. whatever it is calling, that executable needs to load up each time the command is called. Whereas NIFs and ports seem to be able to boot up once and stay running alongside the parent Elixir application. Is that accurate? And there might be some Elixir module we can use? Given the need for this process to be as performant as possible (and not reload executables redundantly or blow up our EC2 bill), what is the best way to accomplish a task like this?

(FYI: the image transformations are relatively simple: mostly flattening layers or doing simple resizing). Others have expressed concern about using Erlang for computationally-heavy operations, but I would like to at least give it a good honest benchmarking. If not Elixir/Erlang, C or Go or Rust (or ???) might be preferable solutions for handling high concurrency and being able to natively perform the image manipulations without “shelling out”.

Thoughts or suggestions welcome. Thanks!

Most Liked

NobbZ

NobbZ

I only glanced over the blog post, but yes, it is “shelling out”.

But to my experience so far, IMs boot up time is neglible.

If you really want to, you can of course try to interface the C++ library in a NIF or C-Port, but I am not sure if this will make a huge difference…

dimitarvp

dimitarvp

What you can do is always have 1 or more executables started via System.cmd and waiting on their stdio for data. That’s a trick I used in other programming languages when I was concerned about the initial overhead of loading a native executable in a child shell; because now the executable is loaded and is only waiting for data.

One example coming to mind was re-coding videos through ffmpeg; I’d start it with all necessary flags but without any input and it would just wait on stdio until I send it a video to process. Once sent, it will process it immediately.

Can’t give you exact details but that’s usually how a pool of disposable native processes is managed.

Last Post!

fireproofsocks

fireproofsocks

That is a good idea, thank you!

Where Next?

Popular in Questions Top

hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement