Can't make erlexec work with stdin as input

Questions / Help
1

I want to transform programmatically an image with FFmpeg. I know you have nice libraries Image, Vix, Stb_image but I want to use the capacity of FFmpeg to take buffer as input. I don’t see how System.cmd nor Port can accept inputs from stdin so I sought for something.

I found Porcelain and erlexec. I saw the last commit was 4 years ago for Porcelaine. Not saying it’s bad but erlexec has its last commit 2 months ago.

I can do it with Porcelaine but not with erlexec and I don’t see why. Maybe someone has some experience with it.

file = File.read!("img.png")

porcelaine = Porcelain.spawn("ffmpeg", ["-i", "-", "img.jpg"], in: :receive, out: :stream)
 {:input, data} = Porcelain.Process.send_input(porcelaine, file)
File.write!("img.jpg", file)
#=> success

but not erlexec: “bad arguments”.

{:ok, pid, os_pid} = :exec.run("ffmpeg -i - img.jpg", [:stdin, stdout: "img2.jpg"])
 :ok = :exec.send(os_pid, file)

[error] GenServer :exec terminating
** (stop) :einval
Last message: {:EXIT, #Port<0.6>, :einval}

Is it related to the way you pass strings to the function?

2

The stdout option to exec expects a value matching the output_dev_opt spec:

You’re passing a binary there, which isn’t on the list. Try a charlist (single quotes, or ~c instead.

3

I removed the stdout option, same result.

In their example, they use: exec:send(I, <<"Test data\n">>). . Not sure what this means because I can’t stringify a file.

4

Perhaps you are not using the options correctly?

Terminal1:

iex(2)> :exec.start([])
{:ok, #PID<0.110.0>}
iex(3)> {:ok, _, i} = :exec.run_link("while true; do read x; echo \"Got: $x\"; done", [:stdin, {:stdout, "/tmp/output"}])
{:ok, #PID<0.111.0>, 2271}
iex(4)> :exec.send(i, "Test1\n")
:ok
iex(5)> :exec.send(i, "Test2\n")
:ok

Terminal2:

$ tail -f /tmp/output
Got: Test1
Got: Test2
5

Yes, you are right. But in fact, I don’t need the :stdout option as FFmpeg knows what to do - namely save into “img.jpg” - as it is included in the command argument.
FFmpeg only needs to know what his input is, stdin.

A test.
In a shell, I pipe from stdin and it works.
In IEX, I send an empty file in the buffer, it works, but when I pass an image, it fails.

In the shell:

> file img.png
# img.png: PNG image data, 1596 x 732, 8-bit/color RGBA, non-interlaced

> cat img.png | ffmpeg -i - out.jpg -y

> file out.jpg
# out.jpg: JPEG image data, JFIF standard 1.02, aspect ratio, density 1x1, segment length 16, comment: "Lavc61.3.100", baseline, precision 8, 1596x732, components 3

#=> success

In IEX:

> {:ok, _, i} = :exec.run("ffmpeg -i - img.jpg", [:stdin])

# a dummy example: I pass an empty file from stdin
> f = File.read!("test.png")
""
> System.cmd("file",  ["test.png"])
{"test.png: empty\n", 0}

> :exec.send(i, f)
:ok
# it did nothing (not file created), but it did not fail!

# the "real" file from stdin
>  System.cmd("file",  ["img.png"])
{"img.png: PNG image data, 1596 x 732, 8-bit/color RGBA, non-interlaced\n", 0}

> f = File.read!("img.png")
<<137, 80, 78, 71,...>>

:exec.send(i, f)
** (stop) :einval

The difference I see is that I am passing data as a string in the first case, and then a binary.

However, it seems to be able to use binaries, as shown below, so I don’t know.

In IEX:

> t = <<1,2>>
> File.write("test2.txt", t, [:binary])
> f = File.read!("test2.txt")
<<1,2>>
> {:ok, _, i} = :exec.run("cat", [:stdin, {:stdout, "bin.txt"}])
> :exec.send(i, f)
:ok
> File.read!("bin.txt")
<<1,2>>
6

I see. The issue is that ffmpeg needs to know that the input sent to it is complete. Otherwise, it’ll sit waiting for more data.

When you use pipes in the shell, the head of the pipe (cat img.png) closes the pipe upon sending the entire content of the file, which helps ffmpeg detect the eof.

In case of erlexec you need to tell the child process that there’s an eof condition like this:

> f = File.read!("test.png")
> {:ok, _, i} = :exec.run_link("ffmpeg -y -i - img.jpg", [:stdin, :stderr])
> :exec.send(i, f)
> :exec.send(i, :eof)
> flush       # This will print out the output of ffmpeg written to stderr and sent to the caller's terminal

This will fix your issue.

It is illustrated here.

7

Ahh, I tried to append \n (or ‘10’ in binary ?) to the file as in the example, but I failed.

With almost zero knowledge of Erlang, and discovering that that atoms are Elixir specific, I wasn’t sure at all what it was.
Thanks for your time.

8

@ndrean I created Exile & ExCmd specifically to support moving large amount of data between external process. If you are interested you can check them out.

I created a Livebook sometime back using ffmpeg : Community Livebooks Thread - Share Yours! - #4 by akash-akya

1 Like
9

Looks nice! Many thanks!
I did not found ex_cmd, more Elixir style.
I will look more closely when I find why System.find_executable("ffmepg") returns nil ! (Livebook only).

I solved my problem with Porcelain by using a GenServer. The use case is receiving chunks as strings via a WebSocket. In the GenServer, I spawn Porcelain to start FFmpeg to read from stdin, and then just forward messages/chunks continuously to the GenServer and Porcelain feeds pipe:0.

I see you request files via a GET request, but in my case I want to receive octet-stream via HTTP and am stuck so I will definitively look more closely.