Hello there,
I would like to generate a storyboard image using elixir. Firstly, I found a ffmpeg incantation which does what I want.
ffmpeg -y -i ~/Videos/moose-encounter_75.mp4 -frames:v 1 -vf 'select=not(mod(n\,257)),scale=160:-1,tile=5x5' -update 1 -fps_mode passthrough ~/Videos/thumb.jpg
The output looks like the following
I found there’s a ffmpeg library for Elixir called FFmpex. There aren’t many ffmpex examples in the wild, but I was able to hack something together which got me 90% there. Following is the code I have so far.
def create_thumbnail(input_file, output_file) do
case get_video_framecount(input_file) do
{:error, reason} -> {:error, reason}
{:ok, framecount} ->
frame_interval = div(framecount, 25)
scale_width = 160
tile_grid = "5x5"
command =
FFmpex.new_command
|> add_global_option(option_y())
|> add_input_file(input_file)
|> add_output_file(output_file)
|> add_file_option(option_vframes(1))
|> add_file_option(option_filter_complex("select=not(mod(n\\,#{frame_interval})),scale=#{scale_width}:-1,tile=#{tile_grid}"))
|> add_file_option(option_vsync(1)) # -vsync is deprecated in ffmpeg but ffmpex doesn't have it's modern replacement func
# |> add_file_option(option_update(1)) # ffmpeg complains but it doesn't necessarily need this. I'm omitting because ffmpex doesn't know this function
# |> add_file_option(option_fps_mode("passthrough")) # -fps_mode is the modern replacement for -vsync, but ffmpex doesn't have that func
execute(command)
end
end
Little side note, get_video_framecount/1
is a function which uses ffmpex to call ffprobe and ultimately get the number of frames from the video stream.
def get_video_framecount(file_path) do
case FFprobe.streams(file_path) do
{:ok, streams} ->
streams
|> Enum.find(fn stream -> stream["codec_type"] == "video" end)
|> case do
nil -> {:error, "No video stream found"}
video_stream ->
nb_frames =
video_stream
|> Map.get("nb_frames", %{})
case nb_frames do
nil -> {:error, "nb_frames not found"}
nb_frames ->
case Integer.parse(nb_frames) do
{number, _} -> {:ok, number}
end
end
end
{:error, reason} -> {:error, reason}
end
end
The problem with create_thumbnail/2
is on the ffmpex option_filter_complex/1
line. I don’t think I have the syntax correct, because I see an error when I run the code.
** (ArgumentError) argument error
(ffmpex 0.11.0) lib/ffmpex.ex:193: FFmpex.validate_contexts!/2
(ffmpex 0.11.0) lib/ffmpex.ex:117: FFmpex.add_file_option/2
(bright 0.1.0) lib/bright/images.ex:80: Bright.Images.create_thumbnail/2
iex:1: (file)
I’m still learning Elixir, so I wasn’t able to make sense of FFmpex’s source code. It looks like the codebase is dynamically generating the options functions (very cool!), but there isn’t much documented usage for the actual option_filter_complex/1
function. The docs linked me to ffmpex/lib/ffmpex/options/advanced.ex at a190c03e706a6c770d7f7fbd3e681803933d0927 · talklittle/ffmpex · GitHub which got me started, but I’ll have to do some more digging in to understand.
My biggest question right now is what type of argument option_filter_complex/1
accepts. Is it supposed to be a string? A map? Not sure yet.
I’m just putting this here for now, documenting what I learn.