Hello there,

I would like to generate a storyboard image using elixir. Firstly, I found a ffmpeg incantation which does what I want.

ffmpeg -y -i ~/Videos/moose-encounter_75.mp4 -frames:v 1 -vf 'select=not(mod(n\,257)),scale=160:-1,tile=5x5' -update 1 -fps_mode passthrough ~/Videos/thumb.jpg

The output looks like the following

thumb800×450 64.3 KB

I found there’s a ffmpeg library for Elixir called FFmpex. There aren’t many ffmpex examples in the wild, but I was able to hack something together which got me 90% there. Following is the code I have so far.

  def create_thumbnail(input_file, output_file) do

    case get_video_framecount(input_file) do
      {:error, reason} -> {:error, reason}
      {:ok, framecount} ->

        frame_interval = div(framecount, 25)
        scale_width = 160
        tile_grid = "5x5"

        command =
          FFmpex.new_command
          |> add_global_option(option_y())
          |> add_input_file(input_file)
          |> add_output_file(output_file)
            |> add_file_option(option_vframes(1))
            |> add_file_option(option_filter_complex("select=not(mod(n\\,#{frame_interval})),scale=#{scale_width}:-1,tile=#{tile_grid}"))
            |> add_file_option(option_vsync(1))                  # -vsync is deprecated in ffmpeg but ffmpex doesn't have it's modern replacement func
            # |> add_file_option(option_update(1))               # ffmpeg complains but it doesn't necessarily need this. I'm omitting because ffmpex doesn't know this function
            # |> add_file_option(option_fps_mode("passthrough")) # -fps_mode is the modern replacement for -vsync, but ffmpex doesn't have that func

        execute(command)
    end
  end

Little side note, get_video_framecount/1 is a function which uses ffmpex to call ffprobe and ultimately get the number of frames from the video stream.

  def get_video_framecount(file_path) do
    case FFprobe.streams(file_path) do
      {:ok, streams} ->
        streams
        |> Enum.find(fn stream -> stream["codec_type"] == "video" end)
        |> case do
          nil -> {:error, "No video stream found"}
          video_stream ->
            nb_frames =
              video_stream
              |> Map.get("nb_frames", %{})

            case nb_frames do
              nil -> {:error, "nb_frames not found"}
              nb_frames ->
                case Integer.parse(nb_frames) do
                  {number, _} -> {:ok, number}
                end
            end
        end

      {:error, reason} -> {:error, reason}
    end
  end

The problem with create_thumbnail/2 is on the ffmpex option_filter_complex/1 line. I don’t think I have the syntax correct, because I see an error when I run the code.

** (ArgumentError) argument error
    (ffmpex 0.11.0) lib/ffmpex.ex:193: FFmpex.validate_contexts!/2
    (ffmpex 0.11.0) lib/ffmpex.ex:117: FFmpex.add_file_option/2
    (bright 0.1.0) lib/bright/images.ex:80: Bright.Images.create_thumbnail/2
    iex:1: (file)

I’m still learning Elixir, so I wasn’t able to make sense of FFmpex’s source code. It looks like the codebase is dynamically generating the options functions (very cool!), but there isn’t much documented usage for the actual option_filter_complex/1 function. The docs linked me to ffmpex/lib/ffmpex/options/advanced.ex at a190c03e706a6c770d7f7fbd3e681803933d0927 · talklittle/ffmpex · GitHub which got me started, but I’ll have to do some more digging in to understand.

My biggest question right now is what type of argument option_filter_complex/1 accepts. Is it supposed to be a string? A map? Not sure yet.

I’m just putting this here for now, documenting what I learn.