Extracting selected images from downloaded video files?

I would use ffmpeg to extract images from the video files. Could probably skip VLC and use ffmpeg directly as well.

Then loading the extracted images from disk and do whatever I have to do in nx

Probably not gonna be the most efficient way of doing it, but might be good enough

No idea on how membrane works.

I am also using ffmpeg to extract images from video, there is a ffmpex wrapper, but I wrote a simple one.

Membrane is good if You want to receive or stream rtmp, or webrtc…

But for YT, I would check this python package, and maybe use erlport to interface python script.

https://github.com/pytube/pytube

There are tools to transform images from/to nx…

by @kip

and

by @cocoa

It seems like FFmpeg only extracts images in PNG format. This might work well with ExPNG, but I think I’ll want to get them into Nx format at some point. Suggestions?

It can use more format… like jpg. But if You want them in nx, the 2 packages have a to_nx() function.

The wonderful image library can load images and convert them to/from tensors

Edit: didn’t notice you already got this suggestion;)

As of image version 0.21.0 which I just published you can leverage the Evision.VideoCapture module to extract images from a video.

You can therefore use either the more idiomatic API in Image.Video or simply leverage the code that is there to use Evision directly.

Depending on the codec support of the OpenCV installation on your system I suspect you may not need to use the FFMEG CLI approach any more.

Example using Image

# Extract using a frame offset
iex> {:ok, video} = Image.Video.open("./test/support/video/video_sample.mp4")
iex> {:ok, _image} = Image.Video.image_from_video(video, frame: 0)

# Extract using a millisecond offset
iex> {:ok, _image} = Image.Video.image_from_video(video, millisecond: 0)

# Capturing images from a camera (note that seeking is not supported
# on image streams. On MacOS you will receive a prompt to permit the
# app to access the camera. You may need to restart the BEAM and/or
# your system if capturing fails immediately after granting permission.
iex> {:ok, video} = Image.Video.open(:default_camera)
iex> {:ok, _image} = Image.Video.image_from_video(video)

Example using Evision directly

# Extract by frame offset
video = Evision.VideoCapture.videoCapture(video_filename)
true = Evision.VideoCapture.set(video, Evision.cv_CAP_PROP_POS_FRAMES(), frame_offset),
cv_image = Evision.VideoCapture.read(video)

Using images in Nx

Both Image and Evision support zero-copy “export” of images to Nx. Just call Image.to_nx/1 or Evision.Mat.to_nx/1 as appropriate.