Image - an image processing library based upon Vix

I’ve published image version 0.64.0.

The primary intent of this release is to stablise the code in readiness for a 1.0 release. Deprecated code has been removed, a standardised color model introduced (via the new library color) and objective classification and detection has been removed to a new image_detection library. Video frame extractions is now using Xav rather than eVision.

As a result, there are quite a few breaking changes - and some new goodies. The breaking changes aren’t dramatic - but they are breaking.

Hopefully this is the last pre-release before 1.0. Feedback on this release will determine that outcome!

Breaking Changes

  • Five long-deprecated functions have been removed:

    • Image.interpretation/1 — use Image.colorspace/1.
    • Image.type/1 (was format/1) — use Image.band_format/1.
    • Image.convert_to_mask/1 — use Image.convert_alpha_to_mask/1.
    • Image.convert_to_mask!/1 — use Image.convert_alpha_to_mask!/1.
    • Image.map_pages/2 — use Image.map_join_pages/2.
  • Image.Color has been removed. Color handling now lives in two new modules and one new dependency:

  • Image.Classification and Image.Generation have moved to a new sibling package, :image_detection.

  • :bumblebee is no longer a dependency of :image. It is configured in the new library image_detection.

  • Image.Video is now backed by Xav (a thin Elixir wrapper around FFmpeg) instead of :evision / OpenCV. The public API surface is largely unchanged but the underlying type, options, and a few semantic details have moved:

    • The video struct is now %Image.Video{} (with fields :reader, :source, :fps, :duration_seconds, :frame_count, :width, :height) rather than %Evision.VideoCapture{}. Pattern-match on the new struct module if your code does so.

    • Image.Video.open/2’s :backend option has been removed. FFmpeg picks the demuxer automatically and there is no concept of pluggable backends in Xav.

    • Image.Video.known_backends/0, available_backends/0, known_backend?/1, known_backend_values/0, and available_backend?/1 have been removed for the same reason. Image.Options.Video (the module that owned the backend table) has been deleted.

    • Camera input is now opened via a platform-specific device path. :default_camera resolves to /dev/video0 on Linux, "0" (AVFoundation device 0) on macOS, and "video=0" on Windows. An integer camera index is mapped to the corresponding /dev/videoN (or platform equivalent). For non-default cameras you can also pass an explicit FFmpeg device string.

    • Frame-based seeking (Image.Video.seek/2 with frame: n, and Image.Video.image_from_video/2 with frame: n) is now implemented as a time-based seek to n / fps followed by zero or more next_frame calls. For most files this lands on the requested frame; for very inter-frame-compressed files FFmpeg may snap to the nearest preceding keyframe.

    • Image.Video.close/1 is now a no-op that returns {:ok, %Image.Video{reader: nil}}. Xav garbage-collects the underlying FFmpeg context, so explicit close is no longer necessary. The function is retained for source compatibility; subsequent operations on the closed struct return {:error, %Image.Error{reason: :video_closed}}.

    • Image and audio frames are decoded by FFmpeg + libswscale rather than by OpenCV’s videoio backend. Pixel-exact comparisons against fixtures generated by the previous version will not match; the test fixture test/support/validate/video/video_sample_frame_0.png has been regenerated.

  • :xav is now an optional dependency. It requires FFmpeg ≥ 6.0 to be installed on the system. Add it to your mix.exs if you use Image.Video:

    {:xav, "~> 0.10", optional: true}
    
  • :evision is no longer needed for Image.Video. It is still required for Image.QRcode and for the Image.to_evision/2 / Image.from_evision/1 interop helpers, which are unchanged. The README’s optional-dependency table reflects the new split.

  • Image.Error is now a structured public exception. It carries :reason (atom or {atom, value} tuple), :operation, :path, :value, and a derived :message. Every fallible function in the library now returns {:ok, value} or {:error, %Image.Error{}} — bare-string error tuples have been eliminated. Bang variants raise the same struct. The new Image.Error.wrap/2 helper attaches structured context to a raw libvips or File.* error. Pattern-match on :reason instead of scraping :message:

    case Image.open(path) do
      {:ok, image} -> ...
      {:error, %Image.Error{reason: :enoent}} -> not_found_handler()
      {:error, %Image.Error{} = err} -> raise err
    end
    

Enhancements

  • Image.dominant_color/2 now accepts a :method option of either :histogram (the existing default) or :imagequant. The :imagequant method routes through libimagequant (via vips_gifsave_buffer) and returns a palette of {r, g, b} tuples ordered by perceptual importance. New :effort and :dither options tune the quantiser. See guides/performance.md for a comparison of the two methods.

  • Colour arguments to drawing, embedding, trimming, gradient, chroma key, comparison, warp-perspective, meme, replace-colour, flatten, and if_then_else operations are now correctly converted to the target image’s colour space. Drawing :red on a Lab image now produces actual Lab red, not the bytes [255, 0, 0] reinterpreted as Lab.

  • New Image.Pixel and Image.ICCProfile modules. Image.Pixel.to_pixel/3 is the canonical way to turn any user-friendly colour input into a libvips-ready pixel for a particular image; Image.Pixel.to_srgb/1 is the image-independent equivalent for callers (SVG renderers, gradients) that need a fixed sRGB output.

  • The :color library is now a dependency of :image. It is now the canonical colour science layer for the project.

  • Image.Video now supports HTTP/HTTPS/RTSP/RTMP URLs as video sources for free, since FFmpeg supports them natively.

  • Image.xav_configured?/0 is the new compile-time predicate that gates the Image.Video module (analogous to Image.evision_configured?/0 and Image.bumblebee_configured?/0).

  • The Image.Video.frame_to_image/1 helper exposes the raw Xav.FrameVix.Vips.Image.t() conversion (used internally by image_from_video/2 and stream!/2). Useful if you have a frame from elsewhere in the Xav ecosystem and want to bring it into Image.

9 Likes

@kip this is an amazing release! Love the transition to xav! Thank you for all your hard work!

Xav is awesome, thanks to all who work on it. I have lots of other ideas I can use it for!