mattei

mattei

I love this. Excellent library!

0
Post #21
kip

kip OP

ex_cldr Core Team

I’ve just published image version 0.2.0. It has the following enhancements:

Enhancements

  • Adds Image.Text to support generating an image from a string. See Image.Text.text/2.

  • Adds Image.Shape to support generating images from shapes. In this release there is a generalised support for polygons and stars. See Image.Shape.polygon/2 and Image.Shape.star/2.

  • Improves Image.compose/3 to support composing with image lists, relative image placement and more.

It was a fun and very intense few days to get this update ready for release, even though it wasn’t what I had planned. Now … back to image streaming.

As always, comments, suggestions, bug reports very welcome.

Thanks again to @akash-akya for the fabulous Vix without which none of this would be possible.

10
Post #22
wmnnd

wmnnd

This looks really neat. I’ll be sure to give it a try when I add image processing to Keila :+1:

0
Post #23
Exadra37

Exadra37

What an awesome work you did here, I just love it :slight_smile:

0
Post #24
stefanchrobot

stefanchrobot

Just perfect! I’ll be needing a library like this soon. Couple of well-intentioned comments:

  • Ready-to-use Livebook for interactive experimenting would be awesome,
  • Not really discoverable on Hex :face_with_diagonal_mouth:,
  • The logo in the docs could use some love, especially since this is an image processing library :wink:.
1
Post #25
kip

kip OP

ex_cldr Core Team

I completely agree with all of this. Contributions would be most welcome :slight_smile:

Next steps:

  1. Vix and eVision integration so Image can do object detection. This also means integration with Nx will be supported.
  2. Finish up image streaming. Be able to stream an image from S3, process it in parallel while also streaming it to a front end. Game changer.
  3. Docs, Tutorials, Livebooks, … really important topic.

And as for the logo … totally agree, it sucks. Very open to suggestions and contributions for something that reflects the intent and passion to make this a great lib.

BTW, the description of the package is:

An approachable image processing library based upon Vix and libvips that is NIF-based, fast, multi-threaded, pipelined and has a low memory footprint.

Which I thought hit all the buzzwords for discoverability. I guess not!

6
Post #26
kip

kip OP

ex_cldr Core Team

Thanks to some fabulous work by @akash-akya in the new Vix version 0.11.0, Image version 0.4.0 is now released with a focus on image streaming.

Changelog

  • Adds support for opening streaming images. This allows images to be streamed from File.stream!/3 or from any t:Enumerable.t/0 including those created from ExAws.S3 by ExAws.stream!/2.

  • Adds support writing streaming images. This allows images to be streamed as an enumerable which can then be consumed by Plug.Conn.chunk/2, by ExAws.S3.upload/3, File.stream/3 or any other function that processes stream resources. See the test/stream_image_test.exs file for examples.

  • Adds a :memory option to Image.write/3. Instead of a path name or stream, use :memory if you’d like to return a binary form of an image in its formatted type. Note that this will run the image transformation pipeline resulting in the entire final image being loaded into memory. Therefore this option should be used sparingly since most use cases do not require this option. It is primarily added to facilitate passing images to other libraries in the Elixir ecosystem.

Use case

One of the benefits of libvips, and therefore Vix, is that transformations are built in pipelines. Its actually quite a lot like Elixir pipelines in nature (albeit the implementation is wildly different).

This means we can support streaming an image into Image.open/2, apply a pipeline of transformations and stream the image out with Image.write/3 or Image.stream/2. These operations all happen concurrently: streaming the image into the pipeline, the pipeline itself, and the streaming out again. Very memory efficient, very Elixir friendly, very easy to apply in an application.

Example

Imagine an example where we want to stream an image from S3, apply some transformations, and stream the image to an HTTP client. Here’s the code:

ExAws.S3.download_file("images", "Hong-Kong-2015-07-1998.jpg", :memory)
|> ExAws.stream!()
|> Image.open!()
|> Image.resize!(200)
|> Image.write(conn, suffix: ".jpg")

Here conn is the conn of a Plug/Phoenix request. All the machinery of streaming is abstracted away. Some additional headers would need to be set for the MIME type and filename but apart from that there is nothing to do.

Another approach makes the stream more explicit but ultimately is the same example as that above:

ExAws.S3.download_file("images", "Hong-Kong-2015-07-1998.jpg", :memory)
|> ExAws.stream!()
|> Image.open!()
|> Image.resize!(200)
|> Image.stream!(suffix: ".jpg")
|> Enum.reduce_while(conn, fn (chunk, conn) ->
  case Plug.Conn.chunk(conn, chunk) do
    {:ok, conn} ->
      {:cont, conn}
    {:error, :closed} ->
      {:halt, conn}
  end
end)

Next steps

The next release will focus on interoperability with eVision and hopefully also Nx. This will enable interaction with the very cool axon and axon_onnx.

Feedback and suggestions

After the next release, primary development will be user-driven. Open an issue, start a discussion or comment here with what you’d like to see.

15
Post #27
kip

kip OP

ex_cldr Core Team

Turns out that Nx integration was really straight forward due primarily to the excellent work of @akash-akya and the Nx team. Introducing Image version 0.5.0 with the following changelog entry:

Enhancements

  • Adds Image.to_nx/1 to convert an image to an Nx tensor.

  • Adds Image.from_nx/1 to convert an Nx tensor into an image.

The conversation take care of ensuring type compatibility and ensuring the correct axes in the right order for Image.

When calling Image.to_nx/1, there is no data copying, just passing a reference to a heap binary so the movement is fast and garbage collection takes care of cleaning up.

15
Post #28
kip

kip OP

ex_cldr Core Team

Image version 0.6.0 has just been published. The changelog reads:

Bug fixes

  • Don’t attempt to compile code that refers to Plug.t if Plug is not configured. Thanks to @kwando. Closes #7.

Enhancements

  • Adds Image.dshash/1 to return a 512-bit image hash.

  • Adds Image.hamming_distance/2 to compare the similarity of two images or two image hashes.

  • Adds Image.radial_gradient/3 to great a radial gradient image. This could be used to composite over another image to add a vignette.

  • Adds Image.Math logical functions boolean_and/2, boolean_or/2, boolean_xor/2, boolean_lshift/2 and boolean_rshift/2. Also includes the ! versions.

  • Add operator use Image.Math overloads for image &&& (and), ||| (or), <<< (lshift) and >>> (rshift).

11
Post #29
kip

kip OP

ex_cldr Core Team

Image version 0.9.0 has just been published.

The major feature is the beginning of integration with the fabulous eVision (OpenCV) by @cocoa. The integration doesn’t copy any data to/from the libvips NIF and the openCV NIF so performance and resource utilisation should be quite good.

In this initial experimental release, eVision is an optional dependency. Only one function is exposed which decodes QRcodes. See Image.QRcode.decode/1.

The full changelog is:

Enhancements

  • Image.open/2 supports opening .gif and .tif images from raw binary data in addition to the previous .jpeg, .png and .webp.

  • Add Image.shape/1

  • Add Image.interpretation/1

  • Add Image.type/1

  • Add initial support for eVision. In this release the function Image.QRcode.decode/1 is provided to decode images that contain a QRcode. See the eVision repository for installation instructions since this library is not yet on hex.pm.

  • Removed most dialyzer warnings. The remaining warnings require an update to Vix (a PR has been submitted).

12
Post #30