Image - an image processing library based upon Vix

Wonderful.
Mind to tag within the repository for the ex_doc source links and links on Hex.

Whoops, sorry about that. Done!

Hey Kip, this is awesome! In addition to streaming, to ‘streamline’ the API, will you also add functions to load images from Memory, and will the S3-thing be a behaviour so other backends (such as GCS) can be added as well?

PS. Have you considered applying for GitHub Sponsors? I would love to sponsor you there as I use many (or even all) of your libraries, and would love to help even if in a small way! If you do, let me know

This looks great! I’ve been using imagemagick via Mogrify so far and while I’d like to use vips, I would prefer having the option to shell out rather than using the NIF when dealing with user-uploaded images (for the reasons you point out in your readme). Is this something you would consider adding to the library?

@kip does it support overlays (drawing an image on top of another one), transparency, and writing a text on an image?

Something like shown in the animation in the home page of:

Screenshot from 2022-05-11 09-03-01516×380 19.8 KB

vix and libvips support everything you can think of, possibly even making you coffee in the morning. Text overlays are definitely on the list of features to expose. I’ve been working on what the right API should look like - feedback welcome. In the Vix readme there is an example of rendering a text overlay (see below).

# render text as image
# see https://libvips.github.io/libvips/API/current/libvips-create.html#vips-text for more details
{:ok, {text, _}} = Operation.text(~s(<b>Vix</b> is <span foreground="red">awesome!</span>), dpi: 300, rgba: true)
# add text to an image
{:ok, img_with_text} = Operation.composite2(img, text, :VIPS_BLEND_MODE_OVER, x: 50, y: 20)

@mayel, I understand the concern, and this is of course a benefit of using mogrify. Nevertheless, shelling out doesn’t resolve the issue of maliciously crafted images, and there is a long history of CVEs. So while you avoid a crash in mogrify bringing down the BEAM, there are other issues. Its a tradeoff. Image, which depends on Vix, is firmly wedded to libvips as a NIF. I am going to try some experiments in running Image in its own BEAM node but that’s as far as it will go. I have to note that so far the stability has been excellent - only crashing when I made a poorly written addition to the NIF (which @akash-akya promptly fixed).

@jeroenvisser101, the streaming API will do exactly what you want. @akash-akya has an experimental branch with the NIF implementation that basically means that image streaming can consume any Enumerable and as an output can produce an Enumerable.

This is fabulous for taking an S3 image, transforming it, and sending it as a chunked image in an HTTP request. There are many other use cases, but I think that one is high on the list of valuable capabilities.

Which is why it will be part of the Image 0.2 release in a week or so.

If you’re interested you can take a look at these tests in Vix.

A couple of addition thoughts on the overall imaging pipeline strategy. What appeals to me about a library based upon libvips is that it is fast, has a small memory footprint and has really good concurrency support. These alone are for me enough to accept the potential issues with a NIF-based solution.

When you also have:

• An architecture in libvips that is very friendly for a NIF implementation. This because only references are ever passed from Elixir-land to NIF-land so no large memory copies.
• An internal streaming architecture in libvips that means transformation evaluation is only done at the point at which an image is rendered (either by writing to a file or sending to a client for example). Its very efficient.
• Really strong support from @jcupitt, who is active, engaged and responsive
• The ability to do all the transformation in memory, with streamed data

Then I think libvips with Vix is a really strong platform - as long as the concerns relevant to any NIF-based solution are recognised. Its analogous a little bit to Nx, the benefits outweigh the risks for a number of use cases.

We are processing a ton of images with internal service that works on top of graphicsmagick but we faced many issues with it (like it can corrupt its internal miff format during text-based transforms). We have a workaround for those but libvips is also an option so I was asking if the library supports that, so maybe we can run an experiment replacing gm with vix. We don’t need external services.