Xav is awesome, thanks to all who work on it. I have lots of other ideas I can use it for!
kip
ex_cldr Core Team
sodapopcan
Hey Kip, congrats on the release!
It seem image_detection does not exist. I though the link was just broken but I can find anything similarly named in hex.pm.
kip
ex_cldr Core Team
It seem
image_detectiondoes not exist
You are right, it’s still a work in progress on my side. If its something you desperately need I can try and accelerate, otherwise its a couple of weeks off.
sodapopcan
Oh no, not desperately by any means. Just dipping back into a project that requires image processing but still in the experimentation phase. Thanks!!
kip
ex_cldr Core Team
Today’s fun library: image_vision is now available on hex.
ImageVision is a simple, opinionated image vision library for Elixir. It sits alongside the image library and answers three questions about any image - illustrated best with some extracts from the included livebook:
What’s in the image?
Where are the objects in the image?
Where are the objects, and then apply an image overlay to visualise them.
What are the positions and dimensions of the objects?
Which pixels belong to which object in the image?
What are the objects at a pixel mask level. Then visualise them using color masks.
kip
ex_cldr Core Team
Three new additions to the elixir-image family hit Hex this week.
image_ocr
image_ocr is an idiomatic Elixir interface to Tesseract 5 via a NIF over the C++ API. Plain text or per-word results with confidence and bounding boxes, auto-normalisation of pixel format before recognition, flexible locale handling, +-joined combinations), and a NimblePool-backed pool for parallel recognition across schedulers.
Tesseract is quite a large install so it’s not available pre-compiled but there are install instructions for most common platforms.
image_qrcode
A QR encoding (via Nayuki’s QR-Code-generator) and decoding (via quirc). Both run on dirty CPU schedulers and ship as precompiled NIFs for Linux (gnu/musl × x86_64/aarch64/armv7), macOS (x86_64/arm64) and Windows (x86_64) — typically no compiler toolchain required at install time.
image_lens_correction
.Corrections for the four most common camera-lens defects: radial (barrel/pincushion) distortion, vignetting, lateral chromatic aberration, and geometric projection.
Driven by the fabulous Lensfun community calibration database (1,013 camera bodies, 1,459 lens calibration sets) bundled as a compact ~5 MB Erlang term file.
The EXIF-driven pipeline (Image.LensFun.Correct.correct/2) reads the camera/lens identification, focal length, aperture and focus distance from image metadata, looks up and interpolates the calibration, and applies all enabled corrections in one pass.
This one has been lingering half-finished for a couple of years so its feels good to get it ready for release.
kip
ex_cldr Core Team
image_vision was also updated this week with some cool new models; all callable in a couple of lines without ML expertise.
Image.FaceDetection
Fast face detection with bounding boxes, confidence scores, and the five canonical landmarks (eyes, nose tip, mouth corners). The default model is YuNet 2023-March. It’s quite small at ~340 KB and runs on the CPU.
Includes crop_largest/2 as the entry point for face-aware crop bias (think gravity: :face).
Image.Background
Provides foreground/background separation.
-
remove/2returns the input with the background made transparent; -
mask/2returns the foreground mask alone for custom compositing.
The default model is BiRefNet lite (MIT, ~210 MB) via Ortex. There’s a mix task for downloading it.
There is a livebook for exploring background removal.
Image.Captioning
Provides a natural-language description of an image. caption/2 returns strings like "a man riding a horse with a bird of prey".
The default model is BLIP base (BSD-3-Clause, ~990 MB) via Bumblebee. Heavy enough that it isn’t auto-started by default — flip autostart: true or add it to your supervisor.
There is a livebook for exploring image captioning.
Image.ZeroShot
Classify an image against arbitrary labels you supply at call time with no retraining.
classify/3returns ranked{label, score}pairs;label/3returns just the best;similarity/3computes CLIP-space cosine similarity between two images.
The default model is OpenAI CLIP ViT-B/32 (MIT, ~600 MB). The default prompt template "a photo of {label}" boosts accuracy on bare-noun labels; override or disable as needed.
There is a livebook for exploring zero-shot classification.
xsteve
I tried to access the github repository for image_ocr - it does not seem to be available/public:
kip
ex_cldr Core Team
Whoops, incorrect source link. Its GitHub - elixir-image/image_ocr: Elixir NIF-based interface to Tesseract OCR · GitHub
I’ve published an update to hex with the correct source link as well. Thanks for letting me know.
kip
ex_cldr Core Team
Today brings the introduction of something I’ve been working on since I started image four years ago.
- An Plug-based image CDN, published in image_plug
- A set of responsive image components, published in image_components
- An image playground for testing and experimentation.
Why build a CDN plug when there are so many image CDNs already?
- They can be very expensive, when the requirements are often not that complex
- Its easy to get tied to one CDN URL structure or API and its difficult to change
- Its complex if you want to iteratively test in a local environment
- Because you might want to build and offer an image CDN as a corporate resource or as a SaaS offering.
- Because libvips via the fabulous vix
Features
- High performance, optimised imaging pipeline.
libvipsviavixmeans we have access to thelibvipsstreaming pipeline that can stream images from disk, or S3 and transform them in a space and time efficient manner. - A pluggable API server build over a common AST-like imaging pipeline allowing you to build your own API structure over the standard imaging pipeline.
- Out-of-the-box support of the URL API structure of several of the big commercial CDN players so that you can use
image_plugin development and a commercial CDN in production without code change. Ships with support for:- Cloudflare Images
- Cloudinary
- Imgix
- ImageKit
- IIIF servers
- Simple client library,
image_componentsthat is used for all CDNs modelled on the excellent unpic javascript library. - Support generation of optimised image sets with the
.imagecomponent as well as art direction if you use the.picturecomponent.
Live Playground
You can experiment with what these libraries provide in a simple playground at https://image-playground.fly.dev. Be gentle, it bundles the image, image_vision (with several large ML models), Nx and EXLA libraries all on a fly.io’s small VPS server.
If you want to explore more, run the playground on your own machines. It’s mountable inside a Phoenix app or you can run it standalone. The README has all the relevant info.
Example
An example image in the playground, face-aware cropping, some vignetting added.
21