There’s mix compile --no-optional-deps --warnings-as-errors to make sure that the application compiles successfully without optional dependencies present. Sounds like that would be useful to have in CI.
LostKobrakai
0
kip
ex_cldr Core Team
kip
ex_cldr Core Team
Yep, I certainly need to take some time to configure CI properly. That’s a chore I find hard to prioritise but you’re right.
1
kip
ex_cldr Core Team
I just published Image version 0.23.0 with the following improvements and bug fixes. In particular Image.normalize/1 and Image.autolevel/1 are steps towards automated image improvement.
Bug Fixes
-
Fix specs for
Image.Options.Write. Thanks to @jarrodmoldrich. Closes #36. -
Fix spec for
Image.exif/1. Thanks to @ntodd for the PR. Closes #35.
Enhancements
-
Adds
Image.normalize/1which normalizes an image by expanding the luminance of an image to cover the full dynamic range. -
Adds
Image.autolevel/1which scales each band of an image to fit the full dynamic range. UnlikeImage.normalize/1, each band is scaled separately. -
Adds
Image.erode/2which erodes pixels from the edge of an image mask. This can be useful to remove a small amount of colour fringing around the edge of an image. -
Adds
Image.dilate/2which dilates pixels from the edge of an image mask. -
Adds
Image.trim/2which trims an image to the bounding box of the non-background area. -
Adds
Image.flatten/1which flattens an alpha layer out of an image -
Image.Options.Write.validate_options/2now validates options appropriate to each image type in order to make validation more robust. -
Adds
:minimize_file_sizeoption toImage.write/2for JPEG and PNG files which iftruewill apply a range of techniques to minimize the size of the image file at the expense of time to save the image and potentially image quality.
13
kip
ex_cldr Core Team
Today is object detection Sunday. Greatly inspired by the fabulous talk by @hansihe at the Warsaw Elixir Meetup earlier this month I transcribed his live coding example and implemented it in a new experimental module, Image.Detection.
Based upon his solid observations I also added some quality of life improvements to the detect branch of Image:
Image.from_kino/2andImage.from_kino!/2to easily consume the image data from aKino.Input.Imagedata source in LivebookImage.Shape.rect/3andImage.Shape.rect!/3to have a composable way to draw rectangles - specifically object bounding boxes in this case.Image.embed/4andImage.embed!/4to make it much easier to conform an image to the dimensions required by an ML model.
The code is ready for fun and experimentation. Given some of the tricky dependency configuration at the moment it can only be used as a GitHub dependency for now. You can add it in a mix.exs as:
{:image, github: "elixir-image/image", branch: "detect"}`
Demo example
Livebook coming this week!
iex> i = Image.open!("./test/support/images/elixir_warsaw_meetup.png")
%Vix.Vips.Image{ref: #Reference<0.3196308165.1633288229.90166>}
iex> Image.Detection.detect(i)
{:ok, %Vix.Vips.Image{ref: #Reference<0.3196308165.1633288229.90638>}}
The code
Its amazing how little code this takes with Nx, Axon and Axon Onnx.
def detect(%Vimage{} = image, model_path \\ default_model_path()) do
# Import the model and extract the
# prediction function and its parameters.
{model, params} = AxonOnnx.import(model_path)
{_init_fn, predict_fn} = Axon.build(model, compiler: EXLA)
# Flatten out any alpha band then resize the image
# so the longest edge is the same as the model size,
# then add a black border to expand the shorter dimension
# so the overall image conforms to the model requirements.
prepared_image =
image
|> Image.flatten!()
|> Image.thumbnail!(@yolo_model_image_size)
|> Image.embed!(@yolo_model_image_size, @yolo_model_image_size)
# Move the image to Nx. This is nothing more
# than moving a pointer under the covers
# so its efficient. Then conform the data to
# the shape and type required for the model.
# Last we add an additional axis that represents
# the batch (we use only a batch of 1).
batch =
prepared_image
|> Image.to_nx!()
|> Nx.transpose(axes: [2, 0, 1])
|> Nx.as_type(:f32)
|> Nx.divide(255)
|> Nx.new_axis(0)
# Run the prediction model, extract
# the only batch that was sent
# and transpose the axis back to
# {width, height} layout for further
# image processing.
result =
predict_fn.(params, batch)[0]
|> Nx.transpose(axes: [1, 0])
# Filter the data by certainty,
# zip with the class names, draw
# bounding boxes and labels and the
# trim off the extra pixels we added
# earlier to get back to the original
# image shape.
result
|> Yolo.NMS.nms(0.5)
|> Enum.zip(classes())
|> draw_bbox_with_labels(prepared_image)
|> Image.trim()
end
Next steps
This is a proof-of-concept only. The API will almost certainly change - not all use cases require painting a bounding box with labels. Feedback however is most welcome!.
Thanks again to @hansihe, the work is all his.
16
kwando
Is there anywhere I can see a full script for this? I tried to get it running in Livebook and a newly created mix project but cant get the dependenies right 
1
kip
ex_cldr Core Team
Sure, it’s referenced in the post in the config line for mix.ibex’s and link behind the Image,Detection link. Just look at the detect branch at https://github.com/elixir-image/image`.
I had to do quite some munging to get the deps right too hence the GitHub dependencies.
1
kip
ex_cldr Core Team
I’ve made a simple Livebook to demonstrate how to install, configure and do object detection:
2
hansihe
Is there an easy way to make a copy of a mutable Image? I looked around the docs for something, but I couldn’t find anything. That’s the reason I used the closure to make images for drawing in the talk.
1
kip
ex_cldr Core Team
@hansihe, A mutable image is already copied for you and operations are serialised behind a genserver. So it is mutable - but only by you. And all operations are serialised so it’s also thread safe.
You can perform multiple mutations on a single copy of the image by using Image.mutuate/2 (I have just updated the documentation on Image.mutate/2 quite a bit to make this clearer).
Mutation example
# The image is copied and operations
# are serialized behind a genserver.
# Only one copy is made but all operations
# will be serialized behind a genserver.
# When the function returns the genserver
# is broken down and the underlying
# mutated `t:Vix.Vips.Image.t/0` is returned.
iex> Image.mutate image, fn mutable_image ->
...> mutable_image
...> |> Image.Draw.rect!(0, 0, 10, 10, color: :red)
...> |> Image.Draw.rect!(2, 20, 10, 10, color: :green)
...> |> Image.Draw.rect!(50, 50, 10, 10, color: :blue)
...> end
By wrapping multiple mutations in an Image.mutate/2 call there should be performance improvements - although for safety each mutation is still serialised through a genserver.
1