There is “kino_vix” to use in livebook, which automatically renders vix images inline. It is a great idea but not fully working correctly at the moment 
kwando
1
kip
ex_cldr Core Team
Please do open an issue if it’s not working. I authored that very minor contribution so I’ll happily fix it too.
1
kip
ex_cldr Core Team
Image 0.18.0 has just been published with the primary new feature being integration with Bumblebee to provide image classification. Bumblebee is just amazing and I can’t wait to add other integrations. There have been requests for StableDiffusion integration so I’ll take a look at that next.
Example
iex> Image.open!("./test/support/images/puppy.webp")
...> |> Image.Classification.classify()
%{predictions: [%{label: "Blenheim spaniel", score: 0.9701485633850098}]}
iex> Image.open!("./test/support/images/puppy.webp")
...> |> Image.Classification.labels()
["Blenheim spaniel"]
Enhancements
6
AstonJ
Amazing work Kip!
Awesome!! Can’t wait!! 
Looking forward to seeing what you add and thanks for all your work, I think all these features are going to interest a lot of people 
2
kip
ex_cldr Core Team
@AstonJ, StableDiffusion “image-to-image” mode is not yet available in Axon/Bumblebee but it is being tracked. I go ahead and add “text-to-image” for now (in the next couple of days).
2
AstonJ
Thanks for the update Kip, I look forward to seeing text-to-image …and image-to-image when/if it becomes available
0
kip
ex_cldr Core Team
Nearly there with Image.Generation.text_to_image/2 (need some more work on documentation before release). Huge amounts of fun with this. On my M1 Max is takes about a minute to generate an image so definitely better to have a supported GPU.
iex> [i] = Image.Generation.text_to_image("impressionist purple numbat in the style of monet")
[%Vix.Vips.Image{ref: #Reference<0.1344162992.4105306142.245952>}]
9
AstonJ
Looking good Kip! I’m sure lots of fun will be had with it once it’s released!
Also I’m not sure whether you (or @seanmor5) are aware but Google extended Stable Diffusion and created DreamBooth. It’s open source and offers a way to tweak Stable Diffusion to produce variations of photos in different styles. It’s what Avatar AI uses and is exactly what I am looking for - it would be awesome if we could have this! 
The project I would use it for is..
Drum roll...
Edit: screenshot snipped! I need to stop talking about potential projects because apparently when you share details of them it gives you the same sense of achievement of having completed them - so ends up working against you! Open to my arm being twisted tho! But it will take a lot of twisting, haha!
I’m in no major rush for it tho - if I start that project it will be when LV is at 1.0 (or when Chris thinks we’re unlikely to see major changes to it).
0
kip
ex_cldr Core Team
Motivated by the discussion on extracting frames from video, newly published Image version 0.22 supports some basic capabilities based upon the excellent Evision.VideoCapture module.
Enhancements
-
Adds
Image.Video.image_from_video/2to extract images from frames in a video file or video camera. Includes support for:frameand:millisecondseek options. Seek options are only supported for video files, not video streams. -
Adds
Image.Video.stream!/2that returns an enumerable stream of frames as images. The stream takes a range as a parameter. For example: -
Adds
Image.Video.scrub/2that scrubs the video head forward a number of frames. -
Adds
Image.Video.seek/2to seek the video head to the requested frame or millisecond. Seeking is supported for video files only, not video streams. Seeking is not guaranteed to be frame accurate (due to underlying OpenCV issues).
Examples
# Extracting an image
iex> {:ok, video} = Image.Video.open "./test/support/video/video_sample.mp4"
iex> {:ok, _image} = Image.Video.image_from_video(video)
iex> {:ok, _image} = Image.Video.image_from_video(video, frame: 0)
iex> {:ok, _image} = Image.Video.image_from_video(video, millisecond: 1_000)
# Streaming images
# Extract every second frame starting at the
# first frame and ending at the last frame.
iex> "./test/support/video/video_sample.mp4"
...> |> Image.Video.stream!(frame: 0..-1//2)
...> |> Enum.to_list()
[
%Vix.Vips.Image{ref: #Reference<0.2048151986.449445916.177398>},
%Vix.Vips.Image{ref: #Reference<0.2048151986.449445916.177400>},
%Vix.Vips.Image{ref: #Reference<0.2048151986.449445916.177402>},
...
]
5
olivermt
@kip there is another regression straight from dependabot here:
== Compilation error in file lib/image/options/video.ex ==
Error: ** (CompileError) lib/image/options/video.ex:8: Evision.VideoCapture.__struct__/0 is undefined, cannot expand struct Evision.VideoCapture. Make sure the struct name is correct. If the struct name exists and is correct but it still cannot be found, you likely have cyclic module usage in your code
This is similar to the bumblebee stuff where you should safeguard the existence of dependent libraries and flag on/off your features I reckon?
0