Koszal

Koszal

I am trying to change a livebook smart cell to a one-file Elixir script, mostly for educational reasons. I successfully Mix.install dependencies, load the model (“Salesforce/blip-image-captioning-base”), create a featurizer, a tokenizer, run the configure step and create a serving.

serving =
  Bumblebee.Vision.image_to_text(model_info, featurizer, tokenizer, generation_config,
    compile: [batch_size: 1],
    defn_options: [compiler: EXLA]
  )

And then I load an image like so:

image = 
  "/tmp/22911352.jpg"
  |> File.read!()
  |> Nx.from_binary(:u8)
  |> ....

I cannot figure out what the last step in the image processing pipeline should be. The original smart cell does the following:

    image =
      image.file_ref
      |> Kino.Input.file_path()
      |> File.read!()
      |> Nx.from_binary(:u8)
      |> Nx.reshape({image.height, image.width, 3})

The height and width are 3468, 4624 but when I hard-code the values and do

      ..
      |> Nx.reshape({3468, 4624, 3})

I get the following error:

  cannot reshape, current shape {4190582} is not compatible with new shape {3468, 4624, 3}

The very last step needs to be

  Nx.Serving.run(serving, image)

This is my first look at livebook - I don’t yet understand where image in the smart cell comes from.

Showing Posts 1 to 3

Koszal

Koszal OP

Oh… Am I missing jpeg decoding machinery? How does kino do it?

josevalim

josevalim

Creator of Elixir

Kino does it on the client. You can check the examples folder in Bumblebee. You can however use libraries like StbImage to read the file for you and NxImage for doing the resizing.

Koszal

Koszal OP

Yes! I have figured it out today!

For my batch job I used the Image module and then all I needed to do was

    {:ok, image} = Image.open(fname)
    {:ok, tensor} = Image.to_nx(image)

    %{results: [%{text: text}]} = Nx.Serving.run(serving, tensor)

Livebook is awesome for exploring ML. I have very little experience with ML and I am barely starting to learn Elixir and after two days I will have classified all images on disk.

Thank you!

— All posts loaded —

Where Next? Top

Trending in Questions Top

katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
achenet
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
asweet-confluent
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews