Is there a way to have SmolDocling running on Bumblebee?

Elixir Framework Forums Nx Forum
,
1

Hi all,

After having read about SmolDocling :front_facing_baby_chick: for parsing PDF:s I’m really keen to give it a spin as an alternative to the Azure Document Intelligence service (possibly!) :slightly_smiling_face:

SmolDocling is a multimodal Image-Text-to-Text model designed for efficient document conversion. It retains Docling’s most popular features while ensuring full compatibility with Docling through seamless support for DoclingDocuments .

However, I do realise that I cannot expect this model to work on Bumblebee out of the gate.

In another post Jonathan Klosko commented on use of a “receipts model” where a few required features were missing in Bumblebee. It would be reasonable to assume that this would be the case for SmolDocling as well.

** (ArgumentError) could not match the class name "Idefics3ForConditionalGeneration" to any of the supported models, please specify the :module and :architecture options
    (bumblebee 0.6.3) lib/bumblebee.ex:434: Bumblebee.do_load_spec/4
    (bumblebee 0.6.3) lib/bumblebee.ex:603: Bumblebee.maybe_load_model_spec/3
    (bumblebee 0.6.3) lib/bumblebee.ex:591: Bumblebee.load_model/2
    #cell:t4ttbdzq2pvyqvx7:1: (file)

Is there any reasonable path to have SmolDocling running on Bumblebee? If so, what would that entail?

Thank you in advance for any input!

Kind regards,
Carl

2 Likes