I’m brand new to AI and using NX/Bumblebee. I have attempted to adapt this example from the bumblebee docs to generate a narrative description of a kids math problem using GPT 2:

  def make_story_question(%MathQuiz.Models.MathQuizItem{} = question) do
    model_name = "openai-community/gpt2"
    {:ok, granite} = Bumblebee.load_model({:hf, model_name})
    {:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, model_name})

    {:ok, generation_config} = Bumblebee.load_generation_config({:hf, model_name})

    serving = Bumblebee.Text.generation(granite, tokenizer, generation_config)

    question_prompt =
      "Write a narrative story question for children for the math problem #{question.first_num} plus #{question.second_num}."
      |> IO.inspect(label: "Prompt")

    # text_input = Kino.Input.text(question_prompt, default: "Tomorrow it will be")
    # text = Kino.Input.read(text_input)

    Nx.Serving.run(serving, question_prompt)
  end

When I run the code above, this function never completes (even after 10 minutes) on a moderately performant desktop. An example output on the text prompt is:

Prompt: “Write a narrative story question for children for the math problem 3 plus 3.”

I suspect that the Nx.Serving.run command is likely opening a server, and never closing. How do I get this function to get the response from the model? Am I doing something stupid basic wrong?