Hey everyone. I try to refrain from coming here with all my problems but I am truly at a loss with this one and I thought it was worth a shot to see if anyone had any ideas on the matter.
The background is that I am just trying to get Whisper set up via Bumblebee. I am getting an error that references a to_operator function that I cannot seem to find anywhere. It is in a module I cannot seem to find, either. The whole thing is just throwing me off.
Anyways, I am following a pretty basic set up as I will show, and then just getting an error that seems to stem from it creating the pow operator with the arguments that it gets, but I just cant get past that in my head. When I removed the EXLA backend line, I got an error stating that the pow operator was undefined.
defp serving do
{:ok, whisper} = Bumblebee.load_model({:hf, "openai/whisper-tiny"})
{:ok, featurizer} = Bumblebee.load_featurizer({:hf, "openai/whisper-tiny"})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "openai/whisper-tiny"})
serving =
Bumblebee.Audio.speech_to_text(whisper, featurizer, tokenizer,
max_new_tokens: 100,
defn_options: [compiler: EXLA]
)
end
def handle_event("predict", value, socket) do
text = {:file, "03_Smile.wav"}
|> then(&Nx.Serving.run(serving, &1))
|> get_in([:results, Access.all(), :text])
{:noreply,
socket
|> assign(text: text)}
end
& then here is the to_operator error message:
[error] GenServer #PID<0.2273.0> terminating
** (FunctionClauseError) no function clause matching in EXLA.Defn.to_operator/4
(exla 0.4.2) EXLA.Defn.to_operator(:pow, [%EXLA.Op{builder: #Reference<0.2409059649.1181614081.76036>, ref: #Reference<0.2409059649.1181614081.76933>}, %EXLA.Op{builder: #Reference<0.2409059649.1181614081.76036>, ref: #Reference<0.2409059649.1181614081.76934>}], #Nx.Tensor<
f32[frames: 3000][frequencies: 400]
Nx.Defn.Expr
parameter a:0 f32[480000]
b = iota nil s64[400][1]
c = concatenate [0, b], 1 s64[400][2]
d = while {0.0, 0, 0, a, c} tuple5
e = elem d, 0 f32[3000][400]
f = iota nil f32[401]
g = multiply 6.283185307179586, f f32[401]
h = divide g, 400 f32[401]
i = cos h f32[401]
j = subtract 1, i f32[401]
k = multiply 0.5, j f32[401]
l = slice k, [0], [400], [1] f32[400]
m = multiply e, l f32[3000][400]
n = fft m, length: 400, eps: 1.0e-10 c64[3000][400]
o = abs n f32[3000][400]
p = pow o, 2 f32[3000][400]
>, %{builder: %EXLA.Builder{ref: #Reference<0.2409059649.1181614081.76036>, parent: nil, name: "#Function<136.4924062/1 in Nx.Defn.Compiler.fun/2>"}, params: %{0 => %EXLA.Op{builder: #Reference<0.2409059649.1181614081.76036>, ref: #Reference<0.2409059649.1181614081.76245>}}, precision: :highest, scope_ids: %{#Reference<0.2409059649.1181483009.76217> => :reshape, #Reference<0.2409059649.1181483009.76213> => :slice, #Reference<0.2409059649.1181483009.76207> => :greater_equal, #Reference<0.2409059649.1181483009.76198> => :add, #Reference<0.2409059649.1181483009.76204> => :multiply, #Reference<0.2409059649.1181483009.76226> => :min, #Reference<0.2409059649.1181483009.76194> => :subtract, #Reference<0.2409059649.1181483009.76232> => :reshape, #Reference<0.2409059649.1181483009.76227> => :max, #Reference<0.2409059649.1181483009.76234> => :slice, #Reference<0.2409059649.1181483009.76202> => :divide, #Reference<0.2409059649.1181483009.76039> => :multiply, #Reference<0.2409059649.1181483009.76210> => :exp, #Reference<0.2409059649.1181483009.76238> => :log, #Reference<0.2409059649.1181483009.76196> => :iota, #Reference<0.2409059649.1181483009.76030> => :parameter, #Reference<0.2409059649.1181483009.76193> => :reshape, #Reference<0.2409059649.1181483009.76225> => :divide, #Reference<0.2409059649.1181483009.76205> => :add, #Reference<0.2409059649.1181483009.76208> => :subtract, #Reference<0.2409059649.1181483009.76243> => :add, #Reference<0.2409059649.1181483009.76223> => :slice, #Reference<0.2409059649.1181483009.76220> => :negate, #Reference<0.2409059649.1181483009.76209> => :multiply, #Reference<0.2409059649.1181483009.76216> => :reshape, #Reference<0.2409059649.1181483009.76201> => :subtract, #Reference<0.2409059649.1181483009.76229> => :slice, #Reference<0.2409059649.1181483009.76203> => :iota, #Reference<0.2409059649.1181483009.76175> => :fft, #Reference<0.2409059649.1181483009.76169> => :elem, #Reference<0.2409059649.1181483009.76219> => :slice, #Reference<0.2409059649.1181483009.76206> => :multiply, #Reference<0.2409059649.1181483009.76197> => :multiply, #Reference<0.2409059649.1181483009.76222> => :divide, #Reference<0.2409059649.1181483009.76231> => :divide, #Reference<0.2409059649.1181483009.76168> => :while, #Reference<0.2409059649.1181483009.76233> => :multiply, #Reference<0.2409059649.1181483009.76191> => :pow, #Reference<0.2409059649.1181483009.76215> => :subtract, #Reference<0.2409059649.1181483009.76190> => :abs, #Reference<0.2409059649.1181483009.76200> => :reshape, #Reference<0.2409059649.1181483009.76192> => :reshape, #Reference<0.2409059649.1181483009.76047> => :iota, #Reference<0.2409059649.1181483009.76042> => :subtract, #Reference<0.2409059649.1181483009.76048> => :concatenate, #Reference<0.2409059649.1181483009.76211> => :multiply, ...}})
(exla 0.4.2) lib/exla/defn.ex:582: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(nx 0.5.1) lib/nx/defn/tree.ex:212: Nx.Defn.Tree.apply_args/4
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(elixir 1.14.3) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(elixir 1.14.3) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(elixir 1.14.3) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(elixir 1.14.3) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
(exla 0.4.2) lib/exla/defn.ex:474: EXLA.Defn.recur_operator/3
(elixir 1.14.3) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(exla 0.4.2) lib/exla/defn.ex:581: EXLA.Defn.cached_recur_operator/4
Last message: %Phoenix.Socket.Message{topic: "lv:phx-F04_pmJLkpzdPyZh", event: "event", payload: %{"event" => "predict", "type" => "click", "value" => %{"value" => ""}}, ref: "8", join_ref: "4"}
State: %{components: {%{}, %{}, 1}, join_ref: "4", serializer: Phoenix.Socket.V2.JSONSerializer, socket: #Phoenix.LiveView.Socket<id: "phx-F04_pmJLkpzdPyZh", endpoint: AppWeb.Endpoint, view: AppWeb.WhisperLive, parent_pid: nil, root_pid: #PID<0.2273.0>, router: AppWeb.Router, assigns: %{__changed__: %{}, flash: %{}, live_action: :index, prediction: nil, running?: false, serving: %Nx.Serving{module: Nx.Serving.Default, arg: #Function<0.94466209/1 in Bumblebee.Audio.SpeechToText.speech_to_text/4>, client_preprocessing: #Function<1.94466209/1 in Bumblebee.Audio.SpeechToText.speech_to_text/4>, client_postprocessing: #Function<2.94466209/3 in Bumblebee.Audio.SpeechToText.speech_to_text/4>, distributed_postprocessing: &Function.identity/1, process_options: [batch_size: nil], defn_options: [compiler: EXLA]}, text: nil}, transport_pid: #PID<0.2266.0>, ...>, topic: "lv:phx-F04_pmJLkpzdPyZh", upload_names: %{}, upload_pids: %{}}
Thanks everyone, you guys are great. Any help is much appreciated, and just trying to see if anyone just make a little more sense of what is going on exactly than I am able to
Thanks again