Axon - Unable to get buffer

Hi All,

I run my training model in livebook. In case I would like to train the same model several times with an, the output sometime become useless. I try to save the model in CubDB.

Do you have any suggestion what can be the issue? The result is usually get returned and saved correctly like 1/3 of the tries.

I’m trying to do something like this.

models_list =
Enum.to_list(1..3)
|> Enum.map(fn _ ->
    SetRuns.setrun(dataset.set1, "set1", {tensor_to_check, signals_to_check})
end)

Enum.map(models_list, fn {trained_model, dataset_name, precision} -> CubDB.put(:models_db, {"#{dataset_name}-precision", precision}, trained_model) end)

%{
  "dense_0" => %{
    "bias" => #Inspect.Error<
  got RuntimeError with message:

      """
      Unable to get buffer.
      """

  while inspecting:

      %{
        data: %EXLA.Backend{
          buffer: %EXLA.DeviceBuffer{
            ref: #Reference<27390.3598233730.3040477197.22534>,
            client_name: :host,
            device_id: 0,
            shape: %EXLA.Shape{
              ref: #Reference<27390.3598233730.2166226945.215890>,
              dims: {12},
              dtype: {:f, 32}
            }
          }
        },
        type: {:f, 32},
        names: [nil],
        __struct__: Nx.Tensor,
        vectorized_axes: [],
        shape: {12}
      }

  Stacktrace:

    (exla 0.6.4) lib/exla/device_buffer.ex:55: EXLA.DeviceBuffer.unwrap!/1
    (exla 0.6.4) lib/exla/backend.ex:133: EXLA.Backend.inspect/2
    (nx 0.6.4) lib/nx/tensor.ex:241: Inspect.Nx.Tensor.inspect/2
    (elixir 1.15.7) lib/inspect/algebra.ex:348: Inspect.Algebra.to_doc/2
    (elixir 1.15.7) lib/inspect.ex:329: Inspect.Map.to_assoc/3
    (elixir 1.15.7) lib/inspect/algebra.ex:479: Inspect.Algebra.container_each/6
    (elixir 1.15.7) lib/inspect/algebra.ex:456: Inspect.Algebra.container_doc/6
    (elixir 1.15.7) lib/inspect.ex:329: Inspect.Map.to_assoc/3

>,