Machine Learning in Elixir (PragProg)

I found a solution Machine Learning in Elixir: Nx.Random.uniform_split wrong arguments 130 - PragProg Customers - Devtalk

Thank you.

@kodepett if you don’t mind, could you copy your message over to the book’s errata page on Devtalk and tag the author @seanmor5?

Hi all, not sure if this is the right place to post.
On chapter 6, Deep learning, page 128, the snippet uses uniform_split/2 which is undefined.
I checked various versions of Nx

Nx.Random.uniform_split(new_key, shape: {})
|> NeuralNetwork.predict(w1, b2, w2, b2)

Hi, I think that’s because you’re using a newer version of axon. Try pinning it as described here.

I’m just about finishing chapter 1, I barely understand a thing but that’s fine. It’s a new adventure. I had a runtime exception while evaluating the final snippet. Below is the full source code; note the deprecation of the map %{} parameter. It looks like the an f64 is being used in place of an f32.

Mix.install([{:axon, "~> 0.5"}, {:nx, "~> 0.5"}, {:explorer, "~> 0.5"}, {:kino, "~> 0.8"}])

# ── Section ──

require Explorer.DataFrame, as: DF
iris = Explorer.Datasets.iris()


cols = ~w(sepal_width sepal_length petal_length petal_width)
normalized_iris = 
  DF.mutate(iris, for col <- across(^cols) do
    {col.name, (col - mean(col)) / standard_deviation(col)}
  end)

normalized_iris = DF.mutate(normalized_iris, [
  species: Explorer.Series.cast(species, :category)
])

shuffled_normalized_iris = DF.shuffle(normalized_iris)

train_df = DF.slice(shuffled_normalized_iris, 0..119)
test_df = DF.slice(shuffled_normalized_iris, 120..149)

feature_columns = [
  "sepal_length",
  "sepal_width",
  "petal_length",
  "petal_width"
]

x_train = Nx.stack(train_df[feature_columns], axis: -1)
y_train = train_df["species"]
          |> Nx.stack(axis: -1)
          |> Nx.equal(Nx.iota({1, 3}, axis: -1))

x_test = Nx.stack(test_df[feature_columns], axis: -1)
y_test = 
  test_df["species"]
|> Nx.stack(axis: -1)
|> Nx.equal(Nx.iota({1, 3}, axis: -1))

model = 
  Axon.input("iris_features", shape: {nil, 4})
|> Axon.dense(3, activation: :softmax)

Axon.Display.as_graph(model, Nx.template({1, 4}, :f32))

data_stream = Stream.repeatedly(fn ->
  {x_train, y_train}
end)

trained_model_state = 
  model
|> Axon.Loop.trainer(:categorical_cross_entropy, :sgd)
|> Axon.Loop.metric(:accuracy)
|> Axon.Loop.run(data_stream, %{}, iterations: 500, epochs: 10)

Output

16:24:37.234 [warning] passing parameter map to initialization is deprecated, use %Axon.ModelState{} instead
Epoch: 0, Batch: 0, accuracy: 0.1083333 loss: 0.0000000

** (ArgumentError) argument at position 3 is not compatible with compiled function template.

%{i: #Nx.Tensor<
    s32
  >, model_state: #Inspect.Error<
  got Protocol.UndefinedError with message:

      """
      protocol Enumerable not implemented for #Nx.Tensor<
        f32[3]
      > of type Nx.Defn.TemplateDiff (a struct). This protocol is implemented for the following type(s): Date.Range, Explorer.Series.Iterator, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Kino.Control, Kino.Input, Kino.JS.Live, List, Map, MapSet, Range, Stream, Table.Mapper, Table.Zipper
      """

  while inspecting:

      %{
        data: %{
          "dense_0" => %{
            "bias" => #Nx.Tensor<
              f32[3]
            >,
            "kernel" => #Nx.Tensor<
              f32[4][3]
            >
          }
        },
        state: %{},
        __struct__: Axon.ModelState,
        parameters: %{"dense_0" => ["bias", "kernel"]},
        frozen_parameters: %{}
      }

  Stacktrace:

    (elixir 1.17.2) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir 1.17.2) lib/enum.ex:166: Enumerable.reduce/3
    (elixir 1.17.2) lib/enum.ex:4423: Enum.reduce/3
    (axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
    (stdlib 6.0) maps.erl:860: :maps.fold_1/4
    (axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
    (stdlib 6.0) maps.erl:860: :maps.fold_1/4
    (axon 0.7.0) lib/axon/model_state.ex:320: Inspect.Axon.ModelState.inspect/2

>, loss: 
  <<<<< Expected <<<<<
  #Nx.Tensor<
    f32
  >
  ==========
  #Nx.Tensor<
    f64
  >
  >>>>> Argument >>>>>
  , optimizer_state: {%{scale: #Nx.Tensor<
       f32
     >}}, loss_scale_state: %{}, y_true: #Nx.Tensor<
    u8[120][3]
  >, y_pred: #Nx.Tensor<
    f64[120][3]
  >}

    (nx 0.9.2) lib/nx/defn.ex:342: anonymous fn/7 in Nx.Defn.compile_flatten/5
    (nx 0.9.2) lib/nx/lazy_container.ex:73: anonymous fn/3 in Nx.LazyContainer.Map.traverse/3
    (elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    (elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    (nx 0.9.2) lib/nx/lazy_container.ex:72: Nx.LazyContainer.Map.traverse/3
    (nx 0.9.2) lib/nx/defn.ex:339: Nx.Defn.compile_flatten/5
    (nx 0.9.2) lib/nx/defn.ex:331: anonymous fn/4 in Nx.Defn.compile/3
    #cell:ti265afq7l6ocfgv:9: (file)

I don’t know if this is the right place to ask for a clarification? In chapter 6, when discussing hidden layers the book says:

It’s common to use hidden widths that are multiples of two

Is it common to use hidden widths that are powers of 2, rather than just multiples of 2? For example the answers to this question seem to be discussing using a width of 265, 512 or 1024.

I have finished Part 1 of Howard’s course in multiple versions and made it halfway through Part II. They cover the same material really, but in reverse order. Moriarty starts with the primitive foundations and builds up to complex applications, whereas Howard does the reverse by having you create an app with his FastAPI library within 5 to 10 minutes, then works backwards until you are creating your own custom deep learning models. Howard has very convincing reasons for this difference in pedagogy, and I think if you have never learned or used in deep learning it’s the best approach. “Practical Deep Learning for Coders” also only covers machine learning, that is methods that don’t use deep learning, tangentially.

So whether you should first learn deep learning in Python, then relearn everything again in Elixir is a matter of time and preference. Howard, in my humble opinion, is really one of the greatest teachers of programming alive today, so it’s hard to not recommend his course when given the opportunity.

Reads like a breeze and goes also deep (looking at you, Chapter 6 ;-). Thank you, I love the editing and overall writing style of the book and the examples you picked to explain something.

This page has a summary: Numerical Elixir (Nx) · GitHub

I have been experimenting with ML throughout the years and I have been able to complete a few client projects via Python via TensorFlow, Swift via Create ML, and Wolfram via Mathematica. Thus, I was wondering, are there posts and/or articles for making sense of Elixir’s ML package ecosystem?