Machine Learning in Elixir (PragProg)

Learning Resources Books
, , , , ,
21

If it is of any help, I am deep into AI in the last 2 years and I still can’t understand most Hugging Face projects. But I think that’s ok: I also don’t understand most of what is on GitHub, except for things written in certain languages, like Elixir or Erlang.

Therefore, the book will give you similar context. It is unlikely you will fully grasp any Hugging Face model, but there are certain models, architectures, and problems that will make a lot more sense once you read it.

Unfortunately there is no quantization at the moment but we are working on it. You can fine-tune models and one of the chapters discuss it, but you will most likely need a GPU and that’s one particular area we want to further improve the UX on.

13 Likes
22

I have the “practical deep learning” course from fast.ai on my radar too. There have been some mentions on the forum here, of people who recommend it as a foundation course.

I’m curious if anyone can comment on how this course and the book of Sean relate to each other, in the context of someone proficient in Elixir but has no knowledge about ML/AI (things might be different if you’re coming from a python background). Is one of the two a better starting point? Will they teach similar things?

2 Likes
23

I have been experimenting with ML throughout the years and I have been able to complete a few client projects via Python via TensorFlow, Swift via Create ML, and Wolfram via Mathematica. Thus, I was wondering, are there posts and/or articles for making sense of Elixir’s ML package ecosystem?

24

This page has a summary: Numerical Elixir (Nx) · GitHub

3 Likes
25

Reads like a breeze and goes also deep (looking at you, Chapter 6 ;-). Thank you, I love the editing and overall writing style of the book and the examples you picked to explain something.

26

I have finished Part 1 of Howard’s course in multiple versions and made it halfway through Part II. They cover the same material really, but in reverse order. Moriarty starts with the primitive foundations and builds up to complex applications, whereas Howard does the reverse by having you create an app with his FastAPI library within 5 to 10 minutes, then works backwards until you are creating your own custom deep learning models. Howard has very convincing reasons for this difference in pedagogy, and I think if you have never learned or used in deep learning it’s the best approach. “Practical Deep Learning for Coders” also only covers machine learning, that is methods that don’t use deep learning, tangentially.

So whether you should first learn deep learning in Python, then relearn everything again in Elixir is a matter of time and preference. Howard, in my humble opinion, is really one of the greatest teachers of programming alive today, so it’s hard to not recommend his course when given the opportunity.

1 Like
27

I don’t know if this is the right place to ask for a clarification? In chapter 6, when discussing hidden layers the book says:

It’s common to use hidden widths that are multiples of two

Is it common to use hidden widths that are powers of 2, rather than just multiples of 2? For example the answers to this question seem to be discussing using a width of 265, 512 or 1024.

28

I’m just about finishing chapter 1, I barely understand a thing but that’s fine. It’s a new adventure. I had a runtime exception while evaluating the final snippet. Below is the full source code; note the deprecation of the map %{} parameter. It looks like the an f64 is being used in place of an f32.

Mix.install([{:axon, "~> 0.5"}, {:nx, "~> 0.5"}, {:explorer, "~> 0.5"}, {:kino, "~> 0.8"}])

# ── Section ──

require Explorer.DataFrame, as: DF
iris = Explorer.Datasets.iris()


cols = ~w(sepal_width sepal_length petal_length petal_width)
normalized_iris = 
  DF.mutate(iris, for col <- across(^cols) do
    {col.name, (col - mean(col)) / standard_deviation(col)}
  end)

normalized_iris = DF.mutate(normalized_iris, [
  species: Explorer.Series.cast(species, :category)
])

shuffled_normalized_iris = DF.shuffle(normalized_iris)

train_df = DF.slice(shuffled_normalized_iris, 0..119)
test_df = DF.slice(shuffled_normalized_iris, 120..149)

feature_columns = [
  "sepal_length",
  "sepal_width",
  "petal_length",
  "petal_width"
]

x_train = Nx.stack(train_df[feature_columns], axis: -1)
y_train = train_df["species"]
          |> Nx.stack(axis: -1)
          |> Nx.equal(Nx.iota({1, 3}, axis: -1))

x_test = Nx.stack(test_df[feature_columns], axis: -1)
y_test = 
  test_df["species"]
|> Nx.stack(axis: -1)
|> Nx.equal(Nx.iota({1, 3}, axis: -1))

model = 
  Axon.input("iris_features", shape: {nil, 4})
|> Axon.dense(3, activation: :softmax)

Axon.Display.as_graph(model, Nx.template({1, 4}, :f32))

data_stream = Stream.repeatedly(fn ->
  {x_train, y_train}
end)

trained_model_state = 
  model
|> Axon.Loop.trainer(:categorical_cross_entropy, :sgd)
|> Axon.Loop.metric(:accuracy)
|> Axon.Loop.run(data_stream, %{}, iterations: 500, epochs: 10)

Output

16:24:37.234 [warning] passing parameter map to initialization is deprecated, use %Axon.ModelState{} instead
Epoch: 0, Batch: 0, accuracy: 0.1083333 loss: 0.0000000

** (ArgumentError) argument at position 3 is not compatible with compiled function template.

%{i: #Nx.Tensor<
    s32
  >, model_state: #Inspect.Error<
  got Protocol.UndefinedError with message:

      """
      protocol Enumerable not implemented for #Nx.Tensor<
        f32[3]
      > of type Nx.Defn.TemplateDiff (a struct). This protocol is implemented for the following type(s): Date.Range, Explorer.Series.Iterator, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Kino.Control, Kino.Input, Kino.JS.Live, List, Map, MapSet, Range, Stream, Table.Mapper, Table.Zipper
      """

  while inspecting:

      %{
        data: %{
          "dense_0" => %{
            "bias" => #Nx.Tensor<
              f32[3]
            >,
            "kernel" => #Nx.Tensor<
              f32[4][3]
            >
          }
        },
        state: %{},
        __struct__: Axon.ModelState,
        parameters: %{"dense_0" => ["bias", "kernel"]},
        frozen_parameters: %{}
      }

  Stacktrace:

    (elixir 1.17.2) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir 1.17.2) lib/enum.ex:166: Enumerable.reduce/3
    (elixir 1.17.2) lib/enum.ex:4423: Enum.reduce/3
    (axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
    (stdlib 6.0) maps.erl:860: :maps.fold_1/4
    (axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
    (stdlib 6.0) maps.erl:860: :maps.fold_1/4
    (axon 0.7.0) lib/axon/model_state.ex:320: Inspect.Axon.ModelState.inspect/2

>, loss: 
  <<<<< Expected <<<<<
  #Nx.Tensor<
    f32
  >
  ==========
  #Nx.Tensor<
    f64
  >
  >>>>> Argument >>>>>
  , optimizer_state: {%{scale: #Nx.Tensor<
       f32
     >}}, loss_scale_state: %{}, y_true: #Nx.Tensor<
    u8[120][3]
  >, y_pred: #Nx.Tensor<
    f64[120][3]
  >}

    (nx 0.9.2) lib/nx/defn.ex:342: anonymous fn/7 in Nx.Defn.compile_flatten/5
    (nx 0.9.2) lib/nx/lazy_container.ex:73: anonymous fn/3 in Nx.LazyContainer.Map.traverse/3
    (elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    (elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    (nx 0.9.2) lib/nx/lazy_container.ex:72: Nx.LazyContainer.Map.traverse/3
    (nx 0.9.2) lib/nx/defn.ex:339: Nx.Defn.compile_flatten/5
    (nx 0.9.2) lib/nx/defn.ex:331: anonymous fn/4 in Nx.Defn.compile/3
    #cell:ti265afq7l6ocfgv:9: (file)
29

Hi, I think that’s because you’re using a newer version of axon. Try pinning it as described here.

1 Like