I have the “practical deep learning” course from fast.ai on my radar too. There have been some mentions on the forum here, of people who recommend it as a foundation course.
I’m curious if anyone can comment on how this course and the book of Sean relate to each other, in the context of someone proficient in Elixir but has no knowledge about ML/AI (things might be different if you’re coming from a python background). Is one of the two a better starting point? Will they teach similar things?
I have been experimenting with ML throughout the years and I have been able to complete a few client projects via Python via TensorFlow, Swift via Create ML, and Wolfram via Mathematica. Thus, I was wondering, are there posts and/or articles for making sense of Elixir’s ML package ecosystem?
Reads like a breeze and goes also deep (looking at you, Chapter 6 ;-). Thank you, I love the editing and overall writing style of the book and the examples you picked to explain something.
I have finished Part 1 of Howard’s course in multiple versions and made it halfway through Part II. They cover the same material really, but in reverse order. Moriarty starts with the primitive foundations and builds up to complex applications, whereas Howard does the reverse by having you create an app with his FastAPI library within 5 to 10 minutes, then works backwards until you are creating your own custom deep learning models. Howard has very convincing reasons for this difference in pedagogy, and I think if you have never learned or used in deep learning it’s the best approach. “Practical Deep Learning for Coders” also only covers machine learning, that is methods that don’t use deep learning, tangentially.
So whether you should first learn deep learning in Python, then relearn everything again in Elixir is a matter of time and preference. Howard, in my humble opinion, is really one of the greatest teachers of programming alive today, so it’s hard to not recommend his course when given the opportunity.
I don’t know if this is the right place to ask for a clarification? In chapter 6, when discussing hidden layers the book says:
It’s common to use hidden widths that are multiples of two
Is it common to use hidden widths that are powers of 2, rather than just multiples of 2? For example the answers to this question seem to be discussing using a width of 265, 512 or 1024.
I’m just about finishing chapter 1, I barely understand a thing but that’s fine. It’s a new adventure. I had a runtime exception while evaluating the final snippet. Below is the full source code; note the deprecation of the map %{} parameter. It looks like the an f64 is being used in place of an f32.
Mix.install([{:axon, "~> 0.5"}, {:nx, "~> 0.5"}, {:explorer, "~> 0.5"}, {:kino, "~> 0.8"}])
# ── Section ──
require Explorer.DataFrame, as: DF
iris = Explorer.Datasets.iris()
cols = ~w(sepal_width sepal_length petal_length petal_width)
normalized_iris =
DF.mutate(iris, for col <- across(^cols) do
{col.name, (col - mean(col)) / standard_deviation(col)}
end)
normalized_iris = DF.mutate(normalized_iris, [
species: Explorer.Series.cast(species, :category)
])
shuffled_normalized_iris = DF.shuffle(normalized_iris)
train_df = DF.slice(shuffled_normalized_iris, 0..119)
test_df = DF.slice(shuffled_normalized_iris, 120..149)
feature_columns = [
"sepal_length",
"sepal_width",
"petal_length",
"petal_width"
]
x_train = Nx.stack(train_df[feature_columns], axis: -1)
y_train = train_df["species"]
|> Nx.stack(axis: -1)
|> Nx.equal(Nx.iota({1, 3}, axis: -1))
x_test = Nx.stack(test_df[feature_columns], axis: -1)
y_test =
test_df["species"]
|> Nx.stack(axis: -1)
|> Nx.equal(Nx.iota({1, 3}, axis: -1))
model =
Axon.input("iris_features", shape: {nil, 4})
|> Axon.dense(3, activation: :softmax)
Axon.Display.as_graph(model, Nx.template({1, 4}, :f32))
data_stream = Stream.repeatedly(fn ->
{x_train, y_train}
end)
trained_model_state =
model
|> Axon.Loop.trainer(:categorical_cross_entropy, :sgd)
|> Axon.Loop.metric(:accuracy)
|> Axon.Loop.run(data_stream, %{}, iterations: 500, epochs: 10)
Output
16:24:37.234 [warning] passing parameter map to initialization is deprecated, use %Axon.ModelState{} instead
Epoch: 0, Batch: 0, accuracy: 0.1083333 loss: 0.0000000
** (ArgumentError) argument at position 3 is not compatible with compiled function template.
%{i: #Nx.Tensor<
s32
>, model_state: #Inspect.Error<
got Protocol.UndefinedError with message:
"""
protocol Enumerable not implemented for #Nx.Tensor<
f32[3]
> of type Nx.Defn.TemplateDiff (a struct). This protocol is implemented for the following type(s): Date.Range, Explorer.Series.Iterator, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Kino.Control, Kino.Input, Kino.JS.Live, List, Map, MapSet, Range, Stream, Table.Mapper, Table.Zipper
"""
while inspecting:
%{
data: %{
"dense_0" => %{
"bias" => #Nx.Tensor<
f32[3]
>,
"kernel" => #Nx.Tensor<
f32[4][3]
>
}
},
state: %{},
__struct__: Axon.ModelState,
parameters: %{"dense_0" => ["bias", "kernel"]},
frozen_parameters: %{}
}
Stacktrace:
(elixir 1.17.2) lib/enum.ex:1: Enumerable.impl_for!/1
(elixir 1.17.2) lib/enum.ex:166: Enumerable.reduce/3
(elixir 1.17.2) lib/enum.ex:4423: Enum.reduce/3
(axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
(stdlib 6.0) maps.erl:860: :maps.fold_1/4
(axon 0.7.0) lib/axon/model_state.ex:359: anonymous fn/2 in Inspect.Axon.ModelState.get_param_info/1
(stdlib 6.0) maps.erl:860: :maps.fold_1/4
(axon 0.7.0) lib/axon/model_state.ex:320: Inspect.Axon.ModelState.inspect/2
>, loss:
<<<<< Expected <<<<<
#Nx.Tensor<
f32
>
==========
#Nx.Tensor<
f64
>
>>>>> Argument >>>>>
, optimizer_state: {%{scale: #Nx.Tensor<
f32
>}}, loss_scale_state: %{}, y_true: #Nx.Tensor<
u8[120][3]
>, y_pred: #Nx.Tensor<
f64[120][3]
>}
(nx 0.9.2) lib/nx/defn.ex:342: anonymous fn/7 in Nx.Defn.compile_flatten/5
(nx 0.9.2) lib/nx/lazy_container.ex:73: anonymous fn/3 in Nx.LazyContainer.Map.traverse/3
(elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(elixir 1.17.2) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(nx 0.9.2) lib/nx/lazy_container.ex:72: Nx.LazyContainer.Map.traverse/3
(nx 0.9.2) lib/nx/defn.ex:339: Nx.Defn.compile_flatten/5
(nx 0.9.2) lib/nx/defn.ex:331: anonymous fn/4 in Nx.Defn.compile/3
#cell:ti265afq7l6ocfgv:9: (file)
Hi all, not sure if this is the right place to post.
On chapter 6, Deep learning, page 128, the snippet uses uniform_split/2 which is undefined.
I checked various versions of Nx
Nx.Random.uniform_split(new_key, shape: {})
|> NeuralNetwork.predict(w1, b2, w2, b2)
