djaouen
Axon question: LSTM with Dropout
I would like to do something like the following:
input = Axon.input("input", shape: {prediction_days, 1})
model =
input
|> Axon.lstm(50)
|> Axon.dropout(rate: 0.2)
|> Axon.lstm(50)
|> Axon.dropout(rate: 0.2)
|> Axon.lstm(50)
|> Axon.dropout(rate: 0.2)
|> Axon.dense(1)
But when I try this out in LiveBook, I get the following error:
** (FunctionClauseError) no function clause matching in Axon.dropout/2
The following arguments were given to Axon.dropout/2:
# 1
{#Axon<
inputs: %{"input" => {60, 1}}
outputs: "lstm_1_output_sequence"
nodes: 6
>,
{#Axon<
inputs: %{"input" => {60, 1}}
outputs: "lstm_1_c_hidden_state"
nodes: 6
>,
#Axon<
inputs: %{"input" => {60, 1}}
outputs: "lstm_1_h_hidden_state"
nodes: 6
>}}
# 2
[rate: 0.2]
Attempted function clauses (showing 1 out of 1):
def dropout(%Axon{} = x, opts)
The Python code I am trying to replicate looks like this:
model = Sequential()
model.add(LSTM(units=50, return_sequences=True, input_shape=[x_train.shape[1], 1]))
model.add(Dropout(0.2))
model.add(LSTM(units=50, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=50))
model.add(Dropout(0.2))
model.add(Dense(units=1))
model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(x_train, y_train, epochs=25, batch_size=32)
Any help? Thanks in advance!
Most Liked
seanmor5
Axon’s LSTM (and all other RNNs) return tuples of {seq, state}. If you want to apply dropout you need to extract the seq element and apply dropout to that from the tuple.
I would advise against this though, it’s usually discouraged to apply normal dropout to RNN outputs because it can hinder learning. You’d need to use recurrent dropout to achieve the regularization effect you’re looking for, but Axon does not support this.
seanmor5
No problem, and don’t feel bad for having questions! It’s how we all learn.
There are some RNN examples in the Axon repo if you want to check those out for some additional help!
Last Post!
djaouen
Nevermind, I updated EXLA to use GitHub (instead of hex.pm) and the error went away on its own. Cheers!
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









