djaouen

djaouen

Axon question: LSTM with Dropout

I would like to do something like the following:

input = Axon.input("input", shape: {prediction_days, 1})

model =
  input
  |> Axon.lstm(50)
  |> Axon.dropout(rate: 0.2)
  |> Axon.lstm(50)
  |> Axon.dropout(rate: 0.2)
  |> Axon.lstm(50)
  |> Axon.dropout(rate: 0.2)
  |> Axon.dense(1)

But when I try this out in LiveBook, I get the following error:

** (FunctionClauseError) no function clause matching in Axon.dropout/2    
    
    The following arguments were given to Axon.dropout/2:
    
        # 1
        {#Axon<
           inputs: %{"input" => {60, 1}}
           outputs: "lstm_1_output_sequence"
           nodes: 6
         >,
         {#Axon<
            inputs: %{"input" => {60, 1}}
            outputs: "lstm_1_c_hidden_state"
            nodes: 6
          >,
          #Axon<
            inputs: %{"input" => {60, 1}}
            outputs: "lstm_1_h_hidden_state"
            nodes: 6
          >}}
    
        # 2
        [rate: 0.2]
    
    Attempted function clauses (showing 1 out of 1):
    
        def dropout(%Axon{} = x, opts)

The Python code I am trying to replicate looks like this:

model = Sequential()

model.add(LSTM(units=50, return_sequences=True, input_shape=[x_train.shape[1],  1]))
model.add(Dropout(0.2))
model.add(LSTM(units=50, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=50))
model.add(Dropout(0.2))
model.add(Dense(units=1))

model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(x_train, y_train, epochs=25, batch_size=32)

Any help? Thanks in advance!

Most Liked

seanmor5

seanmor5

Author of Genetic Algorithms in Elixir

Axon’s LSTM (and all other RNNs) return tuples of {seq, state}. If you want to apply dropout you need to extract the seq element and apply dropout to that from the tuple.

I would advise against this though, it’s usually discouraged to apply normal dropout to RNN outputs because it can hinder learning. You’d need to use recurrent dropout to achieve the regularization effect you’re looking for, but Axon does not support this.

seanmor5

seanmor5

Author of Genetic Algorithms in Elixir

No problem, and don’t feel bad for having questions! It’s how we all learn.

There are some RNN examples in the Axon repo if you want to check those out for some additional help!

Where Next?

Popular in Questions Top

skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement