Deep Learning library with GPU(CUDA/cuBLAS)

[sym_num]

In CIFAR’s code it is by SGD method. I presume that there is stagnation due to saddle points. I am improving by the Momentum method.

[kip]

I only understand about 1% of what you are doing but I think this work is incredibly valuable to the community. Please keep going!

[sym_num]

Thank you very much.

[sym_num]

I gave up CIFAR10 and tested DP2 with a different dataset. The dataset is Fashon-MNIST. Since the file has the same structure as MNIST, the code is simple. The correct answer rate is also good. I use this dataset to check and improve CNN functionality. DP2 will be released at the end of June 2020.

zalandoresearch/fashion-mnist

[sym_num]

Although it is still incomplete, I registered DP2 on Hex. I will improve it further.

deeppipe2

[sym_num]

sorry. There was a bug. The Hex version of DP2 could not find the nifs.so file. I fixed it in ver1.1.2.

deeppipe2

[sym_num]

I posted to medium.

CNN for Deeppipe2

[zacky1972]

I found DeepPipe2 works well on TITAN RTX and Jetson Nano, though the latter requires much execution time.

[sym_num]

I also registered Elxlog on the Hex. Elxlog is a Prolog interpreter and compiler written in Elixir. I am thinking of using Prolog, which is classical artificial intelligence, and Deep-Learning, which is modern artificial intelligence, in combination.

elxlog

[sym_num]

I implemented the Xavier method and the He method in ver1.1.6 for generating initial values. As a result, I obtained a Fashion-MNIST accuracy rate 86% for the two-layer CNN. I think CNN of DeepPipe2 is working.

Then I will work on improving execution speed. Convolution of DP2 is slow. It does not utilize GPU resources on executing multiple channels. The implementation of convolution was naive in the prototype. Needs improvement.

deeppipe2

[sym_num]

In ver1.1.7, I improved the speed of convolution and pooling. As a result, DP2 is approximately 4.5 times faster with the Fashion-MNIST dataset.

It will be in time for the official release on June 30, 2020. I am relieved.

deeppipe2

[sym_num]

In ver1.1.8, I added Adam and RMSprop optimizer. By using this, DP2 can be trained on the CIFAR10 dataset.

DP2 opens the way to deep-learning with practical training time in Elixir.

deeppipe2

[sym_num]

I was able to release ver1.2 two weeks ahead of schedule. Thank you everyone for your support.

I’ll rest for a while. I plan to work on RNN after that.

deeppipe2

[sym_num]

I posted the development history on Medium.

CNN Deep-Learning with GPU in Elixir

[sym_num]

I’m making videos of DP2 explanation.

Introduction of DeepPipe2 Deep-Learning library in Elixir (0).

Introduction of DeepPipe2 Deep-Learning library in Elixir (1)

[ityonemo]

Your pronunciation of stochastic gradient descent is good! Also, this looks fantastic.

[sym_num]

Thank you very much.

[sym_num]

I am working on natural language processing with RNN. A simple example is now working.

I plan to complete the RNN,LSTM by the end of December 2020.

[IRLeif]

Great work, @sym_num!

ありがとうございます。
がんばってください。

[sym_num]

Thank you very much.