Deep Learning library with GPU(CUDA/cuBLAS)

In CIFAR’s code it is by SGD method. I presume that there is stagnation due to saddle points. I am improving by the Momentum method.

1 Like

I only understand about 1% of what you are doing but I think this work is incredibly valuable to the community. Please keep going!

4 Likes

Thank you very much.

2 Likes

I gave up CIFAR10 and tested DP2 with a different dataset. The dataset is Fashon-MNIST. Since the file has the same structure as MNIST, the code is simple. The correct answer rate is also good. I use this dataset to check and improve CNN functionality. DP2 will be released at the end of June 2020.

5 Likes

Although it is still incomplete, I registered DP2 on Hex. I will improve it further.

5 Likes

sorry. There was a bug. The Hex version of DP2 could not find the nifs.so file. I fixed it in ver1.1.2.

2 Likes

I posted to medium.

4 Likes

I found DeepPipe2 works well on TITAN RTX and Jetson Nano, though the latter requires much execution time.

2 Likes

I also registered Elxlog on the Hex. Elxlog is a Prolog interpreter and compiler written in Elixir. I am thinking of using Prolog, which is classical artificial intelligence, and Deep-Learning, which is modern artificial intelligence, in combination.

7 Likes

I implemented the Xavier method and the He method in ver1.1.6 for generating initial values. As a result, I obtained a Fashion-MNIST accuracy rate 86% for the two-layer CNN. I think CNN of DeepPipe2 is working.

Then I will work on improving execution speed. Convolution of DP2 is slow. It does not utilize GPU resources on executing multiple channels. The implementation of convolution was naive in the prototype. Needs improvement.

8 Likes

In ver1.1.7, I improved the speed of convolution and pooling. As a result, DP2 is approximately 4.5 times faster with the Fashion-MNIST dataset.

It will be in time for the official release on June 30, 2020. I am relieved.

8 Likes

In ver1.1.8, I added Adam and RMSprop optimizer. By using this, DP2 can be trained on the CIFAR10 dataset.

DP2 opens the way to deep-learning with practical training time in Elixir.

8 Likes

I was able to release ver1.2 two weeks ahead of schedule. Thank you everyone for your support.

I’ll rest for a while. I plan to work on RNN after that.

6 Likes

I posted the development history on Medium.

6 Likes

I’m making videos of DP2 explanation.


8 Likes

Your pronunciation of stochastic gradient descent is good! Also, this looks fantastic.

2 Likes

Thank you very much.

1 Like

I am working on natural language processing with RNN. A simple example is now working.

I plan to complete the RNN,LSTM by the end of December 2020.bandicam 2020-08-02 17-57-48-960

5 Likes

Great work, @sym_num! :purple_heart:

ありがとうございます。
がんばってください。

2 Likes

Thank you very much.

1 Like