Matrex - A blazing fast matrix library for Elixir/Erlang with C implementation using CBLAS (Machine Learning)

versilov

Could not wait for the missing Elixir ML libraries to appear, so, I wrote one myself, taking https://github.com/sdwolfz/exlearn as a foundation.

The name is Matrex and it’s super-fast (compared to pure Elixir implementations) matrix manipulation lib.

Critical code is written in C using CBLAS subroutines and linked as Erlang NIFs.
It’s about 50-5000 times faster, than pure Elixir.

In the repo you will find MathLab fmincg() ported to Elixir with the help of the library
and logistic regression MNIST digits recognition exercise from Andrew Ng’s ML course implemented in Elixir (15 times faster, than Octave implentation).

It can be used like this:

y = Matrex.load("y.mtx")

j =
      y
      |> Matrex.dot_tn(Matrex.apply(h, :log), -1)
      |> Matrex.substract(
        Matrex.dot_tn(
          Matrex.substract(1, y),
          Matrex.apply(Matrex.substract(1, h), :log)
        )
      )
      |> Matrex.scalar()
      |> (fn
            NaN -> NaN
            x -> x / m + regularization
          end).()

Or like this:

    import Matrex.Operators
   
    h = sigmoid(x * theta)
    l = ones(size(theta)) |> set(1, 1, 0.0)

    j = (-t(y) * log(h) - t(1 - y) * log(1 - h) + lambda / 2 * t(l) * pow2(theta)) / m

I’ve also created a Jupyter notebook with logistic regression algorithm in Elixir built with the help of this library.

Please, check Matrex on GitHub,
take a look at Matrex hex docs,
and tell me what you think of it.

25 comments

#library #machine-learning #matrex #data-science

60 8838 25

2020-01-20 11:26:32 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

chrismcg

This is great to see, nice work!

Post #1

grych

Creator of Drab

Man, this is so great!

Post #2

michalmuskala

This looks really good!

I wonder about the decision of keeping everything in a binary with sizes in front vs having a C struct and a nif resource. Wouldn’t the later be easier to handle and possibly also slightly faster on some operations?

Post #3

versilov

Michal,

I considered using NIF resource, but decided to stay with explicit binary for 3 reasons:

I don’t want to add unnecessary NIFs, when Elixir gives the same speed or operation is not time-critical.
Pattern matching for simple operations, like getting the size of the matirx.
There is an idea to implement Elixir versions of all the NIFs, so they would
work as fallback functions that work everywhere. E.g., when NIFs would fail to compile for some reasons. I don’t know if it’s worth the effort, but don’t want to cut off such possibilities.

And NIF resource leaves us only with , AFAIK?

%Matrex{} struct now works mostly for the purpose of implementing protocols and behaviours (Inspect, Enum and Access). It gives the opportunity to add more fields later, to distinguish matrices with
different element types and sizes, different matrix shapes and so on.

So, I believe, the current setup gives greater flexibility in future development.
But I did not dig deep into NIF resources, so, maybe I am missing something.

Post #4

michalmuskala

I guess you’re right.

Another approach I could see would be to have columns and rows directly in the struct. What do you think? This could allow users to do matrix.rows and matrix.cols or similar. You could possibly unpack those in the elixir layer and call the NIFs with 3 arguments instead of one.
I could imagine then the “data” field could be either a binary or a NIF resource depending on what’s more useful and the Elixir code would treat it as completely opaque.

Post #5

versilov

Yeah, thought about it also, but postponed for two reasons:

Did not want to hinder library release by rewriting the codebase I inherited from exlearn.
It gives only more clean access to some properties (matrix.rows instead of matrix[:rows]),
not the highest priority feature for now.

I definitely plan to experiment with this approach in the future to see how it would work in real life.

Thanks for the thorough feedback, I am amazed with the Elixir community:)

Post #6

OvermindDL1

This does look very awesome!

Have you thought about adding a kernel builder interface? I.E. you’d build up a command queue then ‘submit’ it to the BLAS backend to perform it all as quickly as possible (some have JIT-style engines that can take such a command queue and run it as optimized assembly over the dataset, potentially even to other hardware optimized for the purpose).

Post #7

versilov

Thank you!

Yes, I think what you’ve described is the second library we desperately need in Elixir to make it shine in machine learning.

Have a look at these two repos, cuda and neuro: sirin-tech · GitHub
They use kernels, defined in PTX ISA internal assembly language, to feed them into NVIDIA CUDA interface.
The only drawback is that they have zero documentation, as far as I know.

I plan to contact these guys and may be visit them in person to unite development efforts. I hope, these libs
need only good docs and some polishing, so we can get the next piece of Elixir ML puzzle faster:)

Post #8

PragTob

This looks great, thank you very much for your work!

Post #9

versilov

Thanks for all your praise!

As we move towards version 1.0, here come new features:

Heatmap of the matrix with Matrex.heatmap/3
This one is the coolest. Now you can monitor learning process and get insights from your matrices right inside terminal. Just have a look:

mnist8.png447×480 8.83 KB

mnist_sum.png436×512 11.7 KB

magic_square.png470×507 19.2 KB

logistic_regression.gif305×364 1.7 MB
reshape/3 and concat/2. Fast and convenient matrix creation from enumerable of elements or other matrices.
Here’s how you can load MNIST digits dataset and show a subset of the data with one line of code:

reshape_mnist1892×1388 82.7 KB
Other minor additions: normalize/1, resize/2, min_finite/1, max_finite/1, to_row/1, to_columns/1, list_of_rows/2.