Hi guys, I have a problem that I really don’t know where to start.
I want to create a program with the elixir language in which I generate two matrices of 1 million random data, save each matrix in a different file and finally generate another one as a result of its multiplication. How can i start I am new with this language and I propose things beyond the normal. I want to use my new powerful server that I just bought. help me please.
Elixir is not the best fit for number crunching, but You might have a look at Matrex…
3 Likes
You are technically correct there (the best kind of correct 
) however that’s not the whole story. I haven’t run benchmarks, but I’d expect Elixir and Python to be not too far apart in number crunching abilities. Python is popular for data science and scientific computing not because of its own impressive numerical performance, but because it has well-known libraries that help to organize matrix manipulation pipelines but delegates the actual numerical computation to lower-level native code.
I’d love to see Elixir make headway into this matrix computation domain because it has superior process management and fault tolerance over Python. Matrex is a great example of delegating the number crunching to lower-level native code, just like Python does.
3 Likes
Can you elaborate on the use case?
For personal educational issues I don’t want to use another language just elixir; I want to create 3 matrices, the first and second matrix save 1 million x 1 million floating random numbers … matrix I want to save them in different txt files.
Then call those two files and multiply the numbers to generate another matrix of 1 million x 1 million results and save it in a different txt.
School, I’d venture 
For such large matrices, you may not be able to fit it all in memory In the very least, I expect you’d want to serialize the multiplications rather than doing many 1Mx1M multiplications at the same time.
You can split a matrix multiplication by dividing each matric into smaller pieces. If you are multiplying A with B, you would only need to keep one piece of A and one piece of B in memory at any one time.
So the next issue, I guess, is finding a way to get these pieces quickly to and from disk. In elixir, I guess the answer to that would be using ETS to store them on disk.
Once you have solved this, the rest should be straightforward.
For an optimized approch you would probably be looking at a GPU based system rather than any particular language, and keep both A and B in memory until completion.
From what I remember from my undergrad days matrix multiplication requires O(1) random index access into the matrix. Tuples have that property and you could model your matrices as tuples of tuples, but my guess is it would not perform very well. There is also a common transformation of an MxN matrix into an 1xM*N array, and you could model that using tuples which have the desired O(1) random access property if not for the fact that tuples only go up to 16.7M elements- a far cry from 1 billion elements you need it to (not taking into account sparse matrix modeling of course).
Long story short- as others here already said you’re probably better of delegating matrix multiplication to a specialized implementation in another language (like matrex which is linked above, or fortran+lapack, python+numpy etc.)
