Hello everyone!! This is my first post in this forum, although I’ve been around long enough already to fall in love with this community and elixir/phoenix. I’m quite new to elixir and phoenix and recently I’ve started building a blockchain explorer with it (https://github.com/CityOfZion/neo-scan). It has been a tough path but I’m liking it :). I want now some feedback and suggestions to improve the application since I’m no expert. Also, I’ve some problems that I don’t fully understand. The project is an umbrella with three applications, the phoenix endpoint, the repo and a sync application that fecthes data from the blockchain and syncs the DB.
Question 1: Do I need to wrap each application under an server/API and communicate between them through messages? Right now I’m just calling functions from each other modules…
Question 2: My sync application has two workers, one that makes concurrent http requests to the blockchain node and stores raw data, and other that builds the DB based on the data previously stored. I want them to work in parallel, but so far I cant’ make it work, only one at a time. I’ve tried separating them in different applications but had the same result, the other worker just starts when the first one ends. What am I missing?
Question 3: To store a transaction in the DB it needs to fetch data from it through queries first (to link blocks/transactions), sometimes, when there’s too much data to be queried erlang throws a memory error (heap_alloc: Cannot allocate x bytes of memory of type “heap”). How to deal with this?
Question 4: in my home controller there’s a query for the last elements in the “blocks” and “transactions” table. Since each table has more than a million elements it is taking forever to do the sorting and retrieve. I’ve tried using indexes in descending order but the result was the same, though postgres writting time considerably increased. Am I missing something?
Sorry for the long post, I’ve been trying to sort everything on my on but now it felt like I could use some help. Any other tips and suggestions are appreciated!
Thanks!
Luciano S. Engel