Vlahunter
Hi, i wanted to create a post and get some feedback regarding elixir ecosystem and a project that is on the way.
The short story
My boss assigned me to research around the ecosystem landscape in order to see if a specific ecosystem would help us create robust services in the edge. So far the only other language i am researching around is Golang
About me
I mainly use Node.js but also i use Golang for creating tools/scripts and whatever is needed that i need to build fast and light weight, i am not an Elixir devloper, i just happened to start learning some of the concepts this ecosystem has some months ago and they got me excited enough to have it in the top of the list before i start this research.
The concept
The main idea is not new, i should be able to connect to multiple different controllers (Modbus TCP/RTU), collect the data and either persist or push to a Message Queue. As i imagine, i would have multiple different server applications near the Field, communicating data in one way or another on a Centralized Server. Of course if possible some of the Data Processing could take place in the Field.
[Think of an X amount of controllers connected to a PC-Router per Physical Place]
Proposed Setup in the Field
- Industrial Grade Router
- Industrial Mini-PC (probably with at least 8 GB of RAM)
The Problems
Some things that need to be considered while researching on this topic are the following
- Being able to run in a lightweight manner
- Being able to change configurations or update parts of the app on the fly
- Fault tolerance as much as possible considering that there are a lot of points of failure (network, legacy systems, etc)
Some Golang Findings
- Seems that there is a maturing Ecosystem around the Edge Computing like (EdgeXFoundry)[https://www.edgexfoundry.org/]
- Fast to Develop
- Runs Lightweight
Lessons from the Past
In the past i have built a similar solution as a prototype that used Node.js, Mosquitto and TimeScaleDB but seems that in this setup it would be tough to scale (as far as my DevOps is explaining
)
Main Question
Do you think that the Elixir Ecosystem could possibly offer an advantage on a project like this ? Have you heard of similar Industrial Grade projects run on Elixir/Erlang/BEAM ? Any information would be useful so i could have a more complete picture for this Ecosystem when i will need to present it to my Boss.
Thanks in Advance
PS
This was my first post so please forgive me in case i use the wrong Category. If you need any further clarification, please let me know.
Trending in Discussions
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming











Showing Posts 51 to 42- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
Vlahunter
Yes certainly the MQTT protocol seems a good fit from the data collection onward. I was not aware that these parts were also OS so that means I need to keep digging. Of course if the value return is great, it would be easier for me to discuss this with my boss and communicate with them.
Sebb
Neuron and the modbus plugin are open source. The broker is open source also. If the main effort of your application is to aggregate modbus data from many nodes, it is all there already. All the extra processing you might have to do is easily done with the rule engine or just by an MQTT client.
Concerning message queues: If MQTT is enough for your demands, use it. Very easy, lightweight, robust.
Paid support from EMQX will greatly speed up development and should be very efficient.
I’d definitely write an inquiry to EMQX and see what they think how their product fits with your needs (while it seems like a good fit, I can’t judge)
Vlahunter
I have already found this during my research but ofc it is a commercial product so as a first part of this evaluation it is a no-go , on a later stage maybe my boss would evaluate this. By the way although never used EMQ Broker, I heard very nice feedback on its performance and ease of use (talking for the open source one)
Sebb
Do you know EMQX Neuron? Have a look: EMQX Neuron - Industrial IoT Gateway & Protocol Converter
wanton7
I don’t know that much about ZRAM. I’ve used it in some computers and I’ve read some companies using it in their servers to have more memory available for their server apps. If you use 50% for ZRAM, It basically uses half of your RAM for compressed memory swap. So it works like a in-memory compressed swap file. I think Chrome OS for example uses 50% ZRAM by default.
Vlahunter
That is a fantastic pick of insight! Thanks for sharing your experience.
Sebb
We have a similar application. While the servers in the field have very low load, we also have a central system in the cloud which has to handle all systems. To test the central system I have specs what the maximum load can be (need to know bandwidth, max burst rate, burst frequency). Then frames (in your case modbus-RTU-frames) are generated according to the specs, containing some random dummy data and a sequence number.
Expect: central system gets all frames (if needed: also in correct order).
Erlang is very good for this kind of stuff, will handle more than you think.
Vlahunter
The Heavy Calculations will be very few if any. In a general concept, i would need to parse the data, and then either persist or push to a local MQ (RabbitMQ/NATS/Verne/ still in research)
For that one i cannot really give an educated answer sadly.
Thats the first test that i will be conducting after i will have the app working as a prototype and my simulated PLCs running.
That sounds exacty as a fantastic use case then.
Sebb
Do you have to perform any heavy calculations on the data coming in or just, parse into JSON or whatever, store, put into message-queue?
What is the bandwidth of the connected nodes? Is 38400bps correct?
What is the worst case burst you have to expect?
OTP is made for that.
Having a GenServer for each connected client, makes it easy and robust.
Implementing that with threads or OS-processes gives you some more headaches.
Vlahunter
This seems as i suppose one of the selling points. In my case this will literally be the first important step to move forward.
To be honest i have never heard or used it. Would not that stress the machine in some ways? or affect the systems longevity? Sorry for asking something like that, i am ignorant on this subject.