Vlahunter

Vlahunter

Hi, i wanted to create a post and get some feedback regarding elixir ecosystem and a project that is on the way.
The short story
My boss assigned me to research around the ecosystem landscape in order to see if a specific ecosystem would help us create robust services in the edge. So far the only other language i am researching around is Golang

About me
I mainly use Node.js but also i use Golang for creating tools/scripts and whatever is needed that i need to build fast and light weight, i am not an Elixir devloper, i just happened to start learning some of the concepts this ecosystem has some months ago and they got me excited enough to have it in the top of the list before i start this research.

The concept
The main idea is not new, i should be able to connect to multiple different controllers (Modbus TCP/RTU), collect the data and either persist or push to a Message Queue. As i imagine, i would have multiple different server applications near the Field, communicating data in one way or another on a Centralized Server. Of course if possible some of the Data Processing could take place in the Field.
[Think of an X amount of controllers connected to a PC-Router per Physical Place]

Proposed Setup in the Field

  • Industrial Grade Router
  • Industrial Mini-PC (probably with at least 8 GB of RAM)

The Problems
Some things that need to be considered while researching on this topic are the following

  • Being able to run in a lightweight manner
  • Being able to change configurations or update parts of the app on the fly
  • Fault tolerance as much as possible considering that there are a lot of points of failure (network, legacy systems, etc)

Some Golang Findings

  • Seems that there is a maturing Ecosystem around the Edge Computing like (EdgeXFoundry)[https://www.edgexfoundry.org/]
  • Fast to Develop
  • Runs Lightweight

Lessons from the Past
In the past i have built a similar solution as a prototype that used Node.js, Mosquitto and TimeScaleDB but seems that in this setup it would be tough to scale (as far as my DevOps is explaining :stuck_out_tongue: )

Main Question
Do you think that the Elixir Ecosystem could possibly offer an advantage on a project like this ? Have you heard of similar Industrial Grade projects run on Elixir/Erlang/BEAM ? Any information would be useful so i could have a more complete picture for this Ecosystem when i will need to present it to my Boss.
Thanks in Advance

PS
This was my first post so please forgive me in case i use the wrong Category. If you need any further clarification, please let me know.

Showing Posts 51 to 42

Vlahunter

Vlahunter OP

Yes certainly the MQTT protocol seems a good fit from the data collection onward. I was not aware that these parts were also OS so that means I need to keep digging. Of course if the value return is great, it would be easier for me to discuss this with my boss and communicate with them.

Sebb

Sebb

Neuron and the modbus plugin are open source. The broker is open source also. If the main effort of your application is to aggregate modbus data from many nodes, it is all there already. All the extra processing you might have to do is easily done with the rule engine or just by an MQTT client.

Concerning message queues: If MQTT is enough for your demands, use it. Very easy, lightweight, robust.

Paid support from EMQX will greatly speed up development and should be very efficient.

I’d definitely write an inquiry to EMQX and see what they think how their product fits with your needs (while it seems like a good fit, I can’t judge)

Vlahunter

Vlahunter OP

I have already found this during my research but ofc it is a commercial product so as a first part of this evaluation it is a no-go , on a later stage maybe my boss would evaluate this. By the way although never used EMQ Broker, I heard very nice feedback on its performance and ease of use (talking for the open source one)

Sebb

Sebb

wanton7

wanton7

I don’t know that much about ZRAM. I’ve used it in some computers and I’ve read some companies using it in their servers to have more memory available for their server apps. If you use 50% for ZRAM, It basically uses half of your RAM for compressed memory swap. So it works like a in-memory compressed swap file. I think Chrome OS for example uses 50% ZRAM by default.

Vlahunter

Vlahunter OP

That is a fantastic pick of insight! Thanks for sharing your experience.

Sebb

Sebb

We have a similar application. While the servers in the field have very low load, we also have a central system in the cloud which has to handle all systems. To test the central system I have specs what the maximum load can be (need to know bandwidth, max burst rate, burst frequency). Then frames (in your case modbus-RTU-frames) are generated according to the specs, containing some random dummy data and a sequence number.

Expect: central system gets all frames (if needed: also in correct order).

Erlang is very good for this kind of stuff, will handle more than you think.

Vlahunter

Vlahunter OP

The Heavy Calculations will be very few if any. In a general concept, i would need to parse the data, and then either persist or push to a local MQ (RabbitMQ/NATS/Verne/ still in research)

For that one i cannot really give an educated answer sadly.

Thats the first test that i will be conducting after i will have the app working as a prototype and my simulated PLCs running.

That sounds exacty as a fantastic use case then.

Sebb

Sebb

Do you have to perform any heavy calculations on the data coming in or just, parse into JSON or whatever, store, put into message-queue?
What is the bandwidth of the connected nodes? Is 38400bps correct?
What is the worst case burst you have to expect?

OTP is made for that.
Having a GenServer for each connected client, makes it easy and robust.
Implementing that with threads or OS-processes gives you some more headaches.

Vlahunter

Vlahunter OP

This seems as i suppose one of the selling points. In my case this will literally be the first important step to move forward.

To be honest i have never heard or used it. Would not that stress the machine in some ways? or affect the systems longevity? Sorry for asking something like that, i am ignorant on this subject.

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
AstonJ
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
maennchen
:warning: Security advisory: Decimal DoS vulnerability A vulnerability has been published for decimal where very large exponents can cau...
New
Null-logic-0
What IDE or editor are you using for Elixir development? Personally, I use Zed, and I really like it, but sometimes I wish there were a ...
New

Other Trending Topics Top

marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews