IoT vehicle tracking PoC, Elixir/Erlang on FreeBSD/Linux in Azure

Anyone?

The forum slows down on weekends, real-life and such. ^.^

Either way, Elixir/Erlang is perfectly suited to this, however more information. You say you want to handle IoT so what kind of interface is it, do they connect over standard web interfaces, a raw TCP socket, etc… etc…? More details please.

The IoT solution will be for new products so the current protocol is not really important (there are a couple of them and this is an opportunity to do some unification work there as well). The IoT device pumps telemetry, GPS and other information information over data networks. Some networks may be extremely constrained like Edge or GPRS. Compact packaging of payloads is important, I thought of ASN.1 or ProtoBuff instead of JSON to package information. The back end, should be able to send update payloads to the IoT devices to update its firmware. The connections will probably be raw data (packed/unpacked with ASN.1 or ProtoBuff codecs) over TCP, but I am coming to the community for advice, so I really want to hear suggestions and have an open mind. I just want to do what is best for the customer.

I am glad that you think a BEAM based back end solution will work well. I even though of having a FreeBSD\Linux device (like a Raspberry Pi or Beagle Board) with Erlang/Elixir on the vehicle high availability and updates without downtime, but let’s not get ahead of yourselves.

Also, it’s good netiquette to allow at least 24 hours before bumping threads @axler8tor

I don’t have experience with IoT projects but I did stumble upon this tutorial https://monterail.com/blog/2016/iot-with-elixir-and-coap-part-1-example-on-how-to-easily-prototype-and-build-an-iot-platform - they describe the whole process and have everything on github, so it might be a good starting point.

As for one and a half billion events per month if they are requests and not some fat computations and are evenly distributed it should be around 600 per second which should be fine, here’s one benchmark that tried to spam plug and phoenix systems Phoenix Showdown Comparative Benchmarks @ Rackspace · GitHub

Thanks for the link.

The distribution of events is not uniform for motor vehicles, the bulk of the events are generated during morning and afternoon rush hours. For trucks, the events are more uniformly distributed because they tend to be on the move for more hours of during the day. I am not sure about the volumes, but I will guess that 80% of the events are generated during 40 peak periods per month. The total duration is about 120 hours which puts us at a cool 10_000_000 events per hour during peak times. I have to get the real numbers.

There are alerts generated for some products, like geo-fencing triggers or detecting abnormal driving behaviour. These can be quite computation intensive. Then there are products that suggest alternative routes for trucks based on road conditions in some pretty remote places.

So there are many small things that have to be handled concurrently - sounds like a fun task to use Elixir / BEAM, you’ll probably not get away with a single server though

I’d cover the basics, stress test the whole thing and then decide if I’m ok with the performance of the more expensive tasks without resolving to ports and something like Go or C to handle them. There might be someone able to give you more details on how to handle that.

Just out of curiosity: the current stack that you describe is actually not that bad for this kind of job, is there a problem with it that would justify a complete rewrite?

The event engine sounds like a perfect fit for GenStage/Flow. Here is an article about Discord using it for Overwatch with 25,000 concurrent users and 15k bursts of messages. They had a small buffer problem with where they had to limit their in-progress runs to 100, and they were still able to handle it with minimal shedding and taking advantage of all of the back pressure mechanisms that GenStage offers. I think with a different use case you should be able to do it “easily” with 100% durability.

Also I think “exposing the back end to advanced analytics engines” is an easy and perfect fit for Elixir apps in an umbrella project. This is one of the real strengths IMO of Elixir/Erlang. It’s like “microservices” but a more natural fit with OTP and supervision (which makes me usually avoid using the loaded term microservices).

ok so here is how i would look at it. I work on some similar solutions for a car manufacturer. Sadly we do not use Elixir, but i do use Elixir a lot at home.

1. Gateway : Elixir will shine here. It will mainly depends of your transport layer and your goals for the Service, but should be completely doable and not too hard.

2. Event processing. That is the part GenStage could do, but there would need a work on top of it to be done. I am working on that type of things to try to see if we need to update GenStage to make it better recently. Doable, but GenStage is still a bit “low level API” for that, so it will need a bit of work.

3. Storage facility : that depends a lot on your needs. need more data here.

4. exposing the backend : not really hard either.

Something i would try, especially if you can deal with the latency it adds, is to keep Kafka between some of these backend. Kafka adds resiliency and can absorb some things when load shedding may not be acceptable. It is not always the good solution but it helps… and it make is easy to plug other backend on the firehose later.

Feel free to contact me if you want more ideas

Elixir will be perfect I am 50 % into setting up a nerves project for handling the events from a smart city project.
basically its a net of LORA gataways with a nerves/Raspery pi to handle the events of some 13 000 units passing it to a centos Elixir/RIAK KV system. Then a phoenix app to show the visuals.

Look into nerves if you need gatways/substations that need to interact with hardware. its brilliant.
RIAK KV was chosen because we have alot of small data that we cant afford to loose.