evolutionengine

evolutionengine

Hi guys, sorry if this sounds like a silly question, I am a beginner in general. I would like to know how are you implementing machine learning into your apps. A couple of scenarios I could think of are -

  1. Interact with a ML service in python over REST or similar.
  2. Calling python scripts using erlport or thrift.

Are there any suggestions or other approaches ?

I am trying to build a Phoenix app and planning to use ML to build custom user feed, recommendations and spam filter.

Thanks.

Showing Posts 1 to 9

ityonemo

ityonemo

here’s one guide. I’m going to look for the other one that I saw:

ityonemo

ityonemo

above one only works if you are using tensorflow. this one is a more general python interop story:

evolutionengine

evolutionengine OP

Thanks, checking it out!

patrickdm

patrickdm

Hello and welcome, there are a few posts in the forum that you could find interesting:

Hth

ityonemo

ityonemo

just a caution, unless I’m mistaken I would say except for tensorflex, the others are still experimental and not ready for prod deployment.

sym_num

sym_num

Thank you for introducing my DeepPipe. Currently I am developing DeepPipe2. DP1 was slow and wrong. DP2 utilizes GPU for practical speed. I have finished implementing CNN. I’m also working on RNN for natural language processing.

leifericf

leifericf

Another approach to incorporate machine learning components into Elixir applications is to interoperate with other languages such as Python, e.g. through ports, NIFs, etc.

Erlang/OTP gives us other means to interface with- or incorporate external code, each with varying degrees of ease and robustness guarantees. See the Erlang Interoperability Tutorial for details.

Chapter 15 (Interfacing Techniques) of Programming Erlang by Joe Armstrong also contains some valuable information on this topic. Pages 233 through 242.

Here is an excellent article by @alvises which shows how to do this through Elixir. Elixir Mix ep. #91 goes into this article and related topics in more depth together with its author.

Erlang/OTP has excellent support for interoperability, which is one of its strengths. It might make more sense to leverage the existing Python libraries for machine learning in this way.

You can also develop your machine learning model as a separate application using Python and expose it through and API. TensorFlow comes with its own mechanisms for model serving. Then you can make use said API within your Elixir application and create an Elixir wrapper if needed. One benefit of using an Elixir port instead is that you can put your machine learning model behind a GenServer and supervise it as you would any other process in Elixir.

Another alternative is to develop and train your machine learning model using Python, and then bring the resulting model artefact into Elixir. You can serve and expose the model through an API written in Elixir. Have a look at out Tensorflex by @anshuman23 for inspiration on how this can be accomplished for neural network models through TensorFlow bindings.

Yet another approach which I have been thinking about lately is to embed the Python interpreter into Erlang/OTP or Elixir, and then use Elixir’s excellent meta-programming facilities (e.g. macros) to create a domain-specific language (DSL) specifically for machine learning. Supervision trees and GenServer could be used to spawn new processes representing instances of the Python interpreter to “cross-compile” the Elixir-based DSL to Python code and execute it. I found an old thread in the Erlang mailing list on this topic and an abandoned project on GitHub. This would likely require a considerable amount of effort, but could be an interesting way to vastly expand the libraries available in Elixir by essentially opening the door to all Python packages out there. Using ports is simpler.

Lastly, here are two related forum threads which contain some relevant information:

evolutionengine

evolutionengine OP

Thanks for the detailed response.

leifericf

leifericf

You’re very welcome, my friend!

Did you try some of those approaches?

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4663 61218 595
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
515 12931 101
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
gtcode
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New
Ganna
Hi everyone, I have 8 years of production experience with Erlang/OTP and I’m just starting to get to know Elixir. What would you learn f...
New
AstonJ
A remark @Garrison made… ..has already inspired a thread for the technical side of reclaiming our internet, but there is of course anot...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews