Ok, so I have an app that is currently utilizing 3 ML models. One is called Spacy, I have a feeling this is the least likely to be possible, since I think it’s a lot of Python code wrapped around a model, rather than just a model. The other two are EasyNMT, using OpusMT’s model. And Simalign, which is currently using Bert, specifically bert-base-multilingual-cased.
Despite the fact that i’ve been working with these models for about 6 months, I’m still not always 100% sure I understand the delineation between Python-space and model-space. But correct me if I’m wrong here.
Simalign is ~230 LOC wrapped around BERT. Is there any reason I can’t port that 230 LOC to Elixir, and wrap BERT from Bumblebee, and cut out Python all together? It looks like the code is using Bert mostly for 1. Word embeddings. 2. Tokenization
EasyNMT is about 500 LOC, but I don’t think my code relies on all of it. I’m only calling one main translate function. Would there be any way to pull this into Elixir as well, or am I misunderstanding how Bumblebee fits into all of this?
FYI, my goal here is to minimize my Python footprint for a few reasons. I much prefer, and am much more comfortable with Elixir(both in my ability to write it, and my ability to trust it to do what I need, when I need it, and do it well). I’ve been working on this app for about 9 months, and i’ve not enjoyed the Python piece the whole time, now that I’m coming to deploy it, the Python part has been even more of a nightmare, and even on a GPU instance, it’s quite slow(surprisingly it’s faster on my M1, than a CUDA GPU, except for the EasyNMT piece, so I’m thinking if possible, just bring it all into Elixir, except Spacy, I think that would be a larger lift, but let me know if I’m wrong about that one.)
