wanton7

wanton7

Optimal Azure AI translator API call batching

Can someone give ideas how could I create optimal batches for Azure AI translator calls?

Limits are max 50000 characters max per batch and max 1000 strings per batch. Then there can be x amount destination languages that multiplies characters per language. So string with 100 characters will take space of 200 characters example if you use two destination languages. If you have one string with length 50000 you can only have destination language. These destination languages are defined per batch. So example if I had 6 destination languages I could split them to two batches of 3 languages each and so on.

Marked As Solved

dimitarvp

dimitarvp

So maybe this?

defmodule AzureTranslate do
  def demo() do
    data = [
      {"text 1", 16_000, ~w(en ja za ru)},
      {"text 2", 25_000, ~w(de es)},
      {"text 3", 10_000, ~w(bg ru de sp es za)}
    ]

    requests_for_multiple_texts(50_000, data)
  end

  def requests_for_multiple_texts(max_request_length, texts) do
    texts
    |> Enum.map(fn {text_key, text_length, languages} ->
      {text_key, requests_for_single_text_via_reduce(max_request_length, text_length, languages)}
    end)
  end

  def requests_for_single_text_via_chunk(max_request_length, text_length, languages) do
    languages
    |> Enum.chunk_while(
      {max_request_length, []},
      fn lang, {remaining_bytes, chunk} ->
        if remaining_bytes >= text_length do
          {:cont, {remaining_bytes - text_length, [{text_length, lang} | chunk]}}
        else
          {:cont, Enum.reverse(chunk), {max_request_length - text_length, [{text_length, lang}]}}
        end
      end,
      fn {_remaining_bytes, list} ->
        {:cont, Enum.reverse(list), {}}
      end
    )
  end

  def requests_for_single_text_via_reduce(max_request_length, text_length, languages) do
    {_remaining_bytes, chunk, final_result} =
      languages
      |> Enum.reduce(
        {max_request_length, [], []},
        fn lang, {remaining_bytes, chunk, final_result} ->
          if remaining_bytes >= text_length do
            {remaining_bytes - text_length, [{text_length, lang} | chunk], final_result}
          else
            {max_request_length - text_length, [{text_length, lang}],
             [Enum.reverse(chunk) | final_result]}
          end
        end
      )

    Enum.reverse([Enum.reverse(chunk) | final_result])
  end

  def benchmark() do
    Benchee.run(%{
      "Enum.chunk_while" => fn ->
        requests_for_single_text_via_chunk(50_000, 16_000, ~w(de fr es it pt))
      end,
      "Enum.reduce" => fn ->
        requests_for_single_text_via_reduce(50_000, 16_000, ~w(de fr es it pt))
      end
    })
  end
end

Included:

  • Two alternative implementations of an algorithm to get a singular text and break it apart on several requests;
  • One function that uses the faster of both implementations (requests_for_multiple_texts);
  • Related to above: run the benchmark function (make a small new Elixir project and just include benchee in it, or use Mix.install in a single .exs file ) and you’ll see for yourself which of both is faster. SPOILERS: it’s the one using Enum.reduce. I could probably make an even faster one but wasn’t in the mood to make one that uses only pure recursion and nothing else;
  • Demo data + demo function that, ahem, demonstrates its correctness with an example.

The output might be slightly cryptic, so explaining it:

You get a list of tuples: first element is the text key (or the text itself), the second one is a list of lists of tuples.

The list of tuples represent a single request which might have e.g. same text with 5 languages. Each text here is represented by its length, not the key / text itself. Got tired and didn’t want to bloat the functions further with one more piece of data. :person_shrugging:

The list that encompasses that list of tuples is the list of requests that must be made for this single text to get fully translated.

Not sure if the code is good but it gets the job done. With the demo data above the result is:

[
  {"text 1", [[{16000, "en"}, {16000, "ja"}, {16000, "za"}], [{16000, "ru"}]]},
  {"text 2", [[{25000, "de"}, {25000, "es"}]]},
  {"text 3",
   [
     [{10000, "bg"}, {10000, "ru"}, {10000, "de"}, {10000, "sp"}, {10000, "es"}],
     [{10000, "za"}]
   ]}
]

…which means:

  • "text 1" gets to do 2 requests: one with 3 languages and one with 1 language;
  • "text 2" gets to do 1 request with 2 languages;
  • "text 3" gets to do 2 requests: one with 5 languages and one with 1 language.

Where Next?

Popular in Questions Top

New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
yawaramin
In the Dialyzer docs ( dialyzer — OTP 29.0.2 (dialyzer 6.0.1) ), there is a way to turn off a specific warning for a function: -dialyzer...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31194 112
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31307 143
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54250 245
New

We're in Beta

About us Mission Statement