wanton7
Optimal Azure AI translator API call batching
Can someone give ideas how could I create optimal batches for Azure AI translator calls?
Limits are max 50000 characters max per batch and max 1000 strings per batch. Then there can be x amount destination languages that multiplies characters per language. So string with 100 characters will take space of 200 characters example if you use two destination languages. If you have one string with length 50000 you can only have destination language. These destination languages are defined per batch. So example if I had 6 destination languages I could split them to two batches of 3 languages each and so on.
Marked As Solved
dimitarvp
So maybe this?
defmodule AzureTranslate do
def demo() do
data = [
{"text 1", 16_000, ~w(en ja za ru)},
{"text 2", 25_000, ~w(de es)},
{"text 3", 10_000, ~w(bg ru de sp es za)}
]
requests_for_multiple_texts(50_000, data)
end
def requests_for_multiple_texts(max_request_length, texts) do
texts
|> Enum.map(fn {text_key, text_length, languages} ->
{text_key, requests_for_single_text_via_reduce(max_request_length, text_length, languages)}
end)
end
def requests_for_single_text_via_chunk(max_request_length, text_length, languages) do
languages
|> Enum.chunk_while(
{max_request_length, []},
fn lang, {remaining_bytes, chunk} ->
if remaining_bytes >= text_length do
{:cont, {remaining_bytes - text_length, [{text_length, lang} | chunk]}}
else
{:cont, Enum.reverse(chunk), {max_request_length - text_length, [{text_length, lang}]}}
end
end,
fn {_remaining_bytes, list} ->
{:cont, Enum.reverse(list), {}}
end
)
end
def requests_for_single_text_via_reduce(max_request_length, text_length, languages) do
{_remaining_bytes, chunk, final_result} =
languages
|> Enum.reduce(
{max_request_length, [], []},
fn lang, {remaining_bytes, chunk, final_result} ->
if remaining_bytes >= text_length do
{remaining_bytes - text_length, [{text_length, lang} | chunk], final_result}
else
{max_request_length - text_length, [{text_length, lang}],
[Enum.reverse(chunk) | final_result]}
end
end
)
Enum.reverse([Enum.reverse(chunk) | final_result])
end
def benchmark() do
Benchee.run(%{
"Enum.chunk_while" => fn ->
requests_for_single_text_via_chunk(50_000, 16_000, ~w(de fr es it pt))
end,
"Enum.reduce" => fn ->
requests_for_single_text_via_reduce(50_000, 16_000, ~w(de fr es it pt))
end
})
end
end
Included:
- Two alternative implementations of an algorithm to get a singular text and break it apart on several requests;
- One function that uses the faster of both implementations (
requests_for_multiple_texts); - Related to above: run the
benchmarkfunction (make a small new Elixir project and just includebencheein it, or useMix.installin a single.exsfile ) and you’ll see for yourself which of both is faster. SPOILERS: it’s the one usingEnum.reduce. I could probably make an even faster one but wasn’t in the mood to make one that uses only pure recursion and nothing else; - Demo data + demo function that, ahem, demonstrates its correctness with an example.
The output might be slightly cryptic, so explaining it:
You get a list of tuples: first element is the text key (or the text itself), the second one is a list of lists of tuples.
The list of tuples represent a single request which might have e.g. same text with 5 languages. Each text here is represented by its length, not the key / text itself. Got tired and didn’t want to bloat the functions further with one more piece of data. ![]()
The list that encompasses that list of tuples is the list of requests that must be made for this single text to get fully translated.
Not sure if the code is good but it gets the job done. With the demo data above the result is:
[
{"text 1", [[{16000, "en"}, {16000, "ja"}, {16000, "za"}], [{16000, "ru"}]]},
{"text 2", [[{25000, "de"}, {25000, "es"}]]},
{"text 3",
[
[{10000, "bg"}, {10000, "ru"}, {10000, "de"}, {10000, "sp"}, {10000, "es"}],
[{10000, "za"}]
]}
]
…which means:
"text 1"gets to do 2 requests: one with 3 languages and one with 1 language;"text 2"gets to do 1 request with 2 languages;"text 3"gets to do 2 requests: one with 5 languages and one with 1 language.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









