bigfish1
Unable to recreate file after downloading it in chunks
I am trying to download files from Azure Blob Storage to process after they are uploaded by my users through a separate client app. For small files I have no issue downloading the data and writing it to a file, however larger ones timeout so I am using the http range header to download the file in smaller parts then write it to a file. My issue is that after downloading all the data I get an error (ErlangError) Erlang error: :no_translation:
If I download a small file that the entire thing fits inside the range I provide the same code works perfectly. This leads me to believe I may be doing something wrong when I combine the bitstrings.
Here is the code
ranges = blob_ranges(get_blob_length!(blob_name, player_id)) # Ranges to be used for chunking files
{last_byte, data} = Enum.reduce(ranges, {0, ""}, fn end_byte, acc ->
case get_blob!(blob_name, player_id, [timeout: 120], [{"x-ms-range", "bytes=#{elem(acc, 0)}-#{end_byte}"}]) do
blob ->
{end_byte+1, elem(acc, 1) <> blob}
{:error, err} ->
IO.puts("Request failed: #{Exception.message(err)}")
end
end)
create_temp_file(blob_name, data)
Most Liked
derek-zhou
This will not work. You chunk may straddle across a UTF-8 multi-byte sequence. You need to concatenate the binary raw.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance










