How to download big files, using threads in elixir

How to download big files from urls, with high speed using threads , how can i chunk the downloading process

Are you sure that processes are your bottleneck? For http you could do a head request to get the size of a file (Content-Length response header) and then do multiple requests using the Range header to request certain chunks of the file. That won’t speed up anything if the bottleneck is elsewhere though. E.g. if your network connection is the problem there’s no way to squeeze more data through without changing that.

Generally concurrent downloads make more sense if you’re downloading many, but smaller things.

1 Like

What have you tried?

1 Like