Ajaxdone
How to write to a file that combined by multiple blocks in shorter time
I want to write a big file, but the file is from 1000 blocks, also the blocks are from network. so I think better not doing that in a sequential manner like:
- first read all blocks or wait until block ready to write like 0,1,2,3,4,…wait 5 to come
I want to do like this:
- preallocate 1000Mb file
- spawn 1000 processes to get the blocks
- if block 51 ready then write 51 to 51*1MB offset
- do this until write all.
Question:
- does Elixir support pre allocate?
- can I write to same file at diff offset simultaneously without locking issue?
Marked As Solved
kip
I think you should be able to do random access writes with :file.pwrite/3. If you open the file in :raw mode then you also write directly the file, not via the IO process (if I’ve understood this correctly).
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"