sallaumen
Possible graphic RAM deallocation issue noticed when using Nx with EXLA
Hello everyone, I’ve been studying Nx for a while by now, and have been writing an article about it.
When analyzing its performance, I’ve noticed that EXLA seems to not be deallocating the graphic card RAM (not sure if it is EXLA’s responsibility), and I would like to share this at the forum to check if I’m doing something wrong, or if it is actually a known issue.
I noticed this behavior when I was measuring some performance metrics with the project UnixStats, an authorial project, and it was able to measure CPU, RAM, GPU and Graphic RAM, when I’ve noticed that the Graphical RAM was never deallocated after my process finished training a simple NN (Neural Network) using Axon to solve MNIST.
Here is the graphic plot I’ve made with these measurements when training MNIST in Elixir, using Axon, Nx, and EXLA:
Also, I’ve done the same measurement solving MNIST in Python with NumPy, and the deallocation issue was not present in this scenario, as can be seen in this plot:
By now I will be not providing all my setup details just to not make this issue too long, but if any questions are needed about the current setup or env that my machine is using to run these tests, ask me in the thread I’ll answer them, okay?
Most Liked
seanmor5
XLA (which EXLA binds to) preallocates 90% of system VRAM for use for performance reasons. So what you’re seeing is the EXLA CUDA client process holding on to that for as long as it is alive. You can disable preallocation by setting preallocate: false To your CUDA client settings
You can also adjust the preallocation percentage with memory_fraction: 0.5 for example to only use 50% of available VRAM
seanmor5
Unfortunately there is no direct way to guarantee that XLA lets go of the memory allocated to it. You can transfer buffers from the GPU which will “deallocate” the memory in use (as in memory XLA knows it has available to it) but there is no way to guarantee XLA gets rid of memory it takes ownership of until the XLA process dies.
Preallocation is really the more performant option, is there a specific reason you want to deallocate manually?
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Real-time error tracking, performance insights, and observability for devs.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"