bdarla
Which GPUs can you use with Nx/Axon/etc?
Is there any list of hardware, i.e. GPU that is compatible with Nx/Axon/etc. (apologies if I missed this information)? Does the general “rule” that any Nvidia is a good match?
Most Liked
toranb
I got this working with the RTX 3060 and found it was painless if you can install PopOS and the nvidia drivers + cuda 11.2. I also recorded a short benchmark and included the side by side of fine tuning with CPU /GPU to give people a sense of how much faster the feedback loop is when you bolt on this nvidia GPU
AstonJ
Great question - I’ve been wondering the same!
Does anyone know if the built in GPU of a MacBook Pro would be ok or would I need to get an eGPU? My Mac has an AMD Radeon Pro 5500M 8 GB (and Intel UHD Graphics 630 1536 MB).
Related thread for external GPUs: Which external GPU are you using for Nx/Axon?
AstonJ
So I’ve browsed a few more posts/articles about ML on Macs and the vibe I’m getting is that if you’re a Mac user and need CUDA, don’t bother 
instead:
-
Use AWS or some cloud service (about $3 an hour).
-
Build a Linux workstation and just SSH into it.
-
Or if you want to use your Mac, get something like a Razer Core X and dual boot into Linux or windows (or run from an ssd) - and don’t get a high end GPU unless you really need it (wait for them to come down in price or see what happens in the space).
Does that sound about right? Maybe things have moved along since?
Last Post!
AndyL
I installed an RTX 3060 in a Thunderbolt eGPU case. Also upgraded my Ubuntu dev box to an 16-core i7 / 64GB ram and 4TB NVME drive.
I’ve come to believe that this is a minimal configuration for local ML development, for small training runs and to generate embeddings for prototype apps.
Nvtop has been super useful to verify GPU operation. Besides Nx, Ollama has been a great tool for experimentation and learning, and gives a restful interface for integration with Elixir apps.
To generate one embedding with my setup, it takes ~90 seconds on the CPU, and ~2.7 seconds on the GPU.
Now and then I exceed memory capacity of the 3060 GPU (12GB/~$300). The workaround is to use smaller models, good enough for experimentation and prototyping.
The 3090 GPU (24GB / ~$1500) has 2x more ram but 5x more expensive.
Beyond that Fly.io has on-demand Nvidia A100s - 40GB @ $2.50/hr, 80GB @ $3.50/hr.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"