djaouen
I am trying to use Meta’s Llama 2 with Bumblebee, but I am getting a 401 error when I try to load it. I have been granted access to the repo on Hugging Face, but I think I need to provide an access token when loading the model, and I am not sure how to do so.
{:ok, model} = Bumblebee.load_model({:hf, "meta-llama/Llama-2-7b-chat-hf"})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "meta-llama/Llama-2-7b-chat-hf"})
** (MatchError) no match of right hand side value: {:error, "HTTP request failed with status 401"}
(stdlib 5.0.2) erl_eval.erl:498: :erl_eval.expr/6
#cell:oqtu7kdr36by6ud4yug4rxbxs3qw544i:15: (file)
Where Next?
Trending in Questions
Other Trending Topics
Latest Livebook Threads
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 12 to 3- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
josevalim
Good call, I believe the updated version is here
slashmili
In case someone still needs Llama doc, it’s not available on the latest doc and can be found here https://hexdocs.pm/bumblebee/0.4.2/llama.html
josevalim
You can go ahead and do the bindings for llama.cpp yourself. Check out Erlang NIFs.
Otherwise, docs for LLaMA have been added here: https://hexdocs.pm/bumblebee/llama.html
nutheory
I was able to get “NousResearch/Llama-2-7b-hf” working using 0.3.1. Does anyone know if we can use GGML locally so i don’t have to wait for 5minute inference for two sentences on my M2?
djaouen
Actually, I don’t know what version of Bumblebee I was using, as it’s been long enough that Hugging Face deleted my notebook. But I will certainly look into that if I build out a similar project in the future. Thanks for the info!
drewble
Looks like v0.3.1 has the official support for
Bumblebee.Text.Llama. Is that not working for you?djaouen
Not yet. I was waiting for Bumblebee to get upgraded so that I could use
Bumblebee.Text.Llama.haavars
Did you ever get i to work?
djaouen
Thanks, @regex.sh and @jonatanklosko!
jonatanklosko
Hey @djaouen, you can specify it in repository options:
{:hf, "meta-llama/Llama-2-7b-chat-hf", auth_token: "..."}: )