djaouen
I am trying to use Meta’s Llama 2 with Bumblebee, but I am getting a 401 error when I try to load it. I have been granted access to the repo on Hugging Face, but I think I need to provide an access token when loading the model, and I am not sure how to do so.
{:ok, model} = Bumblebee.load_model({:hf, "meta-llama/Llama-2-7b-chat-hf"})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "meta-llama/Llama-2-7b-chat-hf"})
** (MatchError) no match of right hand side value: {:error, "HTTP request failed with status 401"}
(stdlib 5.0.2) erl_eval.erl:498: :erl_eval.expr/6
#cell:oqtu7kdr36by6ud4yug4rxbxs3qw544i:15: (file)
Where Next?
Trending in Questions
Other Trending Topics
Latest Livebook Threads
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixirconf-us
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
regex.sh
I don’t think Bumblebee supports that for now?
Does it? @seanmor5
EDIT:
Correction, there seems to be auth_token but for cached downloads
https://github.com/elixir-nx/bumblebee/blob/main/lib/bumblebee/huggingface/hub.ex#L41
djaouen
Thanks for looking into this for me. Is there a way to utilize this with
load_model,load_tokenizer, andload_generation_config?jonatanklosko
Hey @djaouen, you can specify it in repository options:
{:hf, "meta-llama/Llama-2-7b-chat-hf", auth_token: "..."}: )djaouen
Thanks, @regex.sh and @jonatanklosko!
haavars
Did you ever get i to work?
djaouen
Not yet. I was waiting for Bumblebee to get upgraded so that I could use
Bumblebee.Text.Llama.drewble
Looks like v0.3.1 has the official support for
Bumblebee.Text.Llama. Is that not working for you?djaouen
Actually, I don’t know what version of Bumblebee I was using, as it’s been long enough that Hugging Face deleted my notebook. But I will certainly look into that if I build out a similar project in the future. Thanks for the info!
nutheory
I was able to get “NousResearch/Llama-2-7b-hf” working using 0.3.1. Does anyone know if we can use GGML locally so i don’t have to wait for 5minute inference for two sentences on my M2?
josevalim
You can go ahead and do the bindings for llama.cpp yourself. Check out Erlang NIFs.
Otherwise, docs for LLaMA have been added here: https://hexdocs.pm/bumblebee/llama.html