djaouen

djaouen

Using Llama 2 With Bumblebee

I am trying to use Meta’s Llama 2 with Bumblebee, but I am getting a 401 error when I try to load it. I have been granted access to the repo on Hugging Face, but I think I need to provide an access token when loading the model, and I am not sure how to do so.

{:ok, model} = Bumblebee.load_model({:hf, "meta-llama/Llama-2-7b-chat-hf"})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "meta-llama/Llama-2-7b-chat-hf"})
** (MatchError) no match of right hand side value: {:error, "HTTP request failed with status 401"}
    (stdlib 5.0.2) erl_eval.erl:498: :erl_eval.expr/6
    #cell:oqtu7kdr36by6ud4yug4rxbxs3qw544i:15: (file)

Marked As Solved

jonatanklosko

jonatanklosko

Creator of Livebook

Hey @djaouen, you can specify it in repository options: {:hf, "meta-llama/Llama-2-7b-chat-hf", auth_token: "..."} : )

Also Liked

josevalim

josevalim

Creator of Elixir

You can go ahead and do the bindings for llama.cpp yourself. Check out Erlang NIFs.

Otherwise, docs for LLaMA have been added here: https://hexdocs.pm/bumblebee/llama.html

josevalim

josevalim

Creator of Elixir

Good call, I believe the updated version is here

djaouen

djaouen

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
yawaramin
In the Dialyzer docs ( dialyzer — OTP 29.0.2 (dialyzer 6.0.1) ), there is a way to turn off a specific warning for a function: -dialyzer...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48475 226
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

We're in Beta

About us Mission Statement