Hi All,
I am trying to store text embeddings of 384 dimensions in index of hnswlib (GitHub - elixir-nx/hnswlib: Elixir binding for the hnswlib library.).
def load_index() do
{:ok, saved_index} = HNSWLIB.Index.load_index(:l2, 384, "my_index.bin")
end
idx = load_index()
IO.inspect idx
#{:ok, idx} = HNSWLib.Index.new(:l2, 384, 1000) #works fine
IO.inspect HNSWLib.Index.get_current_count(idx) #gives error {:error, "The number of elements exceeds the specified limit"}
data = Embed.similarity(text)
IO.inspect Nx.shape(data[0]) # {384}
IO.inspect HNSWLib.Index.add_items(idx, data[0])
IO.inspect HNSWLib.Index.get_current_count(idx) {:ok, 0}
It works fine if I try to add items to newly created index, however throws error : “The number of elements exceeds the specified limit” when I try to load the existing index and try to add same embeddings.
