Better way to run local embeddings with Apple Metal than Ollama?

onnimonni · February 20, 2026, 6:48am

I would want to create local phoenix app to allow semantic search for local files.

In order to do this I need to calculate embeddings for my files and I’m wondering if we have good elixir integration for Apple Metal hardware acceleration to calculate the embeddings with nomic-embed-text or should I just use Ollama instead?

How is the MLX support in the current tooling?

al2o3cr · February 20, 2026, 8:44pm

There’s emlx, a Metal-powered backend for Nx:

Haven’t tried it out lately, though

polvalente · February 21, 2026, 2:00am

EMLX with Nx 0.11 should work great for most things.

I have work in progress for even faster execution times, bringing back the original attempts at a compiled mode.
It didn’t work back then because of a bug in Nx that has been solved in the latest release.

Also, with Nx.runtime_call you can interleave external non-Nx code into your Nx computations, so you can do a best-of-both-worlds approach

onnimonni · February 24, 2026, 8:20pm

The mixing of nx and non nx sound like pure alchemy for me .

Links to examples would be amazing but I will anyway try it out

polvalente · February 24, 2026, 9:17pm

github.com/elixir-nx/nx

exla/test/exla/defn/runtime_call_test.exs

7da307c8c


      
          
          deftransform add_offset_callback(t, opts) do
            t
            |> Nx.as_type(:f32)
            |> Nx.add(opts[:offset])
          end
          
          defn add_offset(x) do
            out = %{x | type: Nx.Type.to_floating(x.type)}
          
            Nx.runtime_call(out, x, fn t -> add_offset_callback(t, offset: 10.0) end)
          end
          
          test "runtime_call with single output" do
            x = Nx.iota({5})
            y = add_offset(x)
          
            expected = Nx.add(Nx.as_type(x, :f32), 10.0)
            assert_equal(y, expected)
          end

There are a few tests where you can get familiarized with the interface. The function docs should be enough, though.