lawik

lawik

Nerves Core Team

Blog Post: Voice Activity Detection in Elixir with Membrane

Building on other people’s work I bashed things together and suddenly I can know when someone is speaking using Elixir and Membrane.

Most Liked

nefty

nefty

Great post! In case anyone else runs into the same issue, the latest silero_vad.onnx on their GitHub repo (version 5) expects different inputs than the version used in this post (version 4). The inputs h and c are now combined into a single state tensor: {"state", "Float32", [2, nil, 128]}
Here is the updated code:

init_state = %{state: Nx.broadcast(0.0, {2, 1, 128})}

live_audio
|> Kino.Control.stream()
|> Kino.listen(init_state, fn
  %{event: :audio_chunk, chunk: data}, %{state: state} ->
    input = Nx.tensor([data])
    sr = Nx.tensor(16_000, type: :s64)
    {output, state_n} = Ortex.run(model, {input, state, sr})
    prob = output |> Nx.squeeze() |> Nx.to_number()
    row = %{x: :os.system_time(), y: prob}
    Kino.VegaLite.push(chart, row, window: 1000)

    {:cont, %{state: state_n}}
end)
lawik

lawik

Nerves Core Team

Thanks! Linked to your post in a small update and also linked the original model that works with the code I had.

delitrem

delitrem

Awesome, very nice work!

By the way, I just remembered about 14 years ago I had a startup, where one of components was a gateway from real CB amateur radio (27Mhz) to our service. It was of course built with Erlang and used some magic around home-grown sox driver, which also had voice/sound activity detection.

P.s.: if I owned the time-machine that times, I would use your solution. :slight_smile:

Where Next?

Popular in Blog Posts Top

MarcinKasprowicz
What I genuinely value at my workplace is that I can easily explore new languages through internal mobility. Throughout my career within ...
New
paulanthonywilson
Following up on the previous post on using UDP multicasting to broadcast and detect peers on a network, I create a registry of those peer...
New
AstonJ
Elixir Blog Posts How to use this section You can post links to your blog posts either in one of the Official Blog Posts threads, or, vi...
New
mssantosdev
Our take on how to build a frontend style guide with Phoenix Components, Atomic Design and plain CSS, with focus on reusability and code ...
New
danschultzer
How to set up the Content Security Policy header with Phoenix LiveView and support inline styles and scripts with CSP nonces.
New
brainlid
In a 2 day spike, I created my own Elixir-based AI Personal Fitness Trainer! The surprising part for me was how useful and helpful I foun...
New
brainlid
You are storing some Phoenix LiveView state in the browser. You want to retrieve that saved state as early as possible to improve the use...
New
ragamuf
Does the world need another How to create a blog article? Maybe not. But then again, creating something out of nothing is what we love....
New
axelson
I describe how we use Hot Reloading with Webpack to develop faster and show how to integrate Webpack 5, webpack-dev-server, and Phoenix f...
New
shanesveller
Introduction and first content post in my new series about running Elixir Phoenix apps on Kubernetes are live! I’m describing an opiniona...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 43487 311
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 127089 1222
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

We're in Beta

About us Mission Statement