lawik

lawik

Nerves Core Team

Blog Post: Voice Activity Detection in Elixir with Membrane

Building on other people’s work I bashed things together and suddenly I can know when someone is speaking using Elixir and Membrane.

Most Liked

nefty

nefty

Great post! In case anyone else runs into the same issue, the latest silero_vad.onnx on their GitHub repo (version 5) expects different inputs than the version used in this post (version 4). The inputs h and c are now combined into a single state tensor: {"state", "Float32", [2, nil, 128]}
Here is the updated code:

init_state = %{state: Nx.broadcast(0.0, {2, 1, 128})}

live_audio
|> Kino.Control.stream()
|> Kino.listen(init_state, fn
  %{event: :audio_chunk, chunk: data}, %{state: state} ->
    input = Nx.tensor([data])
    sr = Nx.tensor(16_000, type: :s64)
    {output, state_n} = Ortex.run(model, {input, state, sr})
    prob = output |> Nx.squeeze() |> Nx.to_number()
    row = %{x: :os.system_time(), y: prob}
    Kino.VegaLite.push(chart, row, window: 1000)

    {:cont, %{state: state_n}}
end)
lawik

lawik

Nerves Core Team

Thanks! Linked to your post in a small update and also linked the original model that works with the code I had.

delitrem

delitrem

Awesome, very nice work!

By the way, I just remembered about 14 years ago I had a startup, where one of components was a gateway from real CB amateur radio (27Mhz) to our service. It was of course built with Erlang and used some magic around home-grown sox driver, which also had voice/sound activity detection.

P.s.: if I owned the time-machine that times, I would use your solution. :slight_smile:

Where Next?

Popular in Blog Posts Top

luckywatcher
Hey all! I just started a blog focused on web development in Elixir. I wrote my first article and thought I’d post it here for the whole...
New
aymanosman
This is a very short article about using AWS Systems Manager Parameter Store to load secret configuration in a simple way.
New
paulanthonywilson
So you’re enjoying using WebSockets with Elixir’s Phoenix Framework, and you want to send some binary messages. Maybe it’s an audio clip,...
New
StuntProgrammer
Hey there! This is the first article in what I hope will be a nice series about how I built (and am building) Lofi Limo. If you have any ...
New
alvises
Just published the first part of series of articles aimed to explain the architecture behind a kv-store engine written in Elixir and impl...
New
brainlid
OTP 26 was released and the Elixir 1.14.4 builds have been updated! Erlang OTP 26 changed how map keys are sorted, or not sorted actually...
New
brainlid
I love LiveView. Navigating between views is so fast! This quick tip makes navigating feel instantaneous by adding a split-second delay b...
New
mudasobwa
Blogged about the motivation and reasoning behind my idea to create yet another FSM library. Long story short: I did it in a proper way :...
New
brainlid
Phoenix 1.7.0 brings a lot of new things when we run mix phx.gen my_app. These new and cool ways of doing things aren’t automatically bro...
New
lawik
Building on other people’s work I bashed things together and suddenly I can know when someone is speaking using Elixir and Membrane.
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48475 226
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29703 241
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31307 143
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New

We're in Beta

About us Mission Statement