Stream audio data to browser using phoenix

Questions / Help
1

Trying to find an example of streaming an audio file or raw audio data using phoenix but can’t find a working example.

I simply want to press a button and have phoenix play raw audio data from the server on the users browser.

Any tips on how to do this?

2

The way you will stream from phoenix depends on how you want the client to process it, thus what are you going to do on the client to play it (phoenix only handles the server-side)?

3

just a regular html5 audio tag with a path to a url is sufficient i think.

4

Then you can just send the file like any other asset, so the question now is where is the sound file coming from that you need to send? :slight_smile:

5

raw mp3 audio data obtained from an api

6

Is it a real-time stream, so as that data is coming in then it’s immediately going out to connected things, or is it being stored somewhere to be served out later?

1 Like
7

no it is a small chunk of data few seconds audio max, that will be stored then served out later

8

In that case this is outside of my knowledge. You’ll need to build the mp3 header properly and all to pack the streamed data through it. I think there is an Elixir/Erlang library that can do this but I’ve not needed to do it as of yet so I’m unsure. ^.^;

9

hmm, i see, no problem, thanks for trying to help though! I’ll have a bit more of a poke around and see.

Cheers!

1 Like
10

As long as you have the link to the mp3 file, you can output it right into the html with an @assigns. I think you can use:

<audio
    controls
    src="/media/examples/t-rex-roar.mp3">
        Your browser does not support the
        <code>audio</code> element.
</audio>
11

i don’t have a link to the mp3 file, i have the raw mp3 data in memory that I want to expose via a fixed url.

12

Sorry I misunderstood. This totally might not work. But you could try:

<audio
    controls
    src="/get-mp3/some-name.mp3">
        Your browser does not support the
        <code>audio</code> element.
</audio>

Then in a controller:

def get_mp3(conn, %{"mp3_name" => mp3_name}) do
  mp3_binary = get_mp3_from_memory(mp3_name)
  conn  |> send_download({:binary, mp3_binary}, filename: "#{mp3_name}.mp3")
end

The problem may be send_download sends a header of content-disposition: attachment. I don’t know if the browser’s audio player will play it in that case.

Another thing to try is

conn 
|> put_resp_content_type("audio/mpeg") 
|> Plug.Conn.send_resp(200, mp3_binary)

Sorry… also not sure if it’ll work =) but it may get something going.

13

maybe https://stackoverflow.com/q/38551798 that uses PubSub

then you need to use similar to

  defp stream_from_file(fpath, bytes) do
    File.stream!(fpath, [], bytes)
    |> Enum.each(fn chunk ->
      PubSub.publish(:radio, {:radio_data, chunk})
    end)
    stream_from_file(fpath, bytes)
  end

to PubSub.publish from whatever process is holding the mp3 in memory…

14

You might be able to utilize the Membrane framework. While it’s still very raw, they recently released 0.2 and one of the features they list is Support for payloads stored in shared memory. Maybe @mspanc or @mat-hek can verify that this is currently a suitable use-case for Membrane?

3 Likes
15

I managed to get it done, i pushed the audio to the browser as a base64 encoded string via websockets then on the browser side i decoded it to an arraybuffer which can be played as audio.

1 Like
16

It’s bad idea, because you need to send 133% of data (after encoding) if I remember correctly. For audio and fast internet it’s not a really big problem, but imagine doing same for video Blue-Ray.

2 Likes
17

good to know, but it is fine for my use case as it is just audio and the length of the audio is less than a second.

2 Likes
18

I’m glad you got it to work! Would you be able to post a short snippet of what you ended up with to help anyone who stumbles across this in the future?

2 Likes
19

Anyway I’m not sure if it’s best solution even for your use case. No matter if file is small or not. Having extra 33% of data transfer is not as scalable solution as Elixir is designed to be. I could be wrong, but browser JavaScript WebSocket standard already determines a way to send raw file contents. If I remember correctly there was even specific type to handle raw file contents, but again I’m not sure about it. Also I’m not sure how well it plays with Phoenix Channels API. Probably somebody from Phoenix Core Team should know best solution for sending files as they have could implemented it in a way they think is best.

1 Like
20

Personally I think given the small amount of data in question (less than a second) I would only further optimize the data transfer if I had some extra time available. But that becomes a business decision and we don’t know all the constraints that @r11na is under.