Hi @Feliks I tried to implement the openai realtime example available here, trying to integrate it into a liveview example app.
As for integrating membrane in liveview I followed this example membrane_demo/webrtc_live_view at master · membraneframework/membrane_demo · GitHub .
In my implementation, what happens is that the audio is captured and (i think) is sent to openAI, because I can receive the response. But the problem is that together with the response I receive also all the mediaCapture sent over to the server (i.e. Echo).
Do you know what could cause this issue?
this is the liveview implementation: pento/lib/pento_web/live/call.ex at main · tomashco/pento · GitHub
this is the pipeline that gets called: pento/lib/pento/call/pipeline.ex at main · tomashco/pento · GitHub
this is the integration with the openai endpoint: pento/lib/openai_endpoint.ex at main · tomashco/pento · GitHub
this is the integration of the websocket: pento/lib/openai_websocket.ex at main · tomashco/pento · GitHub
I’m pretty sure the problem lays in the pipeline, but I don’t understand what is wrong.
thank you to everyone can help!