Req to chat completions with :into and process send and no other libs required. OpenAI compatible endpoints gives you plug and play with many LLMs, including google’s. You can see langchain here for higher level building blocks and integrations, but it’s shockingly easy to get streaming tokens out to a client wether LiveView or js channels client.

This has definitely been a pain point for us as well. I know it’s asking for a lot but an equivalent of something like AI SDK by Vercel for elixir would be huge. It’s one of the few times where we’ve really been bit by the smaller ecosystem of elixir. Most of the time you can just write it yourself but as @kasvith mentioned its a lot of manual work to implement streaming for each provider.

Did you check this article from Sean Moriarity? https://www.hackwithgpt.com/blog/streaming-chatgpt-responses-with-phoenix-liveview/