You can set a secret OPENAI_KEY in livebook, then you must give your current livebook access to it in the secrets tab.
After that you can get the secret with an LB_ prefix and pass it to ChatOpenAI.new!
api_key = System.get_env("LB_OPENAI_KEY")
%{
llm:
ChatOpenAI.new!(%{
model: "gpt-4",
api_key: api_key,
})
}
|> LLMChain.new!()
Yeah I tried that exact same thing and for some reason the api_key in the map is always nil for some reason.
I can retrieve the key from the environment but when I set it in the map it’s nil for some reason
Thanks for the help I’ll continue to try to figure out what should be a very easy thing to do
Today with zero code changes it works! Shut down and restarted the server a few times, and now it’s good to go. Thanks for the help.
Amazing work! Many thanks to all contributors!
By combining this library with Ollama, I can transparently use the GPU of my MBP (Apple Silicon). Interacting with LLMs within Elixir applications becomes very efficient and local-only (I dislike reaching cloud services with sensitive context).
Amazing work! Many thanks @brainlid and all contributors!
I am starting using this library, so far looks great. Have minor question, what be the best way to handle alternative logic based on user input? I mean in code? For example, in my application, different logic will be executed based if the user A. wants to order food, or B. wants to eat at restaurant.
Also, how to provide it contextual data (actually data retrieved from tables in DB) based on user choices during the conversation? i.e. how to hook inside an ongoing chat and feed in DB data?
Thank you!
Hi @acrolink!
A RoutingChain is a good tool for changing how handle different contexts. It’s a basic prompt and chain that looks at the prompt and determines “Do they want to order food to be delivered? Or do they want to eat a restaurant?” The LLM looks at the prompt you provide and the user’s message to make the decision.
The result is to select a new, purpose built chain. You can then run the same message from the user on the selected chain.
To access databases, typically that’s done through a Tool. You define the tools (aka functions) that the LLM should have access to. If the LLM decides to call the tool, the library handles it and executes your Elixir code. Your function’s result is returned to the LLM for it to act on.
Have fun!
I have tried to implement RoutingChain but failed. It would be nice if you could give me an idea how to do that in the context of this file:
Thank you 
Ok, I made some progress. The documentation needs to be updated to this:
selected_route =
RoutingChain.new!(%{
llm: ChatOpenAI.new!(%{model: "gpt-4o-mini", stream: false}),
input_text: "I need help with JAVA programming language",
routes: routes,
default_route: PromptRoute.new!(%{name: "DEFAULT", chain: llm_chain})
})
|> RoutingChain.evaluate()
i.e. using RoutingChain.new! and llm: ChatOpenAI.new!(..)
I have made it work and can think how to implement it before the chat starts, e.g. using buttons (click handlers) determining the input_text to provide for RoutingChain.evaluate().
But would it be possible to do the routing inside an ongoing chat (changing the chain inside an ongoing chat)? If you can give me some direction that would be great. Thank you.
Hi @acrolink!
Thanks for the doc note.
But would it be possible to do the routing inside an ongoing chat (changing the chain inside an ongoing chat)?
The way I handle this is to manage the displayed chat separately from the one sent to the LLM. Just like we don’t want to display the system setup or other prompts to the user, those are sent to the LLM but not displayed to the user.
More advanced LLMs could handle a routing prompt inside a conversation, but the simpler LLMs cannot.
To me, it makes the most sense to perform the routing operation as a standalone chat with the LLM. Like you run it on the side, then use the result to define your next prompt. You don’t have to switch to another chain. You may only want the result of the LLM’s topic choice.
I have a question, I actually need your advice if this is feasible:
Do you think the logic inside this file (in the demo):
https://github.com/brainlid/langchain_demo/blob/main/lib/langchain_demo_web/live/agent_chat_live/index.ex
Can it be re-written using Phoenix channels? Do you think the whole LiveView code can be replaced by Phoenix channels and Dart code (on a flutter mobile application) - I mean to write the chat interface natively in Flutter using phoenix_socket | Dart package ?
I am not sure how to expose the display_messages array to Flutter .. Or is it that the phoenix_socket | Dart package has limited functionality, for this end?
Thank you.