This functionality seems helpful and worth copying.
Providing an abstraction layer for different LLM’s in Elixir, is interesting, but it’s not the first thing I would do.
Partly, it’s a chicken and egg problem. LangChain can work because all LLM providers already have API bindings in python and node.js. Someone would have to provide those for elixir. Secondly, there will likely be only 2-3 “winners” in the LLM space, and it will be more important to support them comprehensively, rather than supporting every possible LLM partially.
I’d be interested in implementing an application level project like AutoGPT in Elixir, starting with OpenAI, adding the “higher-level” LangChain-like functionality, if it proves necessary. Does that sound interesting?
So, indeed LangChain provides all the needed abstractions, but the question remains is there out there another LLM that could be successfully used, in a project like AutoGPT to replace OpenAI? I have tried running GPT4All, also GPT-J-6b, and LLama-cpp and according to my evaluations, none of them performs any good when you give it a complicated prompts, like, asking to choose a tool, or respond in a given format. Granted, I didn’t spend enough time with these, but my intiial impressions were that there is OpenAI, that works, and scaffold for all the other models that may or may not work in the future.
Since this is a Livebook focused library, I want to provide “starter” sample notebooks for people to build their own Livebook UIs. These will be sample livebooks that can also be deployed as apps (using the new Livebook deploy functionality).
I’m beginning with Continuation Kino App — openai_ex v0.1.7, a simple Kino UI for the open ai completion API. It’s very basic, but I’m open to suggestions for improvements and refinements.
@hubertlepicki it might also be worth comparing some of the new supported models on huggingface. although those are full inference nets, rather than just an API interface to one, I suppose they could be integrated into an autogpt like framework.
I’ll try to work up some basic experiments when i get a chance.
Well that turned out to be useful quicker than anticipated! I was getting warning messages from Tesla, and I thought it might be worth using Finch instead.
I’ve basically replaced Tesla with Finch. The new version is 0.1.8.
I had to add a new function to multipart. Until that’s merged, I have to depend on my github clone of the upstream.
Unfortunately, hex.pm doesn’t like non-hex dependencies, so I can’t upload this version over there.
blurb: “The course teaches prompt engineering best practices for application development, different ways to use LLMs, and how to iterate on prompts using the OpenAI API. It covers how to write effective prompts, how to systematically engineer good prompts, and how to build a custom chatbot. The course is beginner-friendly, but it is also suitable for advanced machine learning engineers.”
It is run by Andrew Ng, from Coursera AI fame, and guest from OpenAI.
The course is in python and jupyter (unfortunately not Elixir and Livebook ) but the main thrust is about showing how to organise the process of integrating Open.ai into any system - so not python or jupyter specific.
@neilberkman I’m looking into supporting streaming responses from the chat continuation endpoint. I notice that your solution works by connecting a genserver to eventsource_ex, which appears to use HttpPoison under the hood.
I’ve already moved from Tesla to Finch for connecting to the endpoints. Do you have any pointers to making this work with Finch? It seems to me that I would need to pass in a custom stream(acc) function to Finch.stream which sent messages to a genserver/agent entity every time it was called.
I have an open PR to Req to better enable streaming and made a Livebook example of how it would work with Finch here. You should be able to adapt the FinchStream module to your needs
Fantastic @zachallaun . Thank you!!! I got it working in a couple of hours (had a little trouble parsing the chunks). Now I just need an example for the user guide, and it should be good to release.
Although I do need multipart support as well. Still, it’s nice that there are so many more-or-less usable options.
