Has anyone attempted porting Guidance to Elixir?

Chat / Discussions
1

Microsoft’s Guidance seems to be the future of how developers will build features on top of Large Language Models.
But since it’s in Python, in order for me to use it with Elixir I had to use Erlport.

Has anyone started porting Guidance to Elixir? Maybe I can help :slight_smile:

4 Likes
2

I volunteer to help too. Unfortunately I cannot lead.

3

It will be a good choice to use Doumi.Port if you want to run Python codes in Elixir applications.

4

I used a combination of Erlport and Poolboy, but I’ll check out NimblePool which is what Doum.Port is using for pooling :pray:

2 Likes
5

Thanks!

I think NimblePool is more appropriate for using ports.

Pools in the Erlang VM, and therefore in Elixir, are generally process-based: they manage a group of processes. The downside of said pools is that when they have to manage resources, such as sockets or ports, the additional process leads to overhead.

In such pools, you usually end-up with two scenarios:

  • You invoke the pool manager, which returns the pooled process, which performs the operation on the socket or port for you, returning you the reply. This approach is non-optimal because all of the data sent and returned by the resource needs to be copied between processes
  • You invoke the pool manager, which returns the pooled process, which gives you access to the resource. Then you can act directly on the resource, avoiding the data copying, but you need to keep the state of the resource in sync with the process

NimblePool allows you to implement the second scenario without the addition of processes, which leads to a simpler and more efficient implementation. You should consider using NimblePool whenever you have to manage sockets, ports, or NIF resources and you want the client to perform one-off operations on them. For example, NimblePool is a good solution to manage HTTP/1 connections, ports that need to communicate with long-running programs, etc.

6

And I’m going to port Langchain to Elixir.

I don’t know about Guidance, is it better than Langchain?
If it is, I want to join to porting Guidance to Elixir.

2 Likes
7

Interesting post for me since I don’t know LangChain: Re-implementing LangChain in 100 lines of code

1 Like
8

You might want to look at GitHub - restlessronin/openai_ex: Community maintained OpenAI API Elixir client for Livebook (cc @restlessronin)

2 Likes
9

Yes, and we also have a Qdrant Elixir Client being developed by Marinac. Qdrant is a very fast vector database.

(I’m listing this here to gather resources, as much as anything)

2 Likes