Hi everyone!.
In 3 years working with Elixir, my only frustration has been (as you may expect for a small community) the lack of “client” libraries (libraries that connects to other services, that have to keep the timing of changes or versions of the connected service). We usually don’t have official client libraries for other services, as they are not interested in a small community.
Usually someone creates the client, but the effort of maintaining something which is changing all the time is too much for one person.
I have seen this problem with SqlServer and Azure SQL (TDS / Ecto), Kafka, Azure Storage and Cassandra to mention some of them.
Contributing sometime is possible (I have sent some PRs), but sometimes the maintainer does not have enough time to review your PR, or simply, I don’t have the brain to be able to contribute.
I would love to personally sponsor some of these projects, but none of them offer Github sponsorship.
The Erlang Foundation have an External Process Communication Working Group. But I’m not sure if they are addressing this issue.
¿What do you think we could do to address this issue?
1 Like
¿What do you think we could do to address this issue?
In my opinion codegen is the way to go, as you said, maintaining a client lib against a target that is changing all the time is a ton of effort for a single person, which means that most of the time we just hook up a small wrapper with the needed methods for our project, it works pretty well for a single project, but is tightly coupled and makes no sense to release it since it will be useless outside of a very small and well defined scope.
A couple weeks ago @wojtekmach created this proof of concept, and I think that expanding on his ideas could be a nice starting point.
https://elixirforum.com/t/lets-try-to-get-officially-supported-elixir-libraries-for-stripe-braintree-and-paddle/37924/42?u=thomas.fortes
1 Like
That is a great approach, but I don’t believe it’s generalizable.
For example Brod and Kafka_ex uses code generation techniques for the Kafka Protocol, but all the behaviour (for producing and consuming messages) has a ton of work behind. Erlkaf just uses the NIF for librdkafka, and works great, but then we depend on C libraries.
Then TDS has a well implemented binary protocol, but now they have to handle SSL, Active Directory, etc… a ton of work again.
I was thinking more on how founding these projects.
1 Like
I also stand behind code generation – it’s the best way to go if we want to cover a lot of surface with limited amount of humans behind keyboards to support the projects.
If the protocol has an OpenAPI machine-readable file then a lot of the work can be done by the computer and not by the human.
Failing that – or in addition to that – and if the service is SaaS and if it has a sandbox environment (loads of “if”-s, I’ll admit that) then an integration testing suite running once a week that asserts that the endpoints are still there and are working as expected is a reasonable compromise. I’ve done it in the past with other languages.
1 Like
I’m skeptical about code generation. The only such client I’ve tried was aws-elixir and it had several serious problems and the contribution process had a lot more overhead than other libs I’ve used. Granted, that’s a sample size of one…
Personally I have just been writing my own wrappers for any APIs I need to integrate. Elixir makes that task so frictionless that I honestly haven’t felt any pain. Again, in my case I haven’t had to tackle any really complex integrations, but especially with dealing with modern, half-way decent APIs these wrappers end up being just very simple json encoders and delegators so I genuinely prefer to just maintain my own simple clients.
To take an example where I did use an existing library, here’s Twilio’s Message API: Message Resource Reference for the Twilio Messaging REST API - Twilio
And here’s the ex_twilio function (one of only 2 I actually use): ExTwilio.Message.create(to: target_number, from: twilio_number_you_own, body: body)
To replicate that in my own app without the added dep, I could probably write a 15 line Tesla module, not much more than the wrapper/behavior I need anyway to mock the service in tests in a practical way.
Again, I’m sure there are apps with a lot more demanding integrations to deal with, and maybe code gen could be done really well (I’d be curious to see some model examples) but for now I feel the benefit of having one less dep/config and more transparent code outweighs the convenience of most API client libs, which I think is largely in the opportunity to make APIs a lot simpler to work with then they are directly, which is precisely where I think code generation is weak.
1 Like
I was thinking on data clients (databases, queues and file systems like sqlserver, cassandra, kafka, couchdb, foundationdb, hdfs etc…) . Some of them could use a odbc driver, but in most cases we have to implement the binary protocol, and sometimes they delegate a lot of logic into the client.
1 Like
