How do you deal with a lack of third party SDK's for Elixir?

Phoenix Forum Chat / Discussions
, , ,
1

Hi all! I am new to Elixir, Phoenix, and this forum, so please let me know if this post should go in another category.

First let me say that I do not mean this as an attack on Elixir or Phoenix. The language, framework, and just as importantly, the community seem incredible. I am curious about this because I really want to use Phoenix going forward :smiley:

So my question is, how do you all feel about/deal with the lack of third-party libraries for Elixir and Phoenix? Here is some context for my situation. For my MVP for my startup, I am trying to decide between Ruby on Rails, .NET Blazor Server, and Phoenix. I think a server-side framework will allow me to add features and iterate faster as a one man army. Some of the SDK’s I am wanting to use are:

  • Auth (Auth0, Firebase, Azure B2B are all contenders)
  • S3 client (.NET and Ruby both have an official one)
  • Stripe (I know there is an unofficial Elixir one but I am worried about using an unsupported SDK for something as sensitive as payments)

Some possible solutions I have thought of are:

  • Use unofficial SDK’s and HTTP API’s. The con of this is messing something up like auth or payments would be
really bad.
  • Write microservices in C# for auth and payment management. This adds complexity which is the opposite of what I am going for with a server side framework.

I am planning on deploying with Fly.io for the web app and then self-hosting Minio/Ceph/Rook for S3 and Postgres for database.

1 Like
2

There is another possibility - all of these services provide API documentation. You can use that to write your own integration.

In general I would be more concerned about “harmless” services like logging and metrics where you can accidentally leak your or your users credentials. And in many cases payments API is super simple and hard to get wrong if you do not ever receive payment details.

So in my case it is:

  • use whatever SDK is there (potentially with code review of the dependency)
  • write integration on my own

I never even considered writing gateway application in separate language, as I find it needless in 99% of the cases (remaining 1% is where the API is so complex that writing it from the ground up would be enormous waste of time).

6 Likes
3

Nowadays for some very specific services like Oauth, Stripe and friends, you will most likely find an official SDK for Elixir.

If not, it is very likely the vendor will provide an OpenAPI spec of their API. Why does this matter? Because then you can use the OpenAPI SDK generator to just generate an Erlang or Elixir SDK, and even stuff like server stubs. Tried it recently and it will do 95% of the work for you.

Even if you opt for manually making your own client library, it’s not a big issue imho since the available Elixir HTTP clients allow for a good deal of abstraction to make your own quite easily.

6 Likes
4

If I have to write gateway application in a different language, I will make it a command line application that communicate with the Elixir side with port. There are good usage cases for microservices, but gateway wrappers for API is not one of them.

2 Likes
5

Though if you want a more universal, micro services-oriented approach, something like gRPC makes sense (way more lightweight than HTTP/WS JSON payloads), and you can autogenerate the classes/modules for each side. We do this in our company - a big Rails monolith that talks to the data-driven Elixir services via gRPC calls.

6

Yes, Have a look at Ports. A little Go-App (S3 has a official Go SDK) on a Port is a great way to do that.

But most of the APIs are really simple and Elixir is a geat language to talk to an API.

7

Hi @memoria-guy ! Welcome to the community!

Let’s break things down so I can address (with my opinion) your questions:

how do you all feel about/deal with the lack of third-party libraries for Elixir and Phoenix?

This is a valid question. We usually see these when folks come from other ecosystems that are more “batteries included”. It is good to know your (new member, without our community’s biases) perspective, so thanks for asking. We all have biases and they tend to creep in as time goes by. We often forget how things were when we were first exposed to something new. Expert bias is real and it is great to have input like yours.

Disclaimer: In no way I am saying anything about your skills or anybody’s skills. I am just giving anecdotal (mine) evidence and discussing my experience, which is what @memoria-guy asked for. :heart:

The biggest shift in perspective in this “lack of libraries” discussion happens when someone starts reading the api’s documentation rather than the library’s documentation (a readme with tons of examples for example). Maybe you wanted something that the library didn’t support, maybe the library became a bottleneck (happens quite a bit), maybe you just got curious and started to peek under the hood. Once this shift happens, you start to give less value to the pre-built libraries because they start to become less magical. Ultimately, you start thinking about problems from the bottom up: “Wait, all this does is send a request to a url?”. This is what happened to me at least, ymmv. The code goes from:

token = "some token"
headers = "Headers"
client = ThirdPartyLibrary.new(%{token: token, headers: headers})
payments = client.get_payments()

to:

token = "some_token_from_another_request"
url = "https://example.com/api/endpoint_that_needs_a_bearer_token"
headers = ["Authorization": "Bearer #{token}", "Accept": "Application/json; Charset=utf-8"]
{:ok, response} = HTTPoison.get(url, headers, [])

The snippet above comes from HTTPoison’s readme. Another good library for http requests is Req

Let me get back to the question, I feel like I’m going on a tangent a little, but I thought this context would be beneficial.

To me, the real value these libraries offer comes down to the readmes and examples they provide. They tend to offer A LOT of examples, guides and warnings about common gotchas. Issues in the repo also seem to help a ton. It’s not the library itself, it’s the documentation, support and help you get from using the library. I hope I am making sense here.

So, let’s address what IMO is the root cause here: having the documentation, support and help when you are building things. I’ll list a few places (they are all listed on the website: https://elixir-lang.org/). This is where I go for help when needed and honestly, learn a ton just by reading discussions:

  • ElixirForum (you are doing this already! :heart:)
  • Elixir Slack (browse the channels, there is usually a “the library/api I
    want” channel)
  • Elixir IRC
  • Blog posts from the community (#MyElixirStatus is a great aggregator on Twitter)

Now, with that out of the way, here comes a plot twist: For the exact reason you asked this question I think there are benefits (biggest value add is to newcomers and folks that are considering using Elixir for their projects) to having these “go-to” libraries. Even if they don’t end up being used in favor
of hitting the apis directly at some point in their journey.

With this in mind, let’s address the other concerns you mentioned above:

S3 client (.NET and Ruby both have an official one)

What about using aws-elixir or ExAws? Both work very well. The former is auto generated from the output of a Go official SDK (and is what I’m using in small projects) and I know the latter is used in many production applications in large companies.

You pointed out that there are some official SDKs. I went ahead and asked what the process is here: www.reddit.com/r/aws/comments/qjvh9f/official_sdk_for_elixir to get one of these auto generated ones (the two you listed are auto generated) since if you were able to spot this (and feel less confident about
Elixir) others may have as well.

We have one of these auto generated libraries for Google and as I mentioned above, the real value comes from the documentation, support and examples you get from these libraries. Google has solved this by creating an additional repo that is supported by the community.

As far as documentation and examples, we can always try to help out build that knowledge base regardless of the size of the contribution.

Auth (Auth0, Firebase, Azure B2B are all contenders)

Have you looked into the built-in Auth generator that comes with Phoenix 1.6? Here is an awesome blog post about it: An upcoming authentication solution for Phoenix - Dashbit Blog. IMO this should be the go-to for now projects, but you should make your wn decisions :slight_smile: A generator that adds everything you need, created by folks responsible for Devise and audited by security specialists is good enough for my needs, ymmv.

Auth0 remains a good option as well, they have great docs on how to interact with their api. I saw how Elixir doesn’t have a quickstart app so I requested it (you need to search for “Elixir”, then they give you an option to “Suggest a New Quickstart”).

Also, I emailed them at “hello@auth0.com” and asked about an Elixir example/tutorial/SDK. Let’s see what they say.

Stripe (I know there is an unofficial Elixir one but I am worried about using
an unsupported SDK for something as sensitive as payments)

As you mentioned, there are a couple of unnoficial Elixir libs out there, but ultimately, they are all hitting the extremely well documented Stripe api.

Here is an example of a Stripe client from Dashbit’s open sourced project Bytepack

Maybe this will get you closer to where you need to be with integrating with their API.

Thanks for saying hi and asking these questions! Welcome to the community! Please let me know if you have any questions, happy to help.

Edit 1: The reddit link was broken (seems like the Forum blocks it somehow?). My post has been approved and seems to be up now: https://www.reddit.com/r/aws/comments/qjvh9f/official_sdk_for_elixir/

15 Likes
8

It’s a pain in the ass, yes. But the rest of Elixir is so good that I suck it up. We had to rely on third-party packages made by strangers and sometimes whip something up ourselves.

I do miss having official packages for mainstream services like S3, Stripe and such. It was comfy not gonna lie.

4 Likes
9

Wow, thank you so much for the thoughtful and encouraging reply!

Expert bias is definitely a thing, but noob bias is probably a thing for me lol. I see prebuilt SDK’s as a big advantage for an ecosystem, but maybe I am overthinking it. Programming/web development is just a hobby of mine and I am still learning.

Fortunately because programming is a hobby and will likely never be my full-time career, I also am not boxed in by what languages/frameworks have the most jobs. Most advice online about how to choose a language/framework seems to circle back to the job market, which for me is not relevant
I guess other than the loop of: popular languages = more jobs = more stuff on Stack Overflow, more libraries, more examples, etc.

3 Likes
10

What parts of Elixir/Phoenix do you like that make up for some of its cons? I read a really encouraging post on /r/elixir from an experienced Ruby dev that reignited my interest in Phoenix, so I like to hear people hype Elixir up :slight_smile:

11

Push notifications is easy to build in house.

Background jobs are “solved” and predictable using Oban. You really don’t need to sweat it, and when you do need to sweat you are probably have enough money to hire more engineers to deal with it.

Predictable performance where 99.9% of the time your bottleneck will be the database - not your app.

Stupid simple horizontal scaling.

these are among my favorites. I can focus my time on building sweet features and not really worrying about scaling my Elixir solution.

3 Likes
12

I personally don’t really feel like SDK’s for web services are necessary. Most of using said services like Stripe or S3 is just dealing with auth and the API, which is easy to do in any language. Yeah, you might need to write some more code than you would if Amazon supplied an S3 library, but it’s mostly just HTTP requests.

For more complicated things like database drivers, that’s a real issue because your average developer may not have the expertise or time to develop that kind of a library.

However, I think people worry a bit too much about the lack of libraries/SDKs for services that don’t really need them.

7 Likes
13

When it comes to S3, is there any particular reason you don’t want to use ex_aws? It’s been around for a long time and is pretty neat!

1 Like
14

However, I think people worry a bit too much about the lack of libraries/SDKs for services that don’t really need them.

Seems like I might be one of those people lol.

I am going to take the Elixir plunge and get the Pragmatic Studio course (I really enjoyed the free preview). It quickly was reminded how much fun functional programming is
I haven’t gotten to use the pipe operator since I messed around with F#.

Thank you for everyone who replied!

Also does anyone know if there is a discount code for the Pragmatic Studio combo course? I am a student and $250 ain’t cheap.

Edit: they already replied with a coupon code! Thanks Pragmatic Studio!

1 Like
15

Perhaps I’m a little odd but I generally find it easier to use a HTTP client and write some code to interact with just the endpoints I’m after. A generic API client tends to be a ton of code, and any bugs or quirks can be challenging to get resolved now that you’ve got an external dependency.

If you write the code you have full control, and you can shape it to precisely your use case.

In my experience if the API client has an especially generic interface you may find that you have to write fewer lines of application code if you remove it. Always a nice feeling there :slight_smile:

12 Likes
16

Ya, this applies for me as well.

For example, I found an API client recently that was officially supported but didn’t have all of the filters I needed. So I ended up having to hack around it a bit. I should’ve just written my own requests (as I do 9/10 times) and it would’ve never been an issue.

It’s nice when API clients just work, as they often did for me in Ruby, but honestly I haven’t missed them much.

7 Likes
17

I haven’t done much F# programming but pipe in Elixir is bit different. In F# pipe goes to last argument and in Elixir it goes into first argument.

Also as C# and TypeScript programmer by profession I really like Elixir because it has immutability without escape hatches. That helps to reason about code you read that it can’t mutate anything inside a process. Really like the actor model (lightweight processes) if I remember correctly Erlang was created before whole actor model was a thing. Because of these lightweight processes you don’t need async/await that some other languages have because it’s cheap and fast to start process do asynchronous task. Async/await usually brings lot of virality with it, in .NET/C# standard library for example has synchronous and asynchronous versions of lot of things. Garbage collector is also per process and have their own isolated memory, only big binaries are shared between processes. Because of this pauses are per process and Elixir doesn’t usually suffer from any garbage collector pauses as many languages with garbage collector do. For short lived processes garbage collector might not have even run and memory is released when process dies.

You can build awesome things with building blocks that platform provides. For example all things that LiveView provides is not as easy to implement in other languages/platforms. There are similar things being built with other language but you can’t get same kind of isolation per connection in other platform. I also felt realtime things like synchronizing state between connection with LiveView where MUCH simpler than in C#'s Server-side Blazor for example.

Only negative I have about Elixir is that I wish it had more static typing support. Watched new ElixirConf video Q&A with Chris McCord & José Valim | ElixirConf EU 2021 - YouTube where José was talking about that and it gave me hope something is going to happen in that area.

4 Likes
18

I think there’s a misconception that official SDKs are more well-maintained than open source libs. I have used official SDKs that were very stale, couldn’t be forked, and made things difficult. I’ve open source packages, such as ExAws, which are kept very up-to-date and have delightfully clear documentation.

Sorry if that’s not a very helpful answer. The Elixir community is full of great open source packages that, in my personal opinion, are often better than official options.

6 Likes
19

Im having a similar issue with missing firebase Admin SDK :frowning:

20

You can check out my in progress library Flame