Use GenServer or not

[alexcastano]

Hello, I’m creating a Phoenix app which makes some API calls to Facebook. I found two different libraries to do it:

• exfacebook: This use start_link + pid to work
• facebook: This seems to make the request in the same process.

So my question is which approach is better for an API application? Is there a good reason to use a GenServer to make API calls?

I read some time ago about Supervisors, GenServer, etc, so I am not pretty sure but I suppose I should create my own Supervisor for exfacebook, so in case the process fails it is restarted automatically. Furthermore, I could register the process so any module can make a request without worrying about the PID. Is it this effort worth?

A good explanation about advantages and disadvantages of each one would be great!

Thanks in advance

[OvermindDL1]

@alexcastano

Is there a good reason to use a GenServer to make API calls?

Serializes requests through a central manager so that you can throttle connections so facebook doesn’t start yelling at you.

Overall it depends on what use you want of whether in-process or a dedicated process is better, but for facebook I’d bet a dedicated process is better so you don’t overwhelm their API (perhaps a pool of processes managed by a single process).

[Azolo]

I honestly think that having a separate process to pipe connections through is kinda meh.

It’s one thing to use a connection pool like hackney, but to have to start a process that handle then handles one request at a time seems kind of meh.

Even then I don’t quite understand why hackney uses a pool of processes to make stateless tcp requests. But there are some reasons.

Unless I’m storing a particular open port or some kind of state data, I don’t like having a separate processes.

But if someone had a compelling argument that you needed to start a new persistent process for something like that I would be willing to say that I was wrong.

@OvermindDL1

I’d bet a dedicated process is better so you don’t overwhelm their API (perhaps a pool of processes managed by a single process).

I’d just put a separate GenServer somewhere else that keeps track of requests made and such and it would be started as part of the library’s supervision tree.

The top level library API would just be a wrapper that uses that process by default. That way if people wanted a different way to track and limit their API calls they could just start a different GenServer.

Anyway, just my two cents.

[OvermindDL1]

@Azolo

It’s one thing to use a connection pool like hackney, but to have to start a process that handle then handles one request at a time seems kind of meh.

A GenServer can handle any number of requests, just push them out and wait for responses as messages.

@Azolo

Unless I’m storing a particular open port or some kind of state data, I don’t like having a separate processes.

Which for facebook is useful as they can ban you if you make too many accesses too fast, so it is good to throttle yourself, and a GenServer is an excellent use for that.

@Azolo

I’d just put a separate GenServer somewhere else that keeps track of requests made and such and it would be started as part of the library’s supervision tree.

Indeed, trivially easy, just toss it as a worker in your main supervision tree.

@Azolo

The top level library API would just be a wrapper that uses that process by default. That way if people wanted a different way to track and limit their API calls they could just start a different GenServer.

You mean very much like how my TaskAfter library works, you can define a global handler and/or custom local handlers.

I really think that style is what most libraries ‘should’ be doing. By default start nothing. If an appropriate config is specified then start a global with that setting. Else start your own and use whatever names/pids you want. Or mix and match all you want.

[cmkarlsson]

I really think that style is what most libraries ‘should’ be doing. By default
start nothing. If an appropriate config is specified then start a global with
that setting. Else start your own and use whatever names/pids you want. Or mix
and match all you want.

If you have a normal developer mind-set this makes sense. You work with
libraries.

However, this is erlang world and not the same, by off-loading to an OTP
appliation (which should of course have its own supervision tree and
start things and you shouldn’t have to care what is actually doing).
That is the beauty of it. You don’t have to write lots of code in your
app to interface with it whereas with a library you are more tightly
integrated.

I think elixir folks have somewhat misunderstood the meaning of an OTP
app. On one hand they do recommend that everything should be OTP apps
and should be started. They even start things which doesn’t have a
supervision tree! On the other hand they don’t recommend actually
starting anything but rather just providing gen_servers which can be
incorporated into your applications supervision tree.

They decided to not include “OTP library applications” but this seems to
be what most elixir developers want but they can’t have so they abuse
normal OTP applications instead.

For this particular thread. If you want a “generic” facebook library
that doesn’t actually handle anything it should have been written
without any HTTP requests at all. Just stateless modules that help you
operate the way you like. You have to provide functions that do the
actual HTTP call or whatever and they give you utility functions. Then
you can choose to implement your own gen_servers and connection pooling
if you want to. Then this should be distributed as an OTP library
application.

Rant over!

[sasajuric]

@cmkarlsson

They decided to not include “OTP library applications” but this seems to
be what most elixir developers want but they can’t have so they abuse
normal OTP applications instead.

You can most certainly write library OTP apps with Elixir. In fact the mix new generator will create a libapp, and you have to pass extra parameter if you want it to generate application callback module and start the supervision tree.

[OvermindDL1]

@cmkarlsson

However, this is erlang world and not the same, by off-loading to an OTP
appliation (which should of course have its own supervision tree and
start things and you shouldn’t have to care what is actually doing).
That is the beauty of it. You don’t have to write lots of code in your
app to interface with it whereas with a library you are more tightly
integrated.

That ‘is’ what the global setup is, it is a normal OTP application running full and on its own.

[peerreynders]

I think @cmkarlsson’s point was that the normal developer mindset (non-erlang/imperative) would by default tend towards accessing additional capabilities via library access - meanwhile the default Erlang mentality would/should lean towards encapsulating capabilities as a separate service to enable looser coupling and to aid fault tolerance.

I think, in a way, this is somewhat borne out by the fact that the default “mix new PATH” creates what Erlang seems to describe as a library application:

A library application that cannot be started or stopped, does not need any application callback module.

To get the full blown deal you have to go out of your way with “mix new PATH --sup”:

A --sup option can be given to generate an OTP application skeleton including a supervision tree. Normally [i.e. by default] an app is generated without a supervisor and without the app callback.

Somehow I would have expected the default to be a full blown application, having to specify something like --lib to get a library application. In the end it doesn’t make one iota of a difference as both options are available but it certainly could be construed to reflect on a certain “mind set” behind the selected approach.

Anyway, my interpretation - any mistakes are my own.

[alexcastano]

@OvermindDL1

You mean very much like how my TaskAfter library works, you can define a global handler and/or custom local handlers.

I really think that style is what most libraries ‘should’ be doing. By default start nothing. If an appropriate config is specified then start a global with that setting. Else start your own and use whatever names/pids you want. Or mix and match all you want.

I think this is the way to go with the Elixir libraries too, because of this I was really confused with the Facebook libraries.

[sasajuric]

@peerreynders

Somehow I would have expected the default to be a full blown application, having to specify something like --lib to get a library application. In the end it doesn’t make one iota of a difference as both options are available but it certainly could be construed to reflect on a certain “mind set” behind the selected approach.

IIRC, this was in fact a default previously, but then it was changed b/c there were bunch of libs published as OTP apps with supervision, when they should have been library applications.

Another way to look at it: if you want supervision tree and app callback mod, you need to opt in. If you don’t know what that is, or not sure if you want it, you probably don’t need it, so it’s better if the generator doesn’t throw it in by default

[cmkarlsson]

I think @cmkarlsson’s point was that the normal developer mindset (non-erlang/
imperative) would by default tend towards accessing additional capabilities via
library access - meanwhile the default Erlang mentality would/should lean
towards encapsulating capabilities as a separate service to enable looser
coupling and to aid fault tolerance.

Yes, that is what I was trying to communicate, perhaps rather poorly
Your words are more accurate.

There are always trade-offs with everything. To abstract everything away
into a separate OTP application some options are taken away from you
like, what and if to connection pool, what http client to use. Instead
you get a loosely coupled and stand-alone module.

With a library you have more control but also have to make some
decisions and work yourself.

My initial post was perhaps also unnecessarily provocatively worded but
I feel that an OTP application boundary is good to have. By integrating
straight into your own application you lose that separation and it also
means you change the way you think and design your application.

I don’t know much about the current facebook applications but it feels
like something that could be a self-contained OTP application providing
a service. How it is implemented is non of my concern .

In rel.config:

[{facebook, [ {api_key, <<...>>}, {url, <<..>>}]}].

application:start(facebook).

facebook:do_stuff(...).

[peerreynders]

@cmkarlsson

By integrating straight into your own application you lose that separation and it also
means you change the way you think and design your application.

Hmmm… monolithic versus service design … and we’re straight back to [Applications are Components (AKA μservices in Elixir)] (Applications are Components (AKA Microservices in Elixir))

[brianlow]

Is either approach easier to test?

With a GenServer I think you can insert a mock to verify how your code interacts with the Facebook server. Any similar options with the lib?

[OvermindDL1]

@brianlow

Is either approach easier to test?

With a GenServer I think you can insert a mock to verify how your code interacts with the Facebook server. Any similar options with the lib?

Overwriting its compiled module in-memory, not at all hard to do, not very async-safe if you are testing the original code too though.