Configuring Plugs during runtime

Gyllsdorff

The Plug.init/1 method gets called during the compile step and not during the startup like I thought, is there someway to update a plugs config when the router supervisor tree is being setup?

Background info: All of our applications/servers/api:s connect to a configuration server during startup and loads their config at that point, without the config the applications will not work since the configuration contains things like “How are the authentication rows formatted in redis”. The configuration can change depending on which datacenter the application will end up in so we can’t tell the application this during the compile step or by using environment variables.

Now, this was a easy problem to solve for most of our elixir apps, just add the configuration genserver to the top of the supervisor tree, it loads the config and the rest of the genservers can query it in their init/1 calls. Our only problem is the Plugs init/1 functions which are called during the compilation and we don’t have access to the configuration at that point. The config is a deeply nested, large (200 000 rows) map which means asking for the configuration rows each request might be too expensive.

We can of course just extract the plug functionality into a genserver but I would prefer to avoid the extra step if we could avoid it.

13 comments

#plug #how-to-question

0 5059 13

2018-08-12 14:48:00 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

idi527

You can add a separate plug in the beginning of your “plug pipeline” to assign the necessary values into :private or :assigns fields, which can then be accessed by other plugs down the line.

This separate plug would act like the init callback in a genserver.

Post #1

Gyllsdorff

That would still make a config request for each incoming web request, right?

Besides, I would prefer to not have a dependency between the different plugs. If we add plug that requires some config information we would have to add that config key to the “Setup configuration” plug.

Post #2

idi527

That would still make a config request for each incoming web request, right?

I would imagine that you would request the config at the start up of your web app, and save it into either a module with :code.load_binary/3 (if the config won’t change for the life time of the app) or into a genserver (if it will change). The separate plug would then request those.

I don’t quite get your second point …

Post #3

Gyllsdorff

That was what I was afraid of. I think we will just add a backing genserver for each plug. If the plugs behaved like genservers then we would not have had this problem but perhaps the the current plug behaviour gives you some other benefits.

Post #4

idi527

Plugs are functions, basically … so they can’t really behave like genservers.

Why would you add a genserver for each plug though? Wouldn’t one be enough? It would be a bit more efficient (less message passing between processes, the webserver’s and your genservers’).

But I would probably go with the :code.load_binary/3 approach (you can see how it works in the fastglobal library), because calling genservers might result in a bottleneck.

Post #5

Gyllsdorff

Why would you add a genserver for each plug though? Wouldn’t one be enough?

Each api can have different plugs enabled so we can’t have a large genserver that loads the configuration for every one of them.

For example, we have these two plugs which both need access to configuration values that we get during the app startup.

Is this api enabled plug: The config can enabled/disable certain api:s.
Authenticate: this one need information about which redis server it should talk to, how the redis key is formatted and which keys from redis should be loaded for this particular api.
The CORS headers needs to be configurable by the security team and loaded through the configuration.

because calling genservers might result in a bottleneck.

Agreed.

Post #6

i-n-g-m-a-r

@Gyllsdorff did you find a solution to configuring plugs at runtime?

Post #7

LostKobrakai

See this post:

Post #8

i-n-g-m-a-r

tnx @LostKobrakai, so I guess it’s not possible to configure a plug at runtime, the solution seems to configure the call to a plug (every time) instead of just once on init.

Post #9

LostKobrakai

I’m not sure what you expect. You either have compile time configuration (init/1) or runtime (call/2). “Once on init” is just a variation of “retrieve it every time” in that the caller queries the config from any kind of runtime cache like e.g. the app environment or some started processes instead of recalculating the config from more expensive sources. There are ways of compiling data into actual code (see e.g. GitHub - discord/fastglobal: Fast no copy globals for Elixir & Erlang. · GitHub) but that’s that’s the edge-case solution not the norm.

Post #10

Load more posts (3 remaining)