The Plug.init/1 method gets called during the compile step and not during the startup like I thought, is there someway to update a plugs config when the router supervisor tree is being setup?
Background info: All of our applications/servers/api:s connect to a configuration server during startup and loads their config at that point, without the config the applications will not work since the configuration contains things like “How are the authentication rows formatted in redis”. The configuration can change depending on which datacenter the application will end up in so we can’t tell the application this during the compile step or by using environment variables.
Now, this was a easy problem to solve for most of our elixir apps, just add the configuration genserver to the top of the supervisor tree, it loads the config and the rest of the genservers can query it in their init/1 calls. Our only problem is the Plugs init/1 functions which are called during the compilation and we don’t have access to the configuration at that point. The config is a deeply nested, large (200 000 rows) map which means asking for the configuration rows each request might be too expensive.
We can of course just extract the plug functionality into a genserver but I would prefer to avoid the extra step if we could avoid it.
You can add a separate plug in the beginning of your “plug pipeline” to assign the necessary values into :private or :assigns fields, which can then be accessed by other plugs down the line.
This separate plug would act like the init callback in a genserver.
That would still make a config request for each incoming web request, right?
Besides, I would prefer to not have a dependency between the different plugs. If we add plug that requires some config information we would have to add that config key to the “Setup configuration” plug.
That would still make a config request for each incoming web request, right?
I would imagine that you would request the config at the start up of your web app, and save it into either a module with :code.load_binary/3 (if the config won’t change for the life time of the app) or into a genserver (if it will change). The separate plug would then request those.
That was what I was afraid of. I think we will just add a backing genserver for each plug. If the plugs behaved like genservers then we would not have had this problem but perhaps the the current plug behaviour gives you some other benefits.
Plugs are functions, basically … so they can’t really behave like genservers.
Why would you add a genserver for each plug though? Wouldn’t one be enough? It would be a bit more efficient (less message passing between processes, the webserver’s and your genservers’).
But I would probably go with the :code.load_binary/3 approach (you can see how it works in the fastglobal library), because calling genservers might result in a bottleneck.
Why would you add a genserver for each plug though? Wouldn’t one be enough?
Each api can have different plugs enabled so we can’t have a large genserver that loads the configuration for every one of them.
For example, we have these two plugs which both need access to configuration values that we get during the app startup.
Is this api enabled plug: The config can enabled/disable certain api:s.
Authenticate: this one need information about which redis server it should talk to, how the redis key is formatted and which keys from redis should be loaded for this particular api.
The CORS headers needs to be configurable by the security team and loaded through the configuration.
because calling genservers might result in a bottleneck.
tnx @LostKobrakai, so I guess it’s not possible to configure a plug at runtime, the solution seems to configure the call to a plug (every time) instead of just once on init.
I’m not sure what you expect. You either have compile time configuration (init/1) or runtime (call/2). “Once on init” is just a variation of “retrieve it every time” in that the caller queries the config from any kind of runtime cache like e.g. the app environment or some started processes instead of recalculating the config from more expensive sources. There are ways of compiling data into actual code (see e.g. https://github.com/discordapp/fastglobal) but that’s that’s the edge-case solution not the norm.
well I guess I expect the “runtime cache” to exist within the Plug.
to me init means setup the plug once initially, so not necessarily at compile time, but preferably at runtime using environment variables.
then call means use the configured settings that are already in place.
I would like to not only read values from environment variables into settings but also validate those settings as soon as the application starts (once). it seems “init” would be a nice place to be able to do that, just like Repo.init/2 and Endpoint.init/2.
The distinctive difference between a plug and ecto/endpoints is that a plug doesn’t start a process for itself where it could store that config in. Both ecto and your phoenix endpoint do that. You could use MyAppWeb.Application.start to retrieve your config and put everything into the application environment and in the plug’s call/2 read from Application.get_env.
thx @LostKobrakai - I’m not very satisfied with that approach, but I guess there’s no way around it, unless you’re willing to (re)write your own plugs.
