What is the difference between using {:system, "PORT"} and System.get_env("PORT") in deployment?

At first glance, I have to admit that I’m not sure I like this approach.

I feel that supporting something like Phoenix.start_link(endpoint_plug_module, other_params) would be a much better choice. The benefit of this is that it’s explicit, very flexible, and you can make it code-reload friendly by wrapping the invocation inside your own module. Also, it’s consistent to how other types of processes are started, so it doesn’t introduce a special case.

But the most important benefit of that approach is that I feel it sets a good example. Every now and then, I come across a library that requires me to set the configuration, where a simple parameter passed to start_link would suffice. I feel that, at least to some extent, this style is promoted by Phoenix and Ecto, both of which mandate a config setting and a dedicated module.

Notice that I’m not against using app env in “leaf apps”, i.e. our own custom projects that depend on Phoenix, Ecto, and others. But I do feel that the vast majority of libs shouldn’t require app envs at all. The only exception that comes to mind is Logger, which is a global/singleton kind of library, so I guess having it require configuration is fine in this case.

2 Likes

You cannot make it code reload friendly with the API above. The endpoint is a supervisor and we need to redefine the supervision tree in case the user wants to start listening to a new port or change any stateful configuration. And in order to do so with code reloading, it is only possible if you do so inside init. Any value you pass through start_link will be permanently sticked and not code reloadable.

If you take a step back and ask: where is the best way to configure a supervisor so I can change its children under code reloading, the answer is going to be on init. This is very aligned with the child_spec discussion happening on elixir-lang-core right now.

I believe though it is still worth debating if we should rely less on the application environment, although the need of per environment configuration is almost a given for Ecto and Phoenix. Calling Ecto.start_link(Repo) instead of Repo.start_link() can also be an improvement, so we stop injecting start_link everywhere, but will likely become irrelevant if the child_spec proposal is accepted, since nobody will call Repo.start_link directly anymore and then we will be free to move it elsewhere.

1 Like

I’m not very familiar with code reloading, so maybe I’m missing something, or I have too naive thinking. But I’ll try anyway :slight_smile: Let’s say that I have something like:

defmodule MyEndpoint do
  def start_link, do: Phoenix.Endpoint.start_link(...)
end

Wouldn’t it be enough to produce appup file which would reload the MyEndpoint module, and then instruct the parent supervisor to restart the child?

I definitely think we should rely less on the app env. One thing bothering me is that if I want to start an endpoint during tests, I need to setup some app envs. Even Phoenix itself is hurting from this problem, because it has to manually configure a test endpoint. I had to do the same in our Elixir client for Phoenix channels (see here). This is clumsy, and it’s not immediately obvious how am I supposed to do it. Being able to do something like Phoenix.start_link(...) would be much nicer.

I’m not sure why do we need user’s repo module at all? Why can’t we just Ecto.Repo.start_link(...)?

When it comes to Phoenix, users still needs to provide the endpoint module, but that module could be a pure plug in most cases (one exception being code reloading, in which case you might introduce start_link as I argued above).

Another benefit of this approach is that you can easily run dynamic number of instances. For example, I can start multiple repos as instructed by end-user (we’re actually building such kind of a system at my company). When it comes to endpoint, I could have a single template endpoint plug module, and dynamically start/stop actual web servers depending on the user interaction. These cases are arguably more exotic, but they showcase how dropping app env gives us more flexibility.

Again, just to be clear, as an end-user of Phoenix or Ecto, I would likely use app env to configure my endpoints/repos. However, I do believe that libraries themselves should not enforce that usage.

2 Likes

The database adapter does some code generation in the repo. There are also some named ets tables in the repo (called after the repo module) used for the query cache. There is work being done to remove those limitations, but it’s not there yet.

3 Likes

What I wanted to say is that I feel that whatever is currently accomplished with MyRepo should be achievable without that module. Not really familiar with internals of Ecto, so perhaps I’m missing some fine-print details.

Happy to see that some work is already being done here :thumbsup:

2 Likes

No because changing how the supervisor is started does not change any supervisor that is currently running. Similar to a GenServer, if I upgrade the GenServer module, any GenServer currently running will have the old state and need to go through code_change. The code_change callback for supervisor is also init.

I believe this would be easily achievable on both. But it still means any compile time option needs to be given elsewhere, like on use Phoenix.Endpoint, with the remaining options given to start_link. Putting them on the environment allows you to not care if it is compile or runtime.

1 Like

Maybe I’m missing something here, or we’re talking past each other. In my proposal, the parent of the endpoint knows nothing about the configuration. It only knows that the endpoint is started via MyEndpoint.start_link/0. So if you reload the MyEndpoint module, and then tell its parent to restart the child, I believe that should work properly, since the new version of MyEndpoint.start_link/0 will be invoked, and that new version will pass the changed endpoint options.

I’m assuming here that we need to restart the endpoint if the listening port is changed. My understanding is that this is true for your proposal as well. Or maybe you envision a scenario where port can be somehow dynamically reconfigured without taking the endpoint down?

But it still means any compile time option needs to be given elsewhere

What compile time options are there? Preferably, we should have as few of such as possible, because they are not really flexible.

2 Likes

Can you write some pseudo code showing the supervision tree starting from the application callback? Because as far as I understand in your proposal, if you have this code:

# In the application tree
worker(MyEndpoint, [])

# In MyEndpoint
def start_link do
  Phoenix.start_link(__MODULE__, [http: [port: System.get_env(...)]])
end

Then you can only reload the options if you force start_link to be invoked again, which implies terminating the whole Phoenix.Endpoint supervision tree and starting a new one.

However, if the configuration is moved inside init, then we never need to bring the whole Phoenix.Endpoint supervision tree down, we can reload the tree without bringing any undesired process down, as the granularity is inside its own init. In other words, as you move the configuration up, the harsher the reload will have to be.

I am arguing that the best place to load dynamic configuration in any OTP service is inside init. And that’s what we want to push as best practice.

1 Like

Oh, that’s precisely what I thought - we need to restart the entire endpoint to apply the changed network port.

I guess that you’re suggesting that you can do better by just restarting a part of the endpoint subtree. That’s fine, but I still wonder what does it mean in practice? Can you get away by not restarting cowboy processes? If not, then the question is which processes would in fact survive a network port change.

I don’t think it’s as simple as that. It’s a tradeoff because that approach makes default usage more complex, and my feeling is that most people are not using code reloading anyway. So now we end up with a new, and IMO more complicated mechanism of configuring to support a less likely case of “I want to change the listening port and restart as few processes as possible”. Notice that changing endpoint options is the only case we’re discussing here. If you want to upgrade the endpoint plug, a router, a controller, or a view, AFAICT it should work normally.

Of course, the case you mention is still valid, so it would be nice if Phoenix would make it possible. Having a callback like you suggest, maybe renamed to dynamic_configure or something like that, would be nice. But I wouldn’t want to see this as the primary method of configuration, because it’s more complex and rigid, and imposes some constraints (e.g. it’s hard to start a dynamic number of endpoints then). Another variant is to accept a configuration callback module as the (optional) parameter. If provided, Phoenix would always invoke that module to get the option value.

I guess my general point is that we should consider introducing special solutions and recommendations for people that want to do fine-grained code reloading, while keeping the straightforward usage for default cases (which also covers some variants of code reloading).

3 Likes

I disagree with this because we already have a simple and general purpose mechanism that works on all services. init is available in GenServer, Supervisor, GenStage, Phoenix, Ecto, etc. Why introduce something especial if we already have something consistent and simple that works everywhere?

Furthermore, I don’t even buy the proposed approach is simpler: what is the difference in asking developers to implement MyEndpoint.start_link/0 instead of implementing MyEndpoint.init/1?

1 Like

Sorry, I missed this earlier. They are very few on both Ecto and Phoenix side. They are documented in Ecto.Repo and Phoenix.Endpoint modules (and we are working on reducing them).

2 Likes

I think we’re dealing with multiple issues here so my main goal is kind of diluted. AFAIK we actually don’t have a simple mechanism for Phoenix (or Ecto), because options are driven by hardcoded app env settings. I’d like to see that disappear :slight_smile:

You’ve pointed out that this has some problems with code-reloading, so we’ve now diverged to some more complex corners (which are nonetheless still important). Regardles of that, my point still stands: I’d like to see pure Phoenix.Endpoint.start_link and Ecto.Repo.start_link which take options as function parameters, even if that approach won’t work for (all) code-reloading cases.

MyEndpoint.start_link is not my main proposal. It was an idea of how I could still keep some support for code reloading with pure Phoenix.Endpoint.start_link. Those who don’t want code reloading don’t need to write their own modules, and can use Phoenix.Endpoint.start_link directly. This is IMO as simple and as straightforward as it gets, and it will be good enough for most cases.

I’m not saying that MyEndpoint is the best idea. Perhaps, as I said in the previous message, passing a late-binding configuration module as an (optional) parameter would work better. Or perhaps the callback, such as the one you propose would work better. In fact, maybe the callback which you propose is good enough for all cases as long as that callback is always invoked, instead of being configured through config.exs.

Either way, one deficiency of the current design and the new proposal is the lack of support for dynamic endpoints. Can I dynamically spin off endpoints or Ecto repos without needing to define a dedicated module? Currently the answer is no (unless I compile a module at runtime), and this limitation remains with the new proposal. It’s arguably a less likely case, but IMO so is the case of “I want to change the listening port and restart as few processes as possible” :slight_smile: I think that we could support both cases, but not with the current proposal.

2 Likes

I am afraid that if you look at problems in isolation, then you can come up with specific solutions to those problems, but once you look at all problems together, the individual solutions fall apart or become considerably harder to implement.

For example, adding Phoenix.Endpoint.start_link goes against the compile-time compilation and it promotes bad practices in terms of code upgrades, so there is little interest in going in this direction. Once we add the fact 99% of the Phoenix applications rely on multiple environments, reading from the environment as a default is the best choice. The cost of having to write to the application environment when running tests is a small indirection compared to the benefits.

The dynamic endpoint discussion though is somewhat orthogonal to this and I agree with you. Both Ecto and Phoenix should allow starting the endpoint multiple times, especially after we allow dynamic configuration to happen on init. It is mostly a matter of removing references based on the mod, which is static, and make them be based on the name. Ideally we will continue to store the defaults in app env with dynamic values in init.

1 Like

This is why I was asking about compile-time options. Is there something that really requires compile-time configuration? And if there is, does it necessarily imply how we deal with all parameters?

I never argued against that in the end-user application. Phoenix project generator could still generate the code along the lines of Phoenix.Endpoint.start_link(Application.fetch_env!(:my_app, :endpoint_config)).

We’re still retaining the same set of features and properties for end-users, but Phoenix becomes more flexible than it currently is.

Another important benefit, as I initially mentioned, is that by doing this, Phoenix would promote what I believe is a good practice for libraries: take your options as parameters, not as config values under some magical key. The latter is more confusing (at least for me), and less flexible.

I fail to see those benefits :slight_smile: Arguably the only one is that the approach you propose makes it easier to reconfigure the port and restart as few processes as possible. Such feature could still be achieved with pure Phoenix.Endpoint.start_link and a callback which is always invoked.

I guess after this discussion, I think that what actually bugs me is the fact that I need to configure whether the callback is invoked. If the callback was always invoked (and app env wasn’t mandated by the Phoenix lib), then I’d be fine with it.

So maybe something along the lines of Phoenix.Endpoint.start_link(callback_mod, arg) would also work. Again, the Phoenix project generator could generate the init/1 callback definition which would by default read options from app env. How do you feel about that idea?

If options always have to be provided through config.exs, it’s still going to be confusing. I really dislike that, and don’t see the need for it.

2 Likes

That’s exactly my point. If we are going to do that 99% of the time, I would rather do it by default.

You say Phoenix becomes more flexible by explicitly reading the environment but it really doesn’t, it is the exact same. If you disagree, can you explicitly tell me what you would be able to achieve by moving the configuration out that we can’t today? You say it is more flexible, so please try to provide concrete examples.

Otherwise the only change I see is that it no longer requires writing to put_env in 1% of the cases (like tests and single-file projects) while forcing everyone else to read it from the app environment. And just to clarify, it is not this change that makes dynamic endpoints possible. As I said in the previous message, dynamic endpoints is orthogonal to where you put the default configuration.

Except, as I have argued extensively on this thread, you don’t want pass your options as parameters, but rather define or read them from init, especially if you rely on multiple environments and code loading.

So from my perspective: the application environment is the best place for configuration given the features a Phoenix application requires. If your concern is about libraries copying what Phoenix does even if they don’t have the same requirements as Phoenix, then we should promote better education and not have Phoenix stop using those features correctly. Otherwise we could make the same argument for meta-programming: Phoenix should not use meta-programming, even if it uses it correctly, because other developers will use it in cases they don’t need to.

If you still think passing those options through start_link is the best way to go, we will have to agree to disagree.

Yes, unfortunately init/1 is already taken by Plug so we had to resort to the callback approach. If you have better options in mind, we will be glad to hear them, but it needs to keep today’s semantics of being called on the new process with semantics similar to init. Promoting the use of options through start_link is something we want to discourage for all of the reasons mentioned on this thread, even if you don’t seen to value those reasons the same way that we do (which is fine).

1 Like

Sorry, one last amendment:

The make it explicit, the benefits are:

  • We push everyone to work with multiple environments (without forcing all applications to read from the application environment)

  • We provide a unified place where we can specify configuration that is required at compile-time and configuration that is needed only at runtime

  • Provides good practices when it comes to hot code reloading

Because I feel that we are walking in circles, besides providing examples of where you think your approach is more flexible, please try to provide some examples on how you would configure a Phoenix endpoint considering at least the first two bullets above: compile-time configuration and multiple environments. Your solution of reading configuration on init works only for the latter. You can find a list of compile-time options on the Phoenix.Endpoint docs.

1 Like

I was refering to the current style, which is not flexible enough. At my company we’re fetching some part of configuration from a json file. Previously, we did it from etcd. So the way Phoenix works was not good enough for us, and we had to do some shenanigans during app startup, and update app env to make it work. The same thing holds for Ecto. In other words, in all of our Elixir apps we’ve already had a couple of cases where app env was not a good approach for us. So that definitely sucks, and this is why I think taking options through parameters is more flexible. Because with that I can fetch values in arbitrary ways.

Admittedly, the new proposal amends this issue for Phoenix. But it does it in what I find a strange and a confusing way. With the new proposal, when reading the code, I’ll see something like:

def load_from_system_env(opts) do
  port = System.get_env("PORT") || raise "expected the PORT environment variable to be set"
  {:ok, Keyword.put(opts, :http, [:inet6, port: port])}
end

The issue I take with this is that when I read this code, it’s completely unclear when (and if at all) is it invoked. I need to know particularities of Phoenix, and need to be aware of configuration for each specific env. I find this confusing. I’m also worried that other libs will start to use this pattern.

I think I understand your concerns about code reload, but the points I argued are:

  • It is my impression that most people are not doing it.
  • It can still be done for many cases when options are passed as args.
  • Even for endpoint reconfiguration it can be done, though it does require restarting of the entire endpoint (but not the rest of the system).

So this leaves us with the solution that is optimized for the case of “I want to reconfigure the endpoint with restarting as few things in it as possible”. I’m not going to say this is not a valid case, but I haven’t seen it in practice myself. In contrast, I’ve seen multiple cases where app config is making Phoenix (and Ecto) usage more complex than it could be.

Now, if the callback function was always invoked, I’d like it better, because then Phoenix becomes more like a behaviour, so it’s consistent to the other existing abstractions.

You could consider decoupling the callback module from the behaviour module. I actually dislike the fact that the Endpoint conflates the plug module and the process behaviour.

As I explained I had cases when this doesn’t suffice, and then I needed to resort to some trickery. This is also going to be the case for dynamic endpoints, because I have to devise unique key names, and manually delete them when I’m taking the endpoint down.

The explicit Application.fetch_env in the client code is not much more typing, it can be created by the generator. It is more flexible because you can replace it, or combine it with something else. It still promotes multiple environments. And it reduces the magic, because it’s now clear to a reader where do the settings come from (currently you need to know that Phoenix will read the setting from the key which has the same name as the endpoint module).

I already tackled multiple envs (explicit Application.fetch_env call inserted by the generator). My first take on compile time configs would be to question if they are really needed. If we can turn them into runtime settings it would be great. If not, then this remains an open question.

Does this mean that my GenServers shouldn’t accept options as parameters anymore? Should everybody write a more complex code (and configuring through init is IMO more complex), just because some day they might want to do code reloading?

Note that I’m not saying that Phoenix (or other libs) shouldn’t support proper code reloading. I do however feel that there’s a big difference between saying “If you want this, you need to do that” and “You should always do it like this because you might need it someday”. This is IMO especially relevant in the context of code reloading, which is itself difficult to get right, and should be reserved for very special cases.

3 Likes

Idle-thoughts:

I’m not really seeing how moving options to init/1 would solve child_specs because the options to start_link/3 are already passed there as well, like what if the name changes, or timeout, or spawn_opt or so. What if a supervisor could indeed know to just restart a single child if that child’s spec changed instead of restarting the whole supervisor and its entire children tree? What if child_spec’s were, say, stored in an ets process by the code_loader instead of being passed to Supervisor.start_link/2 somehow? That way the supervisor knows that a specific child changed because its specific childspec changed, so it can kill that sole child and restart it with its specific new spec (well code_change maybe could handle that as well without even needing to restart the child, maybe a spec_change or start_change callback for things that can be handled without an entire restart, defaulting to restart if not handled?). This way if the childspec for, say, a specific GenServer has its, say, spawn_opts: changed then just that genserver is killed instead of that entire supervisor, then restarted with its new spec. It is doable but definitely changes how such things are stored, no more child_specs passed to the Supervisor.start_link/2, have to register them some other way (attributes maybe? but what about dynamic child_specs…)…

Eh, just idle thoughts. ^.^

1 Like

Assume there will always be at least one configuration that is compile time. We cannot make the adapter in Ecto non-compile time until Ecto 3.0 which I don’t plan to force on developers any time soon. There is a handful of compile-time options in Phoenix. In the last months we have removed many compile time configurations but assume some will stay or require a long time to be removed. If you don’t like the assumption, then my advice is to give a try at removing the compile-time options. :slight_smile:

I want to point out you had to resort to some trickery before init was introduced. And, as you said above, init solves the problems you had with configuration (although you also said it is in a confusing way - which I agree). So this is not related to reading from the application env or not.

And again, reading from the environment doesn’t impact dynamic configuration because the unique keys are based on the module name and you always need at least a module name. There is no reason for Phoenix nor Ecto to require process names (although they do today and fixing that is basically the work remaining to support dynamic endpoints).

So my questions still remain:

  1. Please provide an example where not reading from Application.get_env actually makes it possible to do something that is not possible in master. I don’t think there is such example, so it is just a matter of preference.

  2. Please describe a system that can take into account multiple environment and compile-time configuration. Assuming compile-time options do not exist is not a viable option for Phoenix nor Ecto. I am not even including the code reloading requirements.

Unless the second problem can be solved, reading from the application will continue to be the best option. That’s exactly why I said you can solve each independently, but please try to consider a solution that solves all of them.

No, my reply was in the context of Phoenix and Ecto in particular. While I still think init is the best place, regardless of code reloading, anyone can do whatever they want. :slight_smile:

2 Likes

I have a hunch you are confusing the inits, because the supervisors work exactly as you said: it restarts a single child if that child’s spec changed instead of restarting the whole supervisor and its entire children tree. So the question you need to answer is how to make the supervisor notice that the child_spec change and that’s done by running the supervisor’s init again. However, if the child_spec configuration is elsewhere, the supervisor will not notice the change.

Since I have already explained how this is a problem a couple times on this thread, if you still can’t see the issue, I would recommend you to play a bit with code reloading and supervisors and explore what can and what can’t be done, as I believe I ran out of ways to explain the issue. :slight_smile:

4 Likes