but aren’t applications from my deps() now automatically handled? I don’t specify anything
rawkode
0
rawkode
OvermindDL1
If you do that it will not be included in releases.
The library itself really should already have a way I’d think though… Not heard of that library though.
0
sasajuric
Author of Elixir In Action
Unfortunately, this is a quite nasty problem. Your runtime dependencies are started before your app, so the workaround I mentioned won’t work for an app which requires some config value during startup (in my defence, I did say it won’t work for everything 
).
This is IMO a bad decision on behalf of library authors, but that conclusion doesn’t solve your problem.
The way I’d approach this myself is, I’d likely create a bash script which would fetch the secret from wherever and either perform a search/replace in sys.config, or set OS env, and see if I can make it work with distillery (which I think can be done, but not sure).
2
rawkode
This is IMO a bad decision on behalf of library authors, but that conclusion doesn’t solve your problem.
The bad decision being to require configuration to start? I’d like to understand how I can prevent this with my own libraries, and how I can perhaps contribute to KafkaEx and avoid this problem for others.
KafkaEx is a client that speaks to a Kafka server, if the server isn’t defined when it’s started, then what would it do until it is?
Can’t I tell Elixir/mix that I’ll start the application myself? This does feel like a problem Elixir should be coping with, rather than looking to library authors; but perhaps this is due to my misunderstanding of it all.
0
OvermindDL1
It should just not start then, it should supply a supervisor that you would add to your own supervision tree with whatever options you want to give it.
It should not be an ‘application’ type library unless it is truly standalone, otherwise it should be a supervised type library.
3
rawkode
rawkode
Should anyone find this thread through similar problems, I found the following:
#OTP API
def start(_type, _args) do
max_restarts = Application.get_env(:kafka_ex, :max_restarts, 10)
max_seconds = Application.get_env(:kafka_ex, :max_seconds, 60)
{:ok, pid} = KafkaEx.Supervisor.start_link(Config.server_impl, max_restarts, max_seconds)
if Application.get_env(:kafka_ex, :disable_default_worker) == true do
{:ok, pid}
else
case KafkaEx.create_worker(Config.default_worker, []) do
{:error, reason} -> {:error, reason}
{:ok, _} -> {:ok, pid}
end
end
end
I’m testing to see if disable_default_worker solves this 
1
sasajuric
Author of Elixir In Action
What @OvermindDL1 said. A good example of this is Ecto. It requires app env (which I dislike), but it doesn’t connect to the repo when it starts. Instead, you insert repo in your own supervision tree where/when you want to. Therefore, the trick I mentioned initially will work with Ecto, but not with KafkaEx.
The decision to start the connection in :kafka_ex application, as opposed to your application is IMO a bad decision in general (i.e. regardless of the configuration issue). It would be better if the connection process was in your supervision tree, so you can have the complete control of its lifecycle.
There is a thing called included applications. Try setting runtime: false, add the app to included_applications, and then start the app manually from your start callback.
I remember reading some criticism of included apps (though can’t remember what was it about), and my takeaway was that there are some non-obvious pitfalls there, so included apps should better be avoided. Didn’t really research this a lot, so maybe I’m wrong (perhaps someone else can chime in).
A clean solution IMO would be if KafkaEx would give you the start_link function in some module, which would allow you to start the connection when you want to. Since that would happen after your app start is invoked, you’d be able to set the app env as I’ve explained.
2
bitwalker
Leader
Regarding the included_applications bit:
Using it means you are responsible for managing the full lifecycle for the included application - additionally, only one application in the tree can include a given application (i.e. app a and b can’t both have c in their included_applications). This should be self-evident (how can two applications manage the startup of some other application), but it definitely can cause confusion. I haven’t tested what happens if a depends on b, and contains c in it’s included_applications, while b also depends on c but has it in it’s applications list. Presumably nothing good happens there, and you’d need to take over both b and c via included_applications, but as I mentioned, I don’t know for sure.
More often than not, all you really want is to set the start type of a given application to :load, start it post-configuration phase, and that’s that. You can do this pretty easily with Distillery, but I don’t think it’s possible via mix.exs currently AFAIK (or perhaps that’s what extra_applications is for in 1.4+, I’d need to double check). Again though, if you have a dependency which depends on that app being started, you would likely need to set it’s start type to :load as well, but I haven’t tested this (haven’t actually encountered this problem yet).
In any case, it’s not that there is something wrong with using included_applications, it’s just that the other approach is better and less error-prone.
1