What is the reasoning behind baking in the adatper at compile-time

[LostKobrakai]

@D4no0

As far as I know there is no tooling to distinct between the two for library creators, so if your library supports only compile-time config, someone who has no idea will just declare a runtime config, only to find out later that his runtime config did nothing. Add to that the fact that you just started with elixir, and you have a recipe for a mess.

There is tooling to differentiate the two. It‘s ˋApplication.get_envˋ vs. ˋApplication.compile_envˋ. The latter will make startup fail if it detects the runtime config value to be different to the value used at compile time.

[D4no0]

@felix-starman

So if I’m understanding correctly, it’s not necessarily that you need a particular repo module to be able to swap adapters at the drop of a hat, or for different use-cases running in the same VM, but that you need the ability to avoid the maintenance headache of never being able to just use MyApp.Repo, and not even put_dynamic_repo because if they’re different adapters, they’re likely different modules. And even if you did ship different releases for different DB adapters, you might be dealing with constant confusion.

This is the exact problem.

@felix-starman

If shipping a my_app_pg.tar and my_app_sqlite3.tar would be too much of a pain

Since I will be using the same repo for all the business logic, whether I am doing this at compile-time or runtime makes no real difference. We are running CI tests for both types of database, this could be as well extended on dev machine, at the end of the way we are dealing with sql databases that have a lot of similarity.

@felix-starman

Here’s some example code that adapted from the a branch of double I had been working on that builds shims.

Thanks! I will definitely take a look, as I’ve never used things like persistent_term before.

@LostKobrakai

There is tooling to differentiate the two. It‘s ˋApplication.get_envˋ vs. ˋApplication.compile_envˋ. The latter will make startup fail if it detects the runtime config value to be different to the value used at compile time.

Ah, I’ve had that error before! I didn’t remember for sure what were the circumstances. The other question is: is it possible to disable/overcome this error in cases where you want to provide a separate route for compile-time configuration and runtime?

@dimitarvp

Hit me up if you start it. I had the same idea but my hands are super full lately. But could spare an hour or two here and there to do reviews or contribute.

I think the implementation is of less importance, I would rather discuss on how to design this thing to be universally useful and to not redefine the concept of config. I’ve seen a couple config libraries that would go the full custom config road, that IMO is not that useful.

[LostKobrakai]

@D4no0

The other question is: is it possible to disable/overcome this error in cases where you want to provide a separate route for compile-time configuration and runtime?

You can provide full paths (instead of just root level keys) with Application.compile_env. So you can be as granular with “tainting” configuration as compile time as needed. You cannot use one and the same path to a config to be compile time and allow it to change at runtime still though. I’d argue that’s a good thing though, as really if there’s a compile time and runtime dependency working differently they should be configured using differenent configs.

[D4no0]

@LostKobrakai

You cannot use one and the same path to a config to be compile time and allow it to change at runtime still though. I’d argue that’s a good thing though, as really if there’s a compile time and runtime dependency working differently they should be configured using differenent configs.

Aha, nice, this makes a lot of sense! Thanks for the insight! For some reason I was thinking that was not the case initially.

[dimitarvp]

@D4no0

I would rather discuss on how to design this thing to be universally useful and to not redefine the concept of config

Same here, even after your clarification. There have been a good amount of attempts, we should definitely look into what others have done as well.

But again and again, I keep coming back to “strong static typing would have prevented 20 fix for real this time commits that are only done to trigger CI”.

[D4no0]

@dimitarvp

There have been a good amount of attempts, we should definitely look into what others have done as well.

Indeed, some of the libraries I’ve seen have just set the boundaries too high (for example type validation of ENV variables), since that was either too hard or impossible to achieve with how the current config works, they just created their own abstraction.

@dimitarvp

But again and again, I keep coming back to “strong static typing would have prevented 20 fix for real this time commits that are only done to trigger CI”.

Not only that, the fact that you can pass for some reason random keys to config of an application and it doesn’t care is just begging for trouble.

I’ve created a repository where we can collaborate and sent and invite to you, the name and namespace don’t matter currently, as it will be migrated later. The repo currently contains a few markdown documents drafts where an analysis can be done on what features are needed and what other libraries are doing, feel free to update/change the structure for a more fitting one.

[D4no0]

To avoid further off-topic on this thread. I will just make a small conclusion based on answers given:

1. The Repo public API is generated based on adapter (thanks @al2o3cr for the insights). The adapters are a generalized concept that can have different functions implemented. If this option would have the ability to be set at runtime, we would lose all the compile-time guarantees we have currently with our custom Repo functions;
2. There are potential hacks as pointed @felix-starman, however they include some dangerous functionality that can create some hard to debug issues;
3. There is also the option of changing the adapters at compile-time and create different builds. This option is one of the most non-intrusive, however if you are using a single repo module in your application, the only guarantee of something not breaking when switching adapters is to have tests suites that will run for all adapters separately.

This was a very fruitful discussion, thanks for everybody’s input!

[joey_the_snake]

This would be quite dangerous since not all databases support the same functionality, might behave differently for common functionality or might have functions named the same that do different things. So you will have to run tests on both if you are being serious. At which point you lose the benefit you are seeking