Qqwy

Qqwy

TypeCheck Core Team

Specify: Comfortable, Explicit, Multi-Layered Configuration Specifications - v0.7

After two long topics (1 , 2) dedicated to how to best configure your elixir application,
talking with a couple of people at ElixirConf.eu about the current state of configuring applications,
and recognizing some patterns (both good and bad) in the Elixir libraries I and others have written in the last couple of years,

here is my take on the challenge of configuring applications and libraries in Elixir:


logo-text

hex.pm version Build Status

Comfortable, Explicit, multi-Layered configuration specifications.

hex - docs - github

Specify is an Elixir library to standardize the method of doing configuration for your applications and libraries. Rather than trying to force configuration to happen in one way, it enables both library writers and library consumers to specify on multiple different levels how they want to configure something.

The main goals that Specify has in mind are:

  1. Configuration is based on looking at a stack of (also per-specification and globally configurable!) configuration sources, falling back to defaults written in the configuration specification.
  2. Overriding the values in this stack of configuration using plain arguments passed to a function is also always possible, circumventing the ‘singleton problem’ that many configurable things suffer from.
  3. It is very clear what values are supported for a given configuration field, since a validation/parsing function is named when specifying it.
  4. Documentation is automatically generated from a configuration specification, listing names, descriptions, parser functions and default values.
  5. Normalization: After a call tto YourConfig.load you end up with an Elixir struct containing the configuration, with all fields having passed validation and parsing, so you know exactly what fields are there to use and what kind of values to expect. This greatly reduces the number of weird ambiguous errors when something is misconfigured.
  6. Fail-fast: The library raises errors on missing required configuration, things that cannot be parsed and attempting to override unexistent fields. The library raises compilation warnings on missing documentation. The library logs at ‘error’-log-level when a source cannot be reached during an attempted configuration load.

Example

defmodule Cosette.CastleOnACloud do
  require Specify
  Specify.defconfig sources: [Specify.Providers.MixEnv, Specify.Providers.Process] do
    @doc "there are no floors for me to sweep"
    field :floors_to_sweep, :integer, default: 0

    @doc "there are a hundred boys and girls"
    field :amount_boys_and_girls, :integer, default: 100

    @doc "The lady all in white holds me and sings a lullaby"
    field :lullaby, :string

    @doc "Crying is usually not allowed"
    field :crying_allowed, :boolean, default: false
  end
end

Trying to load this using Cosette.CatleOnACloud.load() will fail because a mandatory field is missing.

iex> Cosette.CastleOnACloud.load
** (Specify.MissingRequiredFieldsError) Missing required fields for `Elixir.Cosette.CastleOnACloud`: `:lullaby`.
    (specify) lib/specify.ex:179: Specify.prevent_missing_required_fields!/3
    (specify) lib/specify.ex:147: Specify.load/2

However, by configuring it using

Application.put_env(Cosette.CastleOnACloud, :lullaby, "I love you very much")
# or, in your config/conf.exs file:
config Cosette.CastleOnACloud, lullaby: "I love you very much"

or instead, using the Process Dictionary source:

Process.put(Cosette.CastleOnACloud, :lullaby, "I love you very much")

or specifying an inline lullaby value as part of the overrides: argument:

iex> Cosette.CastleOnACloud.load(overrides: [lullaby: "I love you very much", crying_allowed: true])

we end up with:

%Cosette.CastleOnACloud{
  crying_allowed: true,
  floors_to_sweep: 0,
  lullaby: "I love you very much",
  amount_boys_and_girls: 100
}

Roadmap

Specify is not yet done. Most importantly, it needs some tests and some more effort to make sure that its interface is understandable not only to me but also to the rest of the community :stuck_out_tongue_winking_eye: .
Also, at the very least I still want to create an environment variables provider (both for .env-files and System.get_env).
I am fairly certain that its documentation and examples can be improved to be more understandable. Besides this, I would love to hear answers to the following practical questions:

Does Specify work with how you are used to doing configuration, or does it clash, and in what way?

I have followed along and read through both of afore-mentioned topics so I think/hope that Specify is able to be useful to most common use-cases. Of course, it is only a tool, and definitely is somewhat opinionated.
To make it as generally useful as possible, I very much would like to hear your suggestions/ideas/questions.

What would be the best way to read from environment variables?

Basically, how to deal with the mismatch of Elixir’s snake_case typing and environment variables that usually, but not necessarily, are in CONSTANT_CASE. I am currently thinking of adding additional options to the fields to define an environment variable name (which defaults to the uppercase field name, but is overridable), with an additional option to the defconfig call that will be prefixed to these names, but I am not yet entirely certain this is the best way.

All with all, it was a ton of fun to write Specify (it might have been my most involved macro- and metaprogramming in Elixir to date). Very eager for your feedback!

hex - docs - github


note: starting at v. 0.4.0, Confy has been renamed to Specify.

First Post!

OvermindDL1

OvermindDL1

Took a bit to read it, looks like a fantastic start!

So a few starter questions:

  1. You can make arbitrary configuration loaders and handlers?

  2. What about one that pulls from a database on some kind of update message so it updates the configurations in real time, is there a way to tell something like a gen_server built on this to accept this new configuration (say like changing the pool size in a database pool)?

  3. How do you differentiate between the different stages of configurations? I.E. Compile-time, Load-Time, and Run-Time (dynamic)?

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

Starting today, Confy has been renamed to Specify to be more clear in the main focus of the library, which is to make fields and field-types explicit, regardless of whether we are talking about app-wide configuration, user-created settings or single-function-call options. :slightly_smiling_face:

Specify v.0.4.0 has been released, which replaces Confy v.0.3.0.

Also, what is better than a beautiful library? A beautiful library that has its own logo!
logo-text_25percent

OvermindDL1

OvermindDL1

I’d require the provider to handle that. For PostgreSQL for example you can LISTEN in it to get updates pushed for example.

The GenServer could then decide to switch over to the new configuration, stop and start a new version of itself with the new configuration, or potentially even recompile a module if, for optimization reasons, the compilation process itself is affected by the configuration.

The fact the OTP runtime doesn’t already have something that sends an update message to registered receivers when an application environment value changes is still a bit astounding to me… ^.^;

Qqwy

Qqwy

TypeCheck Core Team

Version 0.6 has been released!

Since the last update on the forum, the following improvements have been made:

  • 0.5 - Adds the nonnegative_integer , positive_integer , nonnegative_float , positive_float and timeout builtin parsers.
  • 0.6 - Adds the mfa and function builtin parsers.

I have one question to the community at large, which is about parsers like the function one. Currently there is no way to restrict a field expecting a function to a particular arity. What would be a good API for this?

I am thinking about two possibilities (but maybe there is a third as well):

  • field(field_name, {:function, arity}, default: some_val)
  • field(field_name, function(arity), default: some_val)

Currently, we already have {:list, :element_parser} (e.g. {:list, :integer}). It would be nice to keep a similar format, but I am not entirely convinced on the former format’s structure. :thinking:

As for the roadmap: Currently the only thing that I’d consider blocking before v 1.0 would be the possibility to nest specifications. (And besides this resolving above issue, I guess).

Last Post!

Qqwy

Qqwy

TypeCheck Core Team

Version 0.7 has been released!

This version adds:

  • The possibility to make configuration source optional, which is rather nice because Specify warns when you do not fill in any fields in a particular configuration source. In certain cases it makes sense that a particular configuration source only exist some of the time (like setting optional system environment variables), which would make the warnings unneccesarily noisy.
  • To make it easier to specify the default sources in e.g. the Mix.Config/Elixir.Config files where no dependencies (and thus not Specify either) are loaded yet, it now is possible to pass a source that you want to configure as {SourceModule, %{some: field, some_other: field}} which will be turned into a %SourceModule{some: field, some_other: field} struct on startup. A module/function/argument-syntax now is also provided, for even more flexibility.

Where Next?

Popular in Announcing Top

grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 54260 488
New
wmnnd
Hi there, for my project DBLSQD, I needed a file storage solution that is a bit more flexible than Arc. Because I thought others might f...
New
pkrawat1
Hey guyz We at @aviabird are working on a payment library in elixir/phoenix. We are targeting March 2018 to add 56 Gateways to it. Have...
New
tmbb
I’ve been working on two packages (not on hex.pm yet) to build admin interfaces for phoenix apps: bureaucrat - which contains a bunch ...
New
mathieuprog
Hello :waving_hand: Allow me to introduce you to Tz, an alternative time zone database support to Tzdata. Why another library? First a...
New
mplatts
With HEEX released we decided to start a components library using Tailwind CSS - check it out here: Petal Components. We also have a boi...
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New

We're in Beta

About us Mission Statement