ffloyd

ffloyd

Runtime-calculated default values for structs

The Problem

Currently, if I define a struct in the following way:

defmodule MyStruct do
  # Both x and y will have the FIXED values until next recompilation
  defstruct x: :rand.uniform(100), y: :rand.uniform(100)
end

I have defaults calculated at compile-time. This means that if I do %MyStruct{} or struct!(MyStruct), I will have the same default values for x and y until the next recompilation. This behavior is explicitly explained in Elixir’s documentation.

This compile-time evaluation has a “cascade effect” throughout the ecosystem. The most notable example is Ecto’s default option in schema field definitions. Another example is Oban structured jobs (their argument schema definitions). Such behavior likely exists because these libraries rely on defining struct defaults under the hood, or simply repeat the existing pattern.

Often, you don’t need defaults evaluated at runtime. After initially creating a schema or struct, a lot of code can be written before the first runtime default (e.g., start_of_period: DateTime.utc_now()) is needed. And that’s the point when the real problem starts:

  • In the case of a simple struct:
    • A developer needs to introduce a function like new()
    • … and change all the places where the struct is created via %Struct{...} to use this function
    • … and find all the places where the struct is created via struct!(mod_name) where mod_name is a variable that could theoretically be defined as an atom in config.exs, which makes such updates tricky.
  • In the case of an Ecto schema:
    • In addition to what’s needed for a simple struct, a developer needs to update every changeset used for creation
    • Alternatively, they can rely on autogenerate, but this only creates values just before insertion, so no business logic can rely on that default before the record is persisted
    • For virtual fields, autogenerate simply does not work, and the developer must track each place where data is read from the database in order to populate virtual fields with dynamic defaults.

In summary: it may require an inadequate amount of work for introducing a mere dynamic struct field default.

The Motivation

Recently, I analyzed a 2M+ line Elixir production codebase from the perspective of compilation determinism. This allowed me to automatically detect errors like default: DateTime.utc_now(). The reality is that even with documentation stating that such values are compile-time, people make this mistake frequently. This makes me think that “defaults are evaluated at compile-time” is not a feature anticipated by developers, but rather a trap that leads to weird bugs. Especially if the developer is new to Elixir.

Also, the fact that defining a pre-calculated and dynamic default requires completely different approaches seems weird to me.

The Proposal

I believe that one of the traits of a convenient programming language is this principle:

Things with similar semantics should have similar syntax.

For example, making things private in Elixir is done by adding p to a definer: defp, defmacrop. Another example is defining named “callables”: runtime callables are defined using def and defp, while compile-time callables are defined using defmacro and defmacrop. This consistent approach reduces cognitive load when working with the language.

So I propose:

defmodule MyStruct do
  defdynstruct x: :rand.uniform(100), y: :rand.uniform(100)
end

The same syntax rules apply, but each time such a struct is created (via struct/1, struct!/2, etc.), the defaults are recalculated. Note: making the %MyStruct{...} literal syntax work with dynamic defaults requires compiler changes—this is discussed in the Implementation section. If you want some values to be pre-calculated and others to be dynamic, you can still do it:

defmodule MyStruct do
  @default_x :rand.uniform(100)
  defdynstruct x: @default_x, y: :rand.uniform(100)
end

This will allow libraries like Ecto to change default behavior to support dynamic values. If a developer needs the previous compile-time behavior, they can use module attributes (as shown with @default_x above) to cache values at compile time.

The Implementation

I tried to fix it without patching Elixir. Below is what I implemented, and it works for most cases:

StructRuntimeDefaults module
defmodule StructRuntimeDefaults do
  @moduledoc """
  Provides runtime evaluation of default values for Elixir structs and Ecto schemas.

  The only thing it cannot handle is when `%StructName{...}` syntax is used for defining a value:
  Elixir compiler replaces it with the struct literal at compile time.

  Such calls should be avoided in favor of `struct!(StructName)` and `struct!(SomeName, ...)`.
  It's _okish_ to do it only in production code and leave tests and test factories to use `%StructName{}` syntax
  when diverse values are not important for testing.

  Such usage is easier to detect and refactor than finding all code paths that create the struct
  (e.g., via `struct/2`, `%StructName{}`, `Repo.get/3`, etc).

  !!! __Use this approach only when other approaches are insufficient!__ !!!

  ## Important Warning

  This implementation relies on Elixir and Ecto internals by overriding `__struct__/0`,
  `__struct__/1`, and `__schema__/1`. While these can be considered "mostly stable" interfaces, they are
  implementation details that could theoretically change in future versions.

  The Elixir compiler can occasionally miss auto-recompilation of modules that depend on the
  modified struct/schema module. After adding or changing runtime defaults, consider running
  a full recompilation if you notice that the changes are not taking effect.
  This problem has happened in the scope of test factories at least once.

  Static default values always take precedence over runtime defaults.

  ## Usage for Simple Structs

      defmodule MyStruct do
        use StructRuntimeDefaults

        defstruct other_data: [], current_time: nil

        struct_runtime_defaults(%{current_time: DateTime.utc_now()})
      end

  ## Usage for Ecto Schemas

      defmodule MySchema do
        use Ecto.Schema
        use StructRuntimeDefaults

        schema "my_table" do
          field :created_date, :date
          timestamps()
        end

        ecto_schema_runtime_defaults(%{created_date: Date.utc_today()})
      end
  """

  defmacro __using__(_ \\ []) do
    quote do
      import unquote(__MODULE__), only: [struct_runtime_defaults: 1, ecto_schema_runtime_defaults: 1]
    end
  end

  defmacro struct_runtime_defaults(defaults) do
    quote do
      # These are the Single Source of Truth (SSOT) for struct creation in Elixir
      # https://github.com/elixir-lang/elixir/blob/31905ca1364c8b841b49f68b7d2cbacbacb7de02/lib/elixir/lib/kernel.ex#L5626
      defoverridable __struct__: 0
      defoverridable __struct__: 1

      def __struct__ do
        Map.merge(super(), unquote(defaults))
      end

      def __struct__(kv) do
        original = super(kv)
        runtime_defaults = unquote(defaults)

        Map.merge(original, runtime_defaults, fn
          _key, nil, rt_def_val -> rt_def_val
          _key, orig_val, _rt_def_val -> orig_val
        end)
      end
    end
  end

  defmacro ecto_schema_runtime_defaults(defaults) do
    quote do
      struct_runtime_defaults(unquote(defaults))

      # Ecto schema support
      #
      # Ecto uses __schema__(:loaded) to initialize structs when loading from DB
      # it's critical to override this to apply runtime defaults on load when _virtual fields_ are involved.
      # https://github.com/elixir-ecto/ecto/blob/2bdbcb6a2c3022ae931ccb9c3e1920596a2da68a/lib/ecto/schema/loader.ex?plain=1#L12
      #
      # Unfortunately, overriding __schema__/1 was not enough at the time of writing because Ecto
      # relies on its own metadata about fields to set defaults and did not rely on __struct__/0 or __struct__/1 in some cases.
      defoverridable __schema__: 1

      def __schema__(atom) do
        case atom do
          :loaded ->
            original = super(:loaded)
            runtime_defaults = unquote(defaults)

            Map.merge(original, runtime_defaults, fn
              _key, nil, rt_def_val -> rt_def_val
              _key, orig_val, _rt_def_val -> orig_val
            end)

          _ ->
            super(atom)
        end
      end
    end
  end
end

Working on this allowed me to understand how dynamic defaults can be achieved. In the current Elixir implementation, the __struct__/0 and __struct__/1 functions are the Single Source of Truth for struct creation. However, AST with default values is evaluated at compile-time. If we change it to be evaluated dynamically inside __struct__(...), we will achieve the desired behavior. The module above proves that it works. With one exception though: the %Struct{...} syntax.

When the Elixir compiler sees %Struct{...} in the code, it runs __struct__(...) at compile-time and puts the calculated value directly into the final code. This behavior cannot be overridden without modifying the Elixir compiler itself. It means that it’s not possible to make universal “dynamic default values for structs” without changing the Elixir compiler.

I’m happy to try creating a PR for Elixir, but I need a green light from the community and core team. I also need alignment on the desired syntax.

So, implementing it for Elixir looks like merely defining defdynstruct that

  • mostly works like defstruct, but
  • places default values evaluation inside __struct__
  • and marks such structs to not be inlined as calculated literals by the compiler

I expect low performance cost, a small amount of code for implementation, and no impact on existing Elixir codebases.

Next Steps

Before proposing this to the Elixir core team mailing list, I want to collect community feedback. I’m specifically seeking input on:

  1. Syntax: do you like proposed defdynstruct syntax?
  2. Missing use cases: Does this solve the problem? Do you see situations where proposed approach is not enough?
  3. Missing implementation concerns: What potential issues or edge cases are missed in the proposal? I found %Struct{} syntax, are there others?
  4. Compatibility: Are there any compatibility concerns that I missed?

Let me know your thoughts! And if you simply like the idea, like the post to show support. Thank you!

Most Liked

LostKobrakai

LostKobrakai

Tbh I disagree with the idea of Repo.insert(%Reminder{}) needing to support dynamic defaults as well. Elixir is very much about being explicit. If you need your data to come from side effects I expect the code to tell me. %Reminder{} |> with_default_target_datetime() |> Repo.insert() is a lot better code than Repo.insert(%Reminder{}) automatically messing around with struct fields.

You’re argueing that struct defaults being a compile time thing prevent libraries from introducing dynamic field defaults. I’d counter that with dynamic defaults being something you want to have an explicit api for and once you do there’s no longer any limitation. I don’t expect ecto ever wanted to provide you with a dynamic defaults feature to begin with.

In the end your argument comes down to “I want to introduce dynamically calculated default values without changing my codebase” – not a great argument imo, such a change should be made explicit by changing code.

LostKobrakai

LostKobrakai

I don’t think we’ll find any alignment here. I still stand by the fact that structs should behave like native datatypes, while you seem to want to make them even more a special case to what they already are and partially out of necessity. I see %Struct{} as a notation to define data and data doesn’t execute code. Yes one can change the definition of the struct, but generally people try to avoid having the before and after of those definitions meet.

Execution is btw another footgun to your proposal. %Struct{} executing code means different runtime behaviour. It’s no longer “O(1)”, but could start an http request, fail somewhere on the path and raise some arbitrary exception crashing a whole process – just from %Struct{}.

christhekeele

christhekeele

Could throw together a custom credo check that warns on %Struct{...} when Struct.new/build is detected, perhaps. A community consensus on new vs build could be handy as well.

Last Post!

ffloyd

ffloyd

Agree. So I made a library as an outcome of the idea and discussion here. Let me know WDYT about it!

https://github.com/ffloyd/better_struct

(will do post in libraries-related part of this forum after getting initial feedback and publishing on hex).

Where Next?

Popular in Proposals: Ideas Top

sevensidedmarble
All the pieces are there to execute arbitrary JS commands from the server. You can put them on a data property and call them from the cli...
New
7rans
I implemented Access behavior for a struct today. Pseudo-code… defmodule MyStruct do defstruct data: %{} @behaviour Access # ... ...
New
pejrich
I propose adding compact_map/2 to the Enum module. What is it? Sometimes you want to map over a collection, but sometimes you want to ma...
New
Astolfo
When generating a release with phx.gen.release --docker a debian version is used by default to avoid “DNS issues” on Alpine. It seems li...
New
virinchi_cv
The Problem Phoenix 1.8 comes aggressively coupled with Daisy UI, a decision which many developers in the community have had mixed feelin...
New
manhvu
In a large repo, working with module need to add alias too much is quite annoyed and not good for organizing code. I think better add su...
New
engineeringdept
In 2026 double submit/session tokens are no longer necessary to prevent against CSRF attacks. Instead, we can use the Sec-Fetch-Site head...
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

We're in Beta

About us Mission Statement