ffloyd

The Problem

Currently, if I define a struct in the following way:

defmodule MyStruct do
  # Both x and y will have the FIXED values until next recompilation
  defstruct x: :rand.uniform(100), y: :rand.uniform(100)
end

I have defaults calculated at compile-time. This means that if I do %MyStruct{} or struct!(MyStruct), I will have the same default values for x and y until the next recompilation. This behavior is explicitly explained in Elixir’s documentation.

This compile-time evaluation has a “cascade effect” throughout the ecosystem. The most notable example is Ecto’s default option in schema field definitions. Another example is Oban structured jobs (their argument schema definitions). Such behavior likely exists because these libraries rely on defining struct defaults under the hood, or simply repeat the existing pattern.

Often, you don’t need defaults evaluated at runtime. After initially creating a schema or struct, a lot of code can be written before the first runtime default (e.g., start_of_period: DateTime.utc_now()) is needed. And that’s the point when the real problem starts:

In the case of a simple struct:
- A developer needs to introduce a function like new()
- … and change all the places where the struct is created via %Struct{...} to use this function
- … and find all the places where the struct is created via struct!(mod_name) where mod_name is a variable that could theoretically be defined as an atom in config.exs, which makes such updates tricky.
In the case of an Ecto schema:
- In addition to what’s needed for a simple struct, a developer needs to update every changeset used for creation
- Alternatively, they can rely on autogenerate, but this only creates values just before insertion, so no business logic can rely on that default before the record is persisted
- For virtual fields, autogenerate simply does not work, and the developer must track each place where data is read from the database in order to populate virtual fields with dynamic defaults.

In summary: it may require an inadequate amount of work for introducing a mere dynamic struct field default.

The Motivation

Recently, I analyzed a 2M+ line Elixir production codebase from the perspective of compilation determinism. This allowed me to automatically detect errors like default: DateTime.utc_now(). The reality is that even with documentation stating that such values are compile-time, people make this mistake frequently. This makes me think that “defaults are evaluated at compile-time” is not a feature anticipated by developers, but rather a trap that leads to weird bugs. Especially if the developer is new to Elixir.

Also, the fact that defining a pre-calculated and dynamic default requires completely different approaches seems weird to me.

The Proposal

I believe that one of the traits of a convenient programming language is this principle:

Things with similar semantics should have similar syntax.

For example, making things private in Elixir is done by adding p to a definer: defp, defmacrop. Another example is defining named “callables”: runtime callables are defined using def and defp, while compile-time callables are defined using defmacro and defmacrop. This consistent approach reduces cognitive load when working with the language.

So I propose:

defmodule MyStruct do
  defdynstruct x: :rand.uniform(100), y: :rand.uniform(100)
end

The same syntax rules apply, but each time such a struct is created (via struct/1, struct!/2, etc.), the defaults are recalculated. Note: making the %MyStruct{...} literal syntax work with dynamic defaults requires compiler changes—this is discussed in the Implementation section. If you want some values to be pre-calculated and others to be dynamic, you can still do it:

defmodule MyStruct do
  @default_x :rand.uniform(100)
  defdynstruct x: @default_x, y: :rand.uniform(100)
end

This will allow libraries like Ecto to change default behavior to support dynamic values. If a developer needs the previous compile-time behavior, they can use module attributes (as shown with @default_x above) to cache values at compile time.

The Implementation

I tried to fix it without patching Elixir. Below is what I implemented, and it works for most cases:

StructRuntimeDefaults module

defmodule StructRuntimeDefaults do
  @moduledoc """
  Provides runtime evaluation of default values for Elixir structs and Ecto schemas.

  The only thing it cannot handle is when `%StructName{...}` syntax is used for defining a value:
  Elixir compiler replaces it with the struct literal at compile time.

  Such calls should be avoided in favor of `struct!(StructName)` and `struct!(SomeName, ...)`.
  It's _okish_ to do it only in production code and leave tests and test factories to use `%StructName{}` syntax
  when diverse values are not important for testing.

  Such usage is easier to detect and refactor than finding all code paths that create the struct
  (e.g., via `struct/2`, `%StructName{}`, `Repo.get/3`, etc).

  !!! __Use this approach only when other approaches are insufficient!__ !!!

  ## Important Warning

  This implementation relies on Elixir and Ecto internals by overriding `__struct__/0`,
  `__struct__/1`, and `__schema__/1`. While these can be considered "mostly stable" interfaces, they are
  implementation details that could theoretically change in future versions.

  The Elixir compiler can occasionally miss auto-recompilation of modules that depend on the
  modified struct/schema module. After adding or changing runtime defaults, consider running
  a full recompilation if you notice that the changes are not taking effect.
  This problem has happened in the scope of test factories at least once.

  Static default values always take precedence over runtime defaults.

  ## Usage for Simple Structs

      defmodule MyStruct do
        use StructRuntimeDefaults

        defstruct other_data: [], current_time: nil

        struct_runtime_defaults(%{current_time: DateTime.utc_now()})
      end

  ## Usage for Ecto Schemas

      defmodule MySchema do
        use Ecto.Schema
        use StructRuntimeDefaults

        schema "my_table" do
          field :created_date, :date
          timestamps()
        end

        ecto_schema_runtime_defaults(%{created_date: Date.utc_today()})
      end
  """

  defmacro __using__(_ \\ []) do
    quote do
      import unquote(__MODULE__), only: [struct_runtime_defaults: 1, ecto_schema_runtime_defaults: 1]
    end
  end

  defmacro struct_runtime_defaults(defaults) do
    quote do
      # These are the Single Source of Truth (SSOT) for struct creation in Elixir
      # https://github.com/elixir-lang/elixir/blob/31905ca1364c8b841b49f68b7d2cbacbacb7de02/lib/elixir/lib/kernel.ex#L5626
      defoverridable __struct__: 0
      defoverridable __struct__: 1

      def __struct__ do
        Map.merge(super(), unquote(defaults))
      end

      def __struct__(kv) do
        original = super(kv)
        runtime_defaults = unquote(defaults)

        Map.merge(original, runtime_defaults, fn
          _key, nil, rt_def_val -> rt_def_val
          _key, orig_val, _rt_def_val -> orig_val
        end)
      end
    end
  end

  defmacro ecto_schema_runtime_defaults(defaults) do
    quote do
      struct_runtime_defaults(unquote(defaults))

      # Ecto schema support
      #
      # Ecto uses __schema__(:loaded) to initialize structs when loading from DB
      # it's critical to override this to apply runtime defaults on load when _virtual fields_ are involved.
      # https://github.com/elixir-ecto/ecto/blob/2bdbcb6a2c3022ae931ccb9c3e1920596a2da68a/lib/ecto/schema/loader.ex?plain=1#L12
      #
      # Unfortunately, overriding __schema__/1 was not enough at the time of writing because Ecto
      # relies on its own metadata about fields to set defaults and did not rely on __struct__/0 or __struct__/1 in some cases.
      defoverridable __schema__: 1

      def __schema__(atom) do
        case atom do
          :loaded ->
            original = super(:loaded)
            runtime_defaults = unquote(defaults)

            Map.merge(original, runtime_defaults, fn
              _key, nil, rt_def_val -> rt_def_val
              _key, orig_val, _rt_def_val -> orig_val
            end)

          _ ->
            super(atom)
        end
      end
    end
  end
end

Working on this allowed me to understand how dynamic defaults can be achieved. In the current Elixir implementation, the __struct__/0 and __struct__/1 functions are the Single Source of Truth for struct creation. However, AST with default values is evaluated at compile-time. If we change it to be evaluated dynamically inside __struct__(...), we will achieve the desired behavior. The module above proves that it works. With one exception though: the %Struct{...} syntax.

When the Elixir compiler sees %Struct{...} in the code, it runs __struct__(...) at compile-time and puts the calculated value directly into the final code. This behavior cannot be overridden without modifying the Elixir compiler itself. It means that it’s not possible to make universal “dynamic default values for structs” without changing the Elixir compiler.

I’m happy to try creating a PR for Elixir, but I need a green light from the community and core team. I also need alignment on the desired syntax.

So, implementing it for Elixir looks like merely defining defdynstruct that

mostly works like defstruct, but
places default values evaluation inside __struct__
and marks such structs to not be inlined as calculated literals by the compiler

I expect low performance cost, a small amount of code for implementation, and no impact on existing Elixir codebases.

Next Steps

Before proposing this to the Elixir core team mailing list, I want to collect community feedback. I’m specifically seeking input on:

Syntax: do you like proposed defdynstruct syntax?
Missing use cases: Does this solve the problem? Do you see situations where proposed approach is not enough?
Missing implementation concerns: What potential issues or edge cases are missed in the proposal? I found %Struct{} syntax, are there others?
Compatibility: Are there any compatibility concerns that I missed?

Let me know your thoughts! And if you simply like the idea, like the post to show support. Thank you!

40 comments

/oban #ecto #structs #default

1 1062 40

2025-11-10 12:39:14 UTC

Showing Posts 40 to 31

Show Best Posts
Show All Posts (oldest first)
Show All Posts (newest first)

ffloyd

Agree. So I made a library as an outcome of the idea and discussion here. Let me know WDYT about it!

https://github.com/ffloyd/better_struct

(will do post in libraries-related part of this forum after getting initial feedback and publishing on hex).

Post #40

a-maze-d

You sparked a really good discussion. Thanks for that.

The fact that default: DateTime.utc_today() is resolved at compile time actually also a surprise of a similar level.

True, but that is really part of the language. It’s like the @tag that also gets evaluated at compile time.

You are right that you could learn this too. The difference is that, once you have learned ot, you don’t get surprises, whereas for your structs I would need to check the focumentation every time. This is the kind of surprise I’m talking about.

As a language you need to be conservative when adding new features to the language (consider the Change Event Horizon). Hence we ahould ask ourselves how common your problem is, and whether it could be solved on a different level before solving it on the language. Here some options I could see:

Solve the problem on a database level, e.g. at database insertion/retrieval time (and I think that would be cleanest, because you solve the problem locally)
Add a feature to Ecto that allows the initialization of default/calculated values. Maybe an _init: &myinit/1 field in your struct?

Even though I might be against your proposal, I very much appreciate you taking the time to raise it. As more use cases surface, the more likely it is that a solution might happen on a language level.

Side note:

If even creators of Ecto (and ExMachina) fell into the trap

It’s up to them to decide that. As demonstrated above there are plenty of ways on how you can extend keep things extendable. And at the end it’s always a) a balancing act and b) a learning process

Post #39

ffloyd

I feel you. Such refactoring wasn’t painful for me until I started working on 2M+ LOC Elixir codebase. It’s one of the biggest (if not the biggest) production monolith written in Elixir. In any other codebase I touched, this problem was in “meh, annoying!” category, not “it’s too long to refactor”.

100% percent agree! That’s the problem I’m trying to actually solve! Bear with me:

The reality I see is that %Schema{} syntax is used to “create struct with defaults”. My strongest proof that even Ecto does it and proposes it. If Ecto were enforcing using Schema.build(...) as the idiomatic way to create a schema instance - it would have dynamic defaults already.

I believe that allowing static defaults in struct definition was a design mistake (that was done for Erlang records, structs merely repeat it). If setting static default were require making a build/new function - Ecto would have build/new functions enforced.

It made me think about making %Schema{} a full-featured constructor. But it violates %A{} == %A{}, so, not a good way.

My latest proposal is not about %MakeMeDynamic{} anymore.

Not exactly. What I’m fighting with right now is the idea that static and dynamic defaults should be implemented in different ways. Ideally, all my problems would be solved by:

Remove ability to set static defaults. Any kind of defaults should require implementing new/build function.

But it’s too late for this proposal.

Should I make a proposal to Ecto then? To make Ecto create build function when defining schema and suggest people to use it over %Schema{} syntax if they need any kind of defaults? It’s a breaking change. How would you estimate chances that people agree on this? (Considering that in the most cases lack of dynamic defaults are not super painful)

Not every Elixir developer uses Ecto. And the design mistake of not enforcing usage of build/new when a struct can have dynamic defaults has a good chance to be repeated in other libraries.

That’s why I want to solve this problem on the language syntax level. It’s more bulletproof solution than relying on traditions and conventions.

My latest proposal is

ffloyd:

What about having a slightly different data-type then: dynamically created struct. It will work just like normal structs, support struct!(...), protocols, etc. The only difference is that you define it using defdynstruct and cannot create them as literal, you must use something like %MyStruct() and %MyStruct(collectable) to instantiate them?

Also, %MyStruct(...) should work for normal structs also.

(Yes, we already have struct! and can use this instead of %MyStruct(...), but it feels like worse UX to me).

Then:

Static defaults via defstruct:

part of a struct definition

guaranteed to be applied

introducing the first static default does not require refactoring

Dynamic defaults via defdynstruct:

part of a struct definition

guaranteed to be applied (you cannot do %MyStruct{})

introducing the first dynamic default requires refactoring, but more trivial one:

in the struct it’s a mere replacement of defstruct with defdynstruct

in the code you own it’s a mere replacement of %MyStruct{...} to %MyStruct(...)

in libraries: you only need to switch to using struct!(...) in places where you work with user-provided structs and avoid do %unquote(struct_mod){} hacks in macro.

Post #38

mudasobwa

Creator of Cure

There are none, if we use dependent types. Sorry for nitpicking, but inventing new wheels for a wheelchair, like ad-hoc validations. would never cover SMT-solving theories. Ad-hoc validations are extremely hard to validate in the wild, unlike a proof in languages like Idris. That’s basically why I started Cure effort.

Post #37

mudasobwa

Creator of Cure

FWIW, the aforementioned estructura provides coerce/3 which calculates (also coerces and validates btw) everything because it goes through Access which is amending the calculated fields. Example.

Post #36

Sorc96

These are exactly my thoughts. This entire thread wouldn’t be needed if constructors were the default way to create structs.

I’m convinced that it’s perfectly fine to have constructor functions that only pass the data into a struct. Some will become more complex later, others will stay trivial, but the changes will never spread outside of the module.

Post #35

tfwright

I agree with everyone who has said that “it takes to long to refactor” is not a very compelling argument. It would be nice if there was a more “blessed” constructor pattern though. It would be great to know that if I am initializing a struct I can/should always call `new` or whatever. Even better if it came with a way to prevent “private” fields from being set somehow…

Post #34

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Yes. For me the split I’m observing comes down to people who feel that code which calls functions should be a function call, and this proposal where doing literal syntax is actually sometimes a way to call functions dynamically. You can tell from this framing which side I’m on I suppose lol.

But more seriously some of the objections to using new such as the time required to convert code base don’t hold water. Sure we are talking about dynamic data now but what about invariants? Should struct literal syntax also run validators on the values? The answer is: no, we already crossed this bridge with changesets. (And no the type system won’t solve this, there are invariants not amenable to types).

If we had a convention around new or build then this would be a non issue. You wouldn’t have 100 spots in your code to refactor you’d have 100 function calls and you make one change to the function. This is already what you have when using changesets.

Post #33

christhekeele

Could throw together a custom credo check that warns on %Struct{...} when Struct.new/build is detected, perhaps. A community consensus on new vs build could be handy as well.

Post #32

ffloyd

That’s a right question. For simplicity let me provide one related example:

We have shared queries - because domains want to preload things defined in other domain. Or sql-join with them. While subquery is defined inside domain A, it becomes part of the query in domain B and now domain B should also take care of initializing virtual field defaults. As result I need to put dynamic defaults in every domain that generates my struct indirectly via query.

And we can easily have dozens of functions that creates your struct directly or indirectly in a single domain of our big monolith.

Post #31

Load more posts (30 remaining)