Are Ecto embedded schemas useful without a database?

One of my friends once told me that he wants to make every struct in his project an Ecto schema, no matter whether it maps to a database table or column.

I think he has his point. By making a struct an Ecto schema, we have a very good casting and validation mechanism for free.

Maybe I’ll try this idea in my future projects, too.

By the way, I’m facing some very serious problems with embedded schema. Because they map to JSONB columns, I lost the type information of my structs.

So recently I went for the custom struct + Ecto type + Erlang serialization approach.

Have you looked at Polymorphic embed?

Yep, I use it a lot, but still, when the data is highly dynamic and recursive, and you don’t know what will appear where at which time, polymorphic embed does not help.

No, it does not

The project I’m working on has such a data structure like this:

@type val() :: 
  nil |
  number() |
  String.t() |
  boolean() |
  Date.t() |
  DateTime.t() |
  [val()] |
  %CustomStruct1{} |
  %CustomStruct2{} |
  %Foo{val: val()}

If Foo needs to be an Ecto embedded schema, how do you code it?

In the end, I just made my own Ecto.Type that maps such data as a binary using :erlang.term_to_binary/2.

Short answer to your opening question: Yes, I believe they can be.

I’m using regular Ecto.Schema for representing database data, but I do not use those same Ecto.Schema structs for driving the user interface. I represent form data using Ecto embedded schemas and their related Changeset processing functions to represent UI/External API data (“UI data” collectively herein) for the purposes of having a ready made validation framework of that data.

More to the point, I don’t want a tight coupling between UI data and the database model because in many cases what works the best for the database structure isn’t necessarily what works the best for user interactions with that data. From the software design side, I want those separate interests to be clearly independent; this way I’m encouraging form design and form data handling to do what’s best for the user interaction in question and the database related code to do what is best for the information architecture. Yes, there is a point where these two interests must come together, but that translation from UI to data oriented business logic can be handled in a discrete, clear way independent of the direct UI and database related concerns.

In a situation like yours, my sense is you only have one of the two concerns that my project has: dealing with the user interface. So it seems to me is the real question you need to answer is: “Does the Ecto.Changeset methodology, a ready-made, well-known, widely-used system for data validation, bring enough value to the data validation problem to make it a worthwhile pursuit?” For many (maybe most), the answer should probably be “yes”; but I’m guessing on that last point. Otherwise, you need to think what alternative is better/simpler/closer fit for your project.

Ecto schemas are essentially DTOs… You use them to move data from one place to the other (including but not exclusive to databases). The advantage you have over maps is being able to define that information in a more type-safe/ structured way.

IMHO, the first thing that you need to consider before using an Ecto schema is: “Do I need to validate or transform this inbound data”? I use “inbound” here very loosely, as this can mean data coming from external sources (like a user form or an API payload) or data coming from other layers within your application (like context boundaries).

Exactly this. I find embedded schemas particularly useful to define value objects (if you are familiar with DDD terminology).

And this is why I like to think of schemas like DTOs… They can represent database records, but they are much more powerful than this… You can also think about schemas as “windows” that let you peek into a subset of your data. This is extremely helpful for doing things like Command/Query segregation and overall decoupling your data from your domain representation.

Thanks! So Ecto schemas (or any schemas probably) are really useful when you need validations and possibly when you find it convenient for UI model not to map backend data closely. IIRC Phoenix forms can even provide some nicer looking error messages out of the box of there’s schema behind.

What about the relationship between schemas? Associations seem to be particularly difficult to build if there’s just a REST API instead of usual database behind as e.g. you need to load Organization first before setting it for a User. This certainly provides validation which.. you may never need if your app is never going to change which Organization user belongs to.

Then specifically for never-changing associations schema’s associations seem to be not that useful and we could just link to e.g. in-memory Organization loaded a little earlier, correct? Or am I missing something?

More or less, yes. Especially embedded schemas and using schemaless Ecto are both amazing for (1) parsing and (2) validating data coming from the outside and then converting it to neat structs that obey the invariants of your business logic.

That’s really up to you though, you can just make them proper DB-backed Ecto schemas? Embedded schemas work well too but need a little grooming i.e. put_assoc et. al. at times.