silverdr
Ecto.Schema / Ecto.Changeset - optional, conditionally present fields
I have a – let’s call it ‘Parent’ schema, which embeds_one child schema. Basically a set of largely mutually exclusive key/value pairs. Something like (simplified here - actual schema is far larger):
field :type, :string
field :time, :time
field :dom, :integer
field :dow, :integer
Depending on the value in :type one of the following fields is required, while others are not relevant. I do all the casting, validating, including conditional validate_required in custom validating functions and I end up with a ready to use, valid changeset that gets persisted in a JSON field in the DB.
The problem is that ALL the fields get stored, even if they are not needed. I would like to have only the relevant fields to end up in the DB but at best I get something like:
{
"type": "dow",
"time": null,
"dom": null,
"dow": 1
}
What can I do to have only
{
"type": "dow",
"dow": 1
}
persisted, instead?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Develop your skills with books, videos, and courses.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 10 of 14 Posts
dimitarvp
Instead of embedding a record, can’t you just have that be a free-from singular JSON field whose contents depend on the
typecolumn? That’s what I would do because that would also emulate pretty closely what strongly statically typed languages do with sum types (enums in Rust).silverdr
Theoretically yes, of course. Although
Thanks for an idea though.
dimitarvp
Suspected that the complexity makes this non-viable, shame.
Then I suggest having a number of custom
Ecto.Types whosedumpfunctions skip fields withnilvalues. That seems like an okay-ish compromising solution. Though whether you’ll find the scattering of validation logic to more files beneficial or annoying is up to you.silverdr
Indeed. What I have now works but I feel like I am shovelling in tons of pure “noise” into the DB. Currently SNR in that column is less than 1:10 and possibly getting worse later…
Thank you for an even better idea.
silverdr
I take there’s no way of defining a custom type for the whole schema, is it? So it’s defining each field in an embedded schema as a custom type, right?
dimitarvp
Each composite field, yes. F.ex. if you need to have a column’s content vary wildly if another column has different values than that first column is a prime candidate for a custom type.
Though you can do it field by field as well but that would be an overkill IMO.
garrison
What you’re really looking for here is a “polymorphic embedded schema”, which is something that comes up every so often. I think someone even wrote a library for it, though I’ve never used it.
My personal opinion: If your data is unstructured, just use a map field (JSON column) and don’t worry so much about the schema - that’s what unstructured means, after all!
If your data is structured and you are worried about the schema, then you should probably be using actual tables. That’s what they’re for!
Of course, if you wanted to go in the opposite direction with an even more degenerate solution, I bet you could put the embedded schema in a virtual field and then parse it back out at the end of the parent’s changeset, drop the nulls, and dump it into a real
mapfield. Of course, then your data wouldn’t be structured on the way out of the DB, but you could probably cook up a solution to that too if you really wanted. TheEcto.Typehack is probably better, though.silverdr
So maybe I misunderstood, actually. My
embedded_schemaconsists of many fields of mostly primitive types. And I kind of hoped that I can use that “model” as a custom type, where I would implement the callbacks (especiallydump/1) serialising the map into compact-ed JSON to send it to DB this way. So I guess now what you’re saying is that maybe I need to rethink it and split the current schema into multiple complex custom types, which would get alternatively stored and this way those fields, which are not relevant for given combination of values would not make it to the DB. But that means I would also need to rewrite validations (as you mentioned I think now). No idea yet how though.rathorevk
There’s a library available for defining custom types, which allows you to dump or load values based on those types. It might be helpful for your use case: Parameter — Parameter v0.14.1
silverdr
That probably depends on how we define “unstructured”. It is a rather large configuration structure (which is what I understand as “structured”) with multiple possible combination of elements and rules governing which “slave” fields need to be present for combination of values in some “master” fields. Still I don’t feel like using associated table for it because a) for 1:1 relation that means unnecessarily more complex and less performant queries, and b) the config schema/structure is being actively developed and iterated upon. Therefore I expect many more changes to come, which would mean lots of DB schema migrations with all their possible drawbacks. Doing it on a single map / json column seems like a more suitable approach, especially that in the end the selected, relevant set of configuration fields is always much smaller than the full schema.
That sounds about right