xlive

xlive

Hey there!

We are using Ecto to parse some API response (it works pretty well!).
We want this response to be converted to a struct, so we first do all the needed validations and at the end we do an apply_action(changeset, :parse).
To do so, we would like to have a module that defines an embedded_schema like the following (we won’t save this):

defmodule MyApp.Response do
  use Ecto.Schema

  @primary_key false
  embedded_schema do
    field(:custom_field, :any)
    ...
  end
end

The “problem” comes with this :any type. When compiling, Ecto complains that only virtual fields can be defined with the :any type.
Here the error:

== Compilation error in file lib/my_app/api/response.ex ==
** (ArgumentError) only virtual fields can have type :any, invalid type for field :custom_field
    (ecto 3.11.1) lib/ecto/schema.ex:2012: Ecto.Schema.__field__/4
    lib/my_app/api/response.ex:36: (module)
    (ecto 3.11.1) lib/ecto/schema.ex:2241: Ecto.Schema.__embeds_module__/4
    lib/my_app/api/response.ex:32: (module)

Is there an explanation of why they must be virtual?

In our use case, this field can have different types (boolean, integer, string, etc.), so we need this flexibility here.

We think that, as embedded_schemas are just saved as blobs or, like in our case, not saved at all, this doesn’t make a lot of sense. But maybe someone has a better understanding about this.

Thanks in advance! :purple_heart:

Showing Posts 1 to 7

sodapopcan

sodapopcan

I’m assuming this is for similar reasons as to why you can’t use nil for embeds_many—which came up again recently—that embeds are meant to be treated the same as regular relations. I think your only option if you want to keep using Ecto is to create your own type. Otherwise you could check out Drops which is a more general schema library.

cevado

cevado

You can use a schemaless changeset

{%MyStruct{}, %{custom_field: :any}}
|> Ecto.Changeset.cast(%{"custom_field" => "1"}, [:custom_field])
|> Ecto.Changeset.apply_changes()
cevado

cevado

But now actually answering that part, and expanding on what @sodapopcan . Understand that althought schemas and changesets are just mapping abstractions. both schema and embedded schemas are expected to be at some point dumped to a database.
Since that is not your intention, i’d recommend to use schemaless changeset(you can use that with structs or plain simple maps).
in case you need features like nesting, relations and some more complex stuff that doesn’t work with schemaless changeset i’d do something like:

@default [virtual: :true]
schema "" do
  field :custom_field1, :any, @default
  field :custom_field2, :any, @default
  ...
end
xlive

xlive OP

Thank you all for answering here!!
@cevado yes.. Actually this variable will be inside an assoc of the main struct
I will try then to find a better way to check these types :folded_hands:
Maybe Drop is a nice candidate! Thx @sodapopcan !!

al2o3cr

al2o3cr

What happens if you pass virtual: true to the field call like the error message suggests?

LostKobrakai

LostKobrakai

With :any ecto cannot be sure it can encode the data for storage in whatever db driver is in use. Therefore it is only allowed for virtual fields, which are not persisted anyways and therefore do not run into that problem.

Adzz

Adzz

Ecto can work for some inputs I found I quickly outgrew it and wrote GitHub - Adzz/data_schema: Declarative schemas for data transformations. · GitHub too. If you need to map data that isn’t essentially a map or a list then ecto schemas aren’t the solution imo.

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews