Ecto select() acting strange

Questions / Help
1

According to documentation: Ecto.Query – Ecto v2.2.11

It is also possible to select a struct and limit the returned fields at the same time: from(City, select: [:name])

I have really hard time understanding this. How are those fields limited? I got the feeling the docs are misleading, because I expect, especially looking at SQL generated, this:

iex(10)> from(c in Client, select: [:name]) |> Repo.all
[debug] QUERY OK source="clients" db=5.6ms decode=0.4ms queue=0.3ms
SELECT c0."name" FROM "clients" AS c0 []

Would return this:

[
  %Office.Litigation.Schemas.Client{
    name: "Dean",
  },
  %Office.Litigation.Schemas.Client{
    name: "John"
  }
  ....
]

But because Structs are Structs and there are some rules, it actually returns this:

[
  %Office.Litigation.Schemas.Client{
    __meta__: #Ecto.Schema.Metadata<:loaded, "clients">,
    address: #Ecto.Association.NotLoaded<association :address is not loaded>,
    address_id: nil,
    clients_emails: #Ecto.Association.NotLoaded<association :clients_emails is not loaded>,
    clients_phones: #Ecto.Association.NotLoaded<association :clients_phones is not loaded>,
    company: nil,
    defendant_cases: #Ecto.Association.NotLoaded<association :defendant_cases is not loaded>,
    emails: #Ecto.Association.NotLoaded<association :emails is not loaded>,
    id: nil,
    inserted_at: nil,
    krs: nil,
    name: "Dean",
    nip: nil,
    phones: #Ecto.Association.NotLoaded<association :phones is not loaded>,
    plaintiff_cases: #Ecto.Association.NotLoaded<association :plaintiff_cases is not loaded>,
    surname: nil,
    updated_at: nil
  },
  %Office.Litigation.Schemas.Client{
    __meta__: #Ecto.Schema.Metadata<:loaded, "clients">,
    address: #Ecto.Association.NotLoaded<association :address is not loaded>,
    address_id: nil,
    clients_emails: #Ecto.Association.NotLoaded<association :clients_emails is not loaded>,
    clients_phones: #Ecto.Association.NotLoaded<association :clients_phones is not loaded>,
    company: nil,
    defendant_cases: #Ecto.Association.NotLoaded<association :defendant_cases is not loaded>,
    emails: #Ecto.Association.NotLoaded<association :emails is not loaded>,                                                                         
    id: nil,
    inserted_at: nil,
    krs: nil,
    name: "Alycia",
    nip: nil,
    phones: #Ecto.Association.NotLoaded<association :phones is not loaded>,
    plaintiff_cases: #Ecto.Association.NotLoaded<association :plaintiff_cases is not loaded>,
    surname: nil,
    updated_at: nil
  }
  ...
]

It took me quite a lot of time trying to debug, and until I realized why what I expected is impossible to achieve i simply did Client |> select([client], %{name: client.name}) |> Repo.all()

And now, is it me who has hard time grasping what’s in the documentation? Should documentation be maybe improved somehow? Or am I totally missing something? :frowning:

1 Like
2

select a struct and limit the returned fields at the same time:

This sounds intuitively correct to me. As you said, structs have the same fields always. This is by design: that’s their role. It’s just that using select only populates a subset of the fields. The others just have a nil value (or whatever other default value was specified).

And as you yourself discovered, what you required is also possible with the other syntax.

But then again, any documentation PRs would very likely be welcomed!

1 Like
3

In addition to what @dimitarvp said, you can return just a map of the value like from(c in Client, select: %{name: c.name}) or just return the name straight out as string like from(c in Client, select: c.name).

A Struct, including Ecto’s schema/structs since they are built on Elixir structs, are maps with an absolute specific set of keys (unless it’s broken in which case you’ll get errors on many accesses) along with a metadata key to state the type of struct it is, thus if a struct gets returned then it will always have all the keys with whatever default value they have specified unless otherwise passed in, so for the Ecto Struct in your example it’s like it’s doing return = %Office.Litigation.Schemas.Client{name: name} so name gets set and all the other keys get the default values as specified in the schema. :slight_smile:

4

Yeah, I wrote at the end of post what I did to get desired effect. What got me confused is this part: “It is also possible to select a struct and limit the returned fields”. And to be honest I’m not the only one, my colleague, native English speaker (I’m not) from around the globe pointed out to me that select: [:name] should be enough to just get the name without anything else.

So what I’m really asking is, do you think that docs need some rewording or not? It took me quite some time to realize that those are after all structs, and have to have all the fields, so maybe I’m not the only one? And if docs need some rewording, does anyone have any suggestions? Because I don’t know how to reword them so it’s crystal clear, that limiting fields mean that only those fields will be fetched from DB, but returned struct will still have all fields. The last part should be obvious, and to me now it is obvious. But given that it wasn’t maybe there are others like me?

I want to simply gather opinions, before I blindly make a pull request :wink:

5

Technically the fields are limited, but as Struct’s in elixir are a set size then the other fields just get filled in with defaults. :slight_smile:

Here’s an example of what a bad struct does in Elixir:

iex(1)> defmodule Blah do defstruct a: 1, b: 2 end
{:module, Blah,
 <<70, 79, 82, 49, 0, 0, 5, 200, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 181,
   0, 0, 0, 18, 11, 69, 108, 105, 120, 105, 114, 46, 66, 108, 97, 104, 8, 95,
   95, 105, 110, 102, 111, 95, 95, 7, 99, ...>>, %Blah{a: 1, b: 2}}
iex(2)> %{__struct__: Blah, a: 42}
%{__struct__: Blah, a: 42}
iex(3)> %{__struct__: Blah, a: 42, b: 6.28}
%Blah{a: 42, b: 6.28}

And that’s just for displaying purposes, many actual calls will fail with errors and so forth. Although you ‘can’ construct a bad struct in Elixir, you definitely shouldn’t, it very much assumes a hardcoded key set. :slight_smile:

As long as it mentions that the Schema is a Struct at runtime then that should be good. :slight_smile:
Which based on the very top of the documentation for it at: Ecto.Schema — Ecto v3.11.1

An Ecto schema is used to map any data source into an Elixir struct. The definition of the schema is possible through two main APIs: schema/2 and embedded_schema/1.

(Emphasis mine) It looks like it does, but it could be a bit more clear in stating that the schema also defines that struct, not just maps on to one. :slight_smile:

6

Personally I do think that the docs could be more straightforward on this. After this section of the current docs:

It is also possible to select a struct and limit the returned fields at the same time:

from(City, select: [:name])

I would add an example showing a couple fetched cities, like %City{name: "Honolulu", population: nil} and a quick note that the non-matching struct fields will be set to their default value, which is usually nil.

Also, as another option, instead of:
Client |> select([client], %{name: client.name}) |> Repo.all()

You could do:
Client |> select([client], map(client, :name)}) |> Repo.all()

It’s a little shorter, but if you had three or more attributes it would be significantly shorter and I think it would be more readable as well (since you don’t have to double-check the mappings).

3 Likes
7

Yep but that one is also limited in that you can only select from a single table, I often, at least eventually, end up adding joins (sometimes a lot of joins) so the %{...} syntax handles that. :slight_smile:

Speaking of, I just noticed the new mix phx.gen.embedded! If this does what I think it does then quite convenient!

1 Like
8

No you are not the only one… this also confused me for a while.

This isn’t a surprise for some here, because they are experienced Elixir developers, but will be always an unexpected surprise for newcomers, specially when this not the behavior in the languages the developer is used to work with. So I agree with @axelson suggestion for improving the docs :wink: