_toni

_toni

Ecto does not return a list for an aggregate function (via fragment)

Hello :wave:

Given the following commerce-like DB context:

Item: { id: integer, model: string ...}

Shop: { id: integer, name: string ...}

# many-to-many table to store what Item's each Shop sells

ShopItem: { shop_id: integer, item_id: integer}

My use-case now tries to aggregate, for each Item, all the Shop.id’s where that Item is sold at.

This is my Ecto query:

query = from i in Item, 
              join: si in ShopItem, 
              on: si.item_id == i.id, 
              group_by: i.id, 
              select: {i.id, fragment("ARRAY_AGG(?)", si.shop_id)}

# Which when run
iex()> query |> Repo.all

# Returns something like ( limited to 20 results)
[
  {1, [16, 41, 21, 39]},
  {2, [27, 15, 50, 35]},
  {3, [1, 53, 27, 15]},
  {4, [43, 39, 29, 2]},
  {5, [40, 32, 21, 2]},
  {6, [5, 25, 52, 20]},
  {7, [14, 8, 40, 20]},
  {8, '%1*\b'},
  {9, [3, 41, 37, 31]},
  {10, [48, 12, 17, 50]},
  {11, [32, 4, 7, 27]},
  {12, [6, 1, 5, 7]},
  {13, [31, 34, 10, 1]},
  {14, [42, 32, 28, 52]},
  {15, [25, 3, 29, 50]},
  {16, [13, 14, 1, 51]},
  {17, ' +\b%'},
  {18, [51, 53, 29, 41]},
  {19, [14, 52, 7, 1]},
  {20, [30, 19, 33, 2]}
]

As you can see, Items 8 and 17 result in a weird aggregation; not a List but a charlist.

If I instead replace ARRAY_AGG with JSON_AGG to see what’s up in there, the maps appear in the aggregation just fine:

query = from i in Item, 
              join: si in ShopItem, 
              on: si.item_id == i.id, 
              group_by: i.id, 
              select: {i.id, fragment("JSON_AGG(?)", si)}   # <--  Change function here

iex()> query |> Repo.all

#result
[
  {1,
   [
     %{
       "id" => 1,
       "shop_id" => 16,
     },
     %{
       "id" => 2,
       "shop_id" => 41,
     },
     %{
       "id" => 3,
       "shop_id" => 21,
     },
     %{
       "id" => 4,
       "shop_id" => 39,
     }
   ]},
   #... more items
  {8,   # <-- conflictive Item 8 before, seems to have relationships perfectly fine here
   [
     %{
       "id" => 37,
       "shop_id" => 37,  
     },
     %{
       "id" => 38,
       "shop_id" => 49,
     },
     %{
       "id" => 39,
       "shop_id" => 42,
     },
     %{
       "id" => 40,
       "shop_id" => 8,
     }
   ]},
   #... more items
]

I run the equivalent query in raw (postgre)SQL and the relationships are also there, perfectly OK. For some reason there are some Item where the fragment-ed ARRAY_AGG is failing to return a List.

# Raw SQL
select 
	i.id,
	array_agg(si.shop_id) as sellers
	from (select * from items limit 200) as i
inner join shops_items as si
on si.item_id = i.id
group by i.id

# Returns
1	{16,41,21,39}
2	{27,15,50,35}
3	{1,53,27,15}
4	{43,39,29,2}
5	{40,32,21,2}
6	{5,25,52,20}
7	{14,8,40,20}
8	{37,49,42,8}   # <--- same relationships as the JSON_AGG case, so it's consistent
9	{3,41,37,31}
10	{48,12,17,50}
11	{32,4,7,27}
12	{6,1,5,7}
13	{31,34,10,1}
14	{42,32,28,52}
15	{25,3,29,50}
16	{13,14,1,51}
17	{32,43,8,37}
18	{51,53,29,41}
22	{40,28,10,37}

Any ideas what’s causing my problem? Thank you!

Marked As Solved

edisonywh

edisonywh

That’s because in the first one ARRAY_AGG returns a list of integer, which are considered charlist in Elixir: https://elixir-lang.org/getting-started/binaries-strings-and-char-lists.html#charlists

The second version returns tuples so that’s not an issue.

Where Next?

Popular in Questions Top

WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: &lt;h1&gt;Create Post&lt;/h1&gt; &lt;%= ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
earth10
Hi, I’m just starting to build a side-project with Elixir and Phoenix and doing some basic test with Elixir alone. What strikes me is th...
New

Other popular topics Top

malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43806 214
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
Lily
In templates/appointment/index.html.eex: &lt;%= for appointment &lt;- @appointments do %&gt; &lt;tr&gt; &lt;td&gt;&lt;%= appoi...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement