acrolink

acrolink

How to use PostgreSQL's DISTINCT ON in Ecto Query

I have this SQL query:

SELECT DISTINCT ON (b.id) b.id as book_id, r.id as record_id, r.due_for_return, r.returned_at
FROM books b
left join records r
on b.id = r.book_id
order by b.id, r.id DESC

Any idea how to convert it to an Ecto Query, especially the DISTINCT ON part ? Thank you.

Marked As Solved

peerreynders

peerreynders

How to use PostgreSQL’s DISTINCT ON in Ecto Query

    create table(:books) do
      add :title, :string
    end
    create table(:records) do
      add :book_id, :id
      add :due_for_return, :date
    end
# lib/books.ex
defmodule Books do
  import Ecto.Query;
  alias Books.{Repo}

  def init do
    Repo.insert_all(
      "books", [
        [title: "One"],
        [title: "Two"],
        [title: "Three"],
        [title: "Four"]
      ]
    )
    Repo.insert_all(
      "records", [
        [book_id: 1, due_for_return: ~D[2017-11-01]],
        [book_id: 1, due_for_return: ~D[2017-12-01]],
        [book_id: 1, due_for_return: ~D[2018-01-01]],
        [book_id: 2, due_for_return: ~D[2018-01-02]],
        [book_id: 2, due_for_return: ~D[2018-02-02]],
        [book_id: 3, due_for_return: ~D[2018-03-03]],
      ]
    )
  end

  def query() do
    "books"
    |> join(:left, [b], r in "records", b.id == r.book_id)
    |> select([b,r], %{
         title: b.title,
         book_id: b.id,
         record_id: max(r.id)
       })
    |> group_by([b], [b.title, b.id])
    |> subquery()
    |> join(:left, [d,r],
         r in "records", d.book_id == r.book_id and d.record_id == r.id
       )
    |> select([d,r], %{
         book_id: d.book_id,
         title: d.title,
         due: r.due_for_return
       })
    |> Repo.all()
  end
end
iex(1)> Books.init()

07:15:35.715 [debug] QUERY OK db=7.9ms
INSERT INTO "books" ("title") VALUES ($1),($2),($3),($4) ["One", "Two", "Three", "Four"]
 
07:15:35.720 [debug] QUERY OK db=1.3ms
INSERT INTO "records" ("book_id","due_for_return") VALUES ($1,$2),($3,$4),($5,$6),($7,$8),($9,$10),($11,$12) [1, {2017, 11, 1}, 1, {2017, 12, 1}, 1, {2018, 1, 1}, 2, {2018, 1, 2}, 2, {2018, 2, 2}, 3, {2018, 3, 3}]
{6, nil}
iex(2)> Books.query()

07:15:43.693 [debug] QUERY OK db=3.1ms
SELECT s0."book_id", s0."title", r1."due_for_return" FROM (SELECT b0."title" AS "title", b0."id" AS "book_id", max(r1."id") AS "record_id" FROM "books" AS b0 LEFT OUTER JOIN "records" AS r1 ON b0."id" = r1."book_id" GROUP BY b0."title", b0."id") AS s0 LEFT OUTER JOIN "records" AS r1 ON (s0."book_id" = r1."book_id") AND (s0."record_id" = r1."id") []
[
  %{book_id: 1, due: {2018, 1, 1}, title: "One"},
  %{book_id: 2, due: {2018, 2, 2}, title: "Two"},
  %{book_id: 3, due: {2018, 3, 3}, title: "Three"},
  %{book_id: 4, due: nil, title: "Four"}
]
iex(3)> 

PostgreSQL GROUP BY

Also Liked

snnowwy

snnowwy

Hi acrolink et al,
I know I’m a bit late to this party but I was reading this for inspiration on a similar issue today and I ended up just using a fragment like so:

books_query =
  from(
    b in Mango.Books.Book,
    left_join: r in Mango.Records.Record,
    on: r.book_id == b.id,
    select: %{
      :book_id => fragment("distinct on (?) ?", b.id, b.id),
      :record_id => r.id,
      :due_for_return => r.due_for_return,
      :returned_at => r.returned_at
    },
    order_by: [asc: b.id, desc: r.due_for_return]
  )

My particular case didn’t allow for the methods outlined above so I gave that a go and it worked.

Also, I’ve sorted by due_for_return instead of id just in case something crazy happened and your record.id’s ended up out of sync with due_for_return.

I’d be interested to see how this pans out for you.

S.

peerreynders

peerreynders

You could also move the query into a view. That way the Ecto select is extremely simple and you can change the query inside the database if needed later.

PostgreSQL Views

peerreynders

peerreynders

Is it slow now?

I don’t know anything about PostgreSQL’s performance characteristics. And ultimately you would have to benchmark the query on your configuration.

There may be cases where an active column can be the best solution but there is the trade off of the required additional update the needs to happen within the same transaction as the insert (both of which affect the index).

Another alternative is to formulate the query around due_for_return and put an index on that but I don’t think it’s going to perform any better than using the primary key index.

Where Next?

Popular in Questions Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36128 110
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

We're in Beta

About us Mission Statement