wilkinson4

wilkinson4 OP

Hello,

We recently did some optimizations on a slow query in our system, and found the following article from dba exchange that has a nice syntax for selecting an array of items per row

This ended up being a more performant solution from what we were using previously. Below is a simple example of what we’re trying to do in raw postgresql

select *
from calls c
  left join lateral (
    select array (
        select cr.id
        from call_recordings cr
        where cr.call_id = c.id
          and cr.s3_key is not null) as call_recording_ids) as call_recordings on true
limit 10;

As far as I can tell there isn’t an easy way to do this in Ecto. The closest translation to Ecto I’ve got so far is this

  def test_select_array do
    call_recordings_query =
      from cr in Recording,
        select: %{
          call_recording_ids:
            fragment(
              "(SELECT ARRAY(SELECT cr.id FROM call_recordings cr WHERE cr.call_id = ? AND cr.s3_key IS NOT NULL))",
              parent_as(:call).id
            )
        }

    from(c in Call,
      as: :call,
      left_lateral_join: cr in subquery(call_recordings_query),
      limit: 10,
      select: %{
        call_id: c.id,
        call_recording_ids: cr
      }
    )
  end

But even that isn’t quite right, and it’s janky. Looking for any ideas/guidance on this. Maybe this is something Ecto supports without a fragment and I’m not aware of it, or could support?

– Will

Marked As Solved Switch mode

LostKobrakai

LostKobrakai

import Ecto.Query

array_query = 
  from cr in "call_recordings",
    where: cr.call_id == parent_as(:call).id,
    where: not is_nil(cr.s3_key),
    select: %{
      id: cr.id
    }

call_recordings_query =
      from cr in subquery(array_query),
        select: %{
          call_recording_ids: fragment("array_agg(?)", cr.id)
        }
query = 
  from(c in "calls",
    as: :call,
    left_lateral_join: cr in subquery(call_recordings_query), on: true,
    limit: 10,
    select: %{
      call_id: c.id,
      call_recording_ids: cr
    }
  )

{sql, _} = Repo.to_sql(:all, query)
IO.puts(sql)
# SELECT c0."id", s1."call_recording_ids" 
# FROM "calls" AS c0 
# LEFT OUTER JOIN LATERAL (
#   SELECT array_agg(ss0."id") AS "call_recording_ids" 
#   FROM (
#     SELECT ssc0."id" AS "id" 
#     FROM "call_recordings" AS ssc0 
#     WHERE (ssc0."call_id" = c0."id") AND (NOT (ssc0."s3_key" IS NULL))
#   ) AS ss0
# ) AS s1 ON TRUE 
# LIMIT 10

Functionally this should yield the same result as the requested query

Also Liked

D4no0

D4no0

You could move the fragment to a macro to make the query cleaner, here is an example on how that looks like:

defmacro st_transform(wkt, srid) do
  quote do: fragment("ST_Transform(?, ?)", unquote(wkt), unquote(srid))
end
Schultzer

Schultzer

This is where Ecto start to show it’s brittleness, as one of the best ORM I have ever worked with, it still falls short in a lot of real world cases and especially on a enterprise level where you usually have dedicated DBAs that don’t know Elixir, GitHub - elixir-dbvisor/sql: SQL provides state-of-the-art, high-performance SQL integration for Elixir, built to handle extreme concurrency with unmatched expressiveness and ergonomic query composition. Write safe, composable, parameterized queries directly, without translating to Ecto or any ORM. · GitHub could help you out here, still fresh but the idea is simple, lower the barrier between Elixir and SQL.

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
roeland
Kia ora, We have been using elixir-google-api to connect to Google Drive. However, with the updates to Tesla due to CVEs this is now bro...
New
rahultumpala
Hello, I have an Elixir backend that implements a custom protocol over TCP. I want to load test the backend and assess the performance o...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New

We're in Beta

About us Mission Statement