dli

dli

Support :insert_all in data-modifying CTE / WITH statements

Ecto currently supports some data-modifying WITH statements / CTEs for Postgres:

Options: […]
:operation - one of :all , :update_all , or :delete_all indicating the operation type of the CTE query. If blank, it defaults to :all , making the CTE query a SELECT query.
[…]
For Postgres built-in adapter, it is possible to define data-modifying CTE queries:

update_categories_query =
  Category
  |> where([c], is_nil(c.parent_id))
  |> update([c], set: [name: "Root category"])
  |> select([c], c)

{"update_categories", Category}
|> with_cte("update_categories", as: ^update_categories_query, operation: :update_all)
|> select([c], c)

Postgres, however, also supports INSERT statements inside CTEs:

-- EXAMPLE
with expensive_calc as (
   select customer_id, sum(amount) as total 
   from orders 
   group by customer_id
),
new_segments as (
   insert into segments (customer_id, total)
   select customer_id, total 
   from expensive_calc
   returning id, customer_id
)
insert into segment_history (segment_id, customer_id)
select id, customer_id 
from new_segments;

This allows multiple INSERTs to access the same source data, as well as the newly inserted data without crossing the database boundary.

The closest solutions in Ecto are:

  1. Create temp table for expensive_calc (needs fragment)
  2. Repeat expensive_calc as subquery (likely does the calculation twice)

… then use two sequential Repo.insert_all statements. However, this needs serialization of the RETURNING data and unnecessarily crosses the database ↔︎ app boundary.

Other examples:

Ecto syntax

I assume this would require a new macro insert/3 similar to update/3 which allows adding returning: [p.id, p.title, ...]? Haven’t given this too much thought.

Judging by the ecto_sql source and associated PR, this was initially planned but not implemented yet:

https://github.com/elixir-ecto/ecto_sql/blob/8012355131c64048824a6b0799def36fae208555/lib/ecto/adapters/postgres/connection.ex#L619-L621

Maybe now is the right time?

First Post!

dli

dli

Bump to gather comments from top contributors — wdyt @josevalim @ericmj @michalmuskala?

Where Next?

Popular in Proposals: Ideas Top

hst337
Elixir compiler and language specification Purpose of the proposal Elixir language is in mature state and no breaking or heavy changes ar...
New
c4710n
This is an idea for improving the workflow of iex -S mix phx.server The Problem and The Solution in use When running multiple Phoenix en...
New
rmoorman
Current situation Currently, the structure of the HTML returned by phoenix is determined by the layouts (components/layouts/[root,app].ht...
New
dkuku
This is a proposal to make the map key mismatch errors a bit better: Every time I have a typo It’s very challenging for me even when I u...
New
bartblast
This could resolve to {[a: 1, b: 2]}. Was it ever considered to allow such syntax? Notice this: {:abc, a: 1, b: 2} and this: my_fun(:abc,...
New
byhemechi
Many web frameworks (e.g. Remix, Gatsby) have an option for their link components that begins the navigation request on hover so that whe...
New
eagle-head
Hi everyone, I’ve been researching Content Security Policy Level 3 support in Phoenix and wanted to share my findings and a proposal for...
New

Other popular topics Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement