elixirnewbie

elixirnewbie

Bulk inserts with Postgrex

What’s the best way to do bulk inserts with Postgrex?
If I have a list [[1,2],[3,4],[5,6]] what’s the ideal way to handle it in Postgrex.

Most Liked

OvermindDL1

OvermindDL1

Wooo, quite a number, I’d probably use the PostgreSQL COPY command.

Some relevant threads:

elixirnewbie

elixirnewbie

@benwilson512 I am using Postgrex.I know with Ecto you can do Repo.insert_all .I am trying to insert 50,000 records.So, till now I have
1.50000 |> Stream.map(fn(item) -> [item,item * 2] end) |> Stream.chunk(1000,1000,[]) |> Task.async_stream(fn(list) -> list |> Enum.each(fn(record) -> Postgrex.query!(pid, "INSERT INTO mytable(val1,val2) VALUES($1,$2) ", list) end) end)

It works but I am trying to insert multiple values in a single transaction

hubertlepicki

hubertlepicki

So Postgrex can insert multiple records, and this is precisely what Ecto is doing. If you use Ecto’s insert_all, it inserts multiple records using single INSERT command like this:

INSERT INTO "table" ("a","b","c") VALUES ($1,$2,$3),($4,$5,$6), ... ,($97,$98,$99) ["A", "B", "C", "A", "B", "C", ... , "A", "B", "C"]

If you are using Postgrex directly, I think you need to assemble that SQL on your own, I don’t think it gives you any shortcuts.

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49134 226
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New

We're in Beta

About us Mission Statement