elixirnewbie

elixirnewbie

Bulk inserts with Postgrex

What’s the best way to do bulk inserts with Postgrex?
If I have a list [[1,2],[3,4],[5,6]] what’s the ideal way to handle it in Postgrex.

Most Liked

OvermindDL1

OvermindDL1

Wooo, quite a number, I’d probably use the PostgreSQL COPY command.

Some relevant threads:

elixirnewbie

elixirnewbie

@benwilson512 I am using Postgrex.I know with Ecto you can do Repo.insert_all .I am trying to insert 50,000 records.So, till now I have
1.50000 |> Stream.map(fn(item) -> [item,item * 2] end) |> Stream.chunk(1000,1000,[]) |> Task.async_stream(fn(list) -> list |> Enum.each(fn(record) -> Postgrex.query!(pid, "INSERT INTO mytable(val1,val2) VALUES($1,$2) ", list) end) end)

It works but I am trying to insert multiple values in a single transaction

hubertlepicki

hubertlepicki

So Postgrex can insert multiple records, and this is precisely what Ecto is doing. If you use Ecto’s insert_all, it inserts multiple records using single INSERT command like this:

INSERT INTO "table" ("a","b","c") VALUES ($1,$2,$3),($4,$5,$6), ... ,($97,$98,$99) ["A", "B", "C", "A", "B", "C", ... , "A", "B", "C"]

If you are using Postgrex directly, I think you need to assemble that SQL on your own, I don’t think it gives you any shortcuts.

Where Next?

Popular in Questions Top

siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> somethi...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement