Is there no way to insert them in one go using Repo.insert ?
Don’t want to use insert_all because first i wil have to convert my structs into maps and then manualy set the timestamps so that I dont get the “null value in column “inserted_at” violates not-null constraint” error. And also according to the docs, its a lower-level function.
Dont need to use Multi, becuase its not a transaction that I am dealing with.
So, Using Enum.each(list_of_structs, &Repo.insert) my only option ?
Multi transactions are required for the scenario where all the inserts are needed to go to DB or none at all. I dont have that scenario. The inserts I am trying to do are independent of each other. If half of the inserts are successful and half are not, I am still fine.
In that case you can just use Task.async_stream and pass it a function that inserts one record. It will transparently parallelize the inserts over all your CPU cores and every operation is independent and isolated.
If this is a pattern that’s important for you (insert several structs but in one database call) then perhaps worth investing in a helper function that converts to maps and adds the time stamps so you can use insert_all/2? Another approach might be to modify the database schema to have a default timestamp so you don’t have to think about it.
Yes, I do have timestamps() in the schema as well as in the migration file and as well as in the DB. and thats why I have to set the timestamps manually in the map if I were to use insert_all. This is what I have mentioned in the point 1 of my post. Do you mean something else ?
I’m really doing a bad job of explaining myself today
The timestamps() call in the Ecto schema defines the columns on the DB. My alternative suggestion was to define a default value for those timestamps so that even for an insert_all/2 the timestamps would be set by the database.
insert_all has been done for this. I would rather write a function to transform data into something suitable for insert_all, as mentionned by @kip, this can be done in one pass, with stream if working on large collection.
It might not change a lot for small dataset, but when You run batches of 10k elements, it’s better to use one insert_all, than 10k independant inserts.
I know this is an old thread, but I would appreciate why insert_all does not allow a list of structs. In other words, why does it not work as insert but for multiple records so that elixir can insert all records in one go.
*_all functions are all intentionally lower level apis. They all resolve to a single sql statement. With structs (and even more so with changesets) you can however call insert in ways you need multiple insert sql statements. These requirements are at odds with each other.
This is an ancient thread and an ancient answer but this is marked as the solution so I feel like it should be clarified that under essentially no circumstances should you ever do this.
If performance is not important, you can simply insert the rows sequentially and move on with your life.
If performance is important then you absolutely must be using insert_all(). The reduction in overhead from batching is enormous for a multitude of reasons.
Trying to parallelize single inserts across cores is the “worst of both worlds”. And as I’m sure 2025 Dimi would gladly point out, you should especially never do this with SQLite.
