I was trying to upsert rows from a database export that I processed with Python and exported back to CSV to be loaded back from Elixir + Ash.
Instead of fetching all rows, updating the resources in memory, and using a bulk_update
(not sure about that as I’ve never tried to use it for now. EDIT: welp, this is definitely not how bulk_update
works ), I figured it would be easier to just use
bulk_insert
and use my id
attribute as the upsert_identity
:
%{records: inserted_skills, error_count: skills_error_count, errors: skills_errors} =
Ash.bulk_create(unique_skills_data, Skill, :create,
return_records?: true,
upsert?: true,
upsert_identity: :id,
upsert_fields: [:slug, :name, :is_product, :acronym, :acronym_of],
return_errors?: true,
select: [:id, :slug, :name]
# skip_unknown_inputs: :*
)
As you can see, I initially tried using skip_unknown_inputs
because I was getting an UnknownInput
error on the id
attribute. Since my id
isn’t writable and should not be accepted by actions, I didn’t see another workaround that wouldn’t risk data consistency.
After that, I couldn’t understand why the upsert wasn’t working (it just inserted new rows with new IDs), and I thought my code was at fault (whether Python notebook exporting the CSV or the Elixir import script).
It turns out that if I just make my id
attribute accepted by the create
action (and writable, as it’s a requirement to make it accepted by actions), the issue doesn’t arise anymore.
I’m not sure whether you’d consider that a bug or a feature. On my side, I’d consider it somewhat of a bug, especially since I’m not trying to write to it (it’s not in my upsert_fields
list, which then would make sense to error).
I think for now I’ll consider updating in memory and using EDIT: [make my bulk_update
id
attribute writable & accepted] (unless there’s a better way I’m not aware of), but figured out it would be interesting to raise a discussion on the topic!
All the best