Upsert user with password that is hashed in changeset

I have the following schema and chageset:

defmodule Cybord.Accounts.User do
  @moduledoc """
  User schema module.
  Don't use any direct calls to functions in this module.
  Always go through Cybord.Accounts context

  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :email, :string
    field :password, :string, virtual: true
    field :password_hash, :string
    field :is_active, :boolean, default: false

    timestamps(type: :utc_datetime_usec)

  @doc false
  def changeset(user, attrs) do
    |> cast(attrs, [:email, :is_active, :password])
    |> validate_required([:email, :is_active, :password])
    |> unique_constraint(:email)
    |> put_password_hash()

  defp put_password_hash(
           valid?: true,
           changes: %{password: password}
         } = changeset
       ) do
    change(changeset, Bcrypt.add_hash(password))

  defp put_password_hash(changeset) do

and the following upsert function:

 def upsert_user(%{password: password} = attrs) do
    |> User.changeset(attrs)
    |> Repo.insert(
      on_conflict: [
        set: [
          password_hash: Bcrypt.add_hash(password).password_hash,
          updated_at: DateTime.utc_now()
      conflict_target: :email,
      return: true

When updating the changeset comes back invalid but I would rather use the “change” fields from the changeset then manipulating the attrs in the function. I tried:

 on_conflict: [
        set: [
          password_hash: &1.change.password_hash,
          updated_at: DateTime.utc_now()

but I get a unhandled &1 outside of a capture error.
Generally, this solution feels a bit hacky and an antipattern.

Can anybody suggest a more concise solution for this?