Anybody working on or interested in ash_mongo (ash datalayer for mongodb)

Ash Forum Ash Questions / Help
,
1

Having worked with ash quite a bit, it is quite hard for me to let it go. So when we had to talk to a legacy mongo db from our elixir application, I very quickly ended up at: Oh boy, looks like I have to write my own mongo data layer …

For now my very minimal AshMongo.DataLayer lives inside our application, but it probably deserves it’s own project / repository.

I am wondering if anybody else had to scratch the same itch.

2

@zachdaniel working on my ash_mongo data layer, I am stumbling on corner cases, and I am often not sure if this is me misunderstanding how ash is supposed to work, is this is a known limitation or a bug somewhere in ash. If you don’t mind, I plan to ask you here in elixir forum; but if you prefer, I can also open github issues, or address my questions to someone else. Also noteworthy: I am doing an “work with AI assistant” experiment, meaning I had some help finding / isolating / describing the issue. I did verify the behavior, but did not dig into the ash internal, and I will try to make clear what I have reviewed and what is just the AI’s best guess. Again, if this bothers you, please tell me.

So here comes the first issue:

Problem

CiString values in IN operator filters get normalized to regular strings, losing case-insensitive behavior.

Example

defmodule TestResource do
    use Ash.Resource,
    domain: nil,
    data_layer: Ash.DataLayer.Ets
        
  attributes do
    uuid_primary_key :id
    attribute :name, :string
  end
end

> require Ash.Query
Ash.Query
> q = Ash.Query.filter(TestResource, name in ["a", ~i"b"])
#Ash.Query<resource: TestResource, filter: #Ash.Filter<name in ["a", "b"]>>
> IO.inspect(q.filter)
#Ash.Filter<name in ["a", "b"]>

and according to the helpful assistant this is the root cause (I did not review this part):

Root Cause

In Ash.Query.Operator.In.new/2 line 19:

def new(%Ash.Query.Ref{} = left, right) when is_list(right) do
  {:ok, %__MODULE__{left: left, right: MapSet.new(right)}}  # CiString → string here
end

MapSet.new() normalizes CiString structs to strings, losing case-insensitive comparison.

our workaround right now is to use an OR conditions instead:

filter(username == ~i"a" or username == “b”)

Environment

  • Ash 3.5.37
  • Elixir 1.18.4
3

and here comes the next one (I did observe the behaviour and checked that the referenced code exists)

Ash Framework Issue: Runtime Upsert Options Not Properly Passed to Data Layers

Problem Summary

When using Ash.create/3 with runtime upsert options (like upsert?: false, upsert_fields: [...]), these options are not properly handled or passed to the data layer, causing incorrect behavior.

Issues Identified

Issue 1: Runtime upsert?: false is ignored when action has upsert?: true

Location: lib/ash/actions/create/create.ex, lines 100-104

Current code:

opts =
  opts
  |> Keyword.put(
    :upsert?,
    action.upsert? || opts[:upsert?] || get_in(changeset.context, [:private, :upsert?]) || false
  )

Problem: The OR logic means if action.upsert? is true, runtime opts[:upsert?] = false is ignored.

Expected behavior: Runtime options should override action settings.

Issue 2: Runtime upsert_fields option is never passed to data layer

Location: lib/ash/actions/create/create.ex, lines 125-127

Current code:

changeset =
  Ash.Changeset.set_context(changeset, %{
    private: %{upsert?: true, upsert_identity: upsert_identity}
    # Missing: upsert_fields from opts
  })

Problem: Unlike bulk_create which properly sets upsert_fields in the changeset context, regular create doesn’t.

Compare with bulk_create (lib/ash/actions/create/bulk.ex):

|> Map.put(:context, %{
  private: %{
    upsert?: opts[:upsert?] || action.upsert? || false,
    upsert_identity: opts[:upsert_identity] || action.upsert_identity,
    upsert_fields: Ash.Changeset.expand_upsert_fields(
      opts[:upsert_fields] || action.upsert_fields,  # <-- Properly handled
      resource
    )
  }
})

Expected behavior: Runtime upsert_fields should be accessible to data layers via changeset.context[:private][:upsert_fields].

to be clear: my expectation is that we should be able to set upsert_fields as a runtime option, but maybe this is not possible for a normal create action … but in that case it would be nice if ash complained when I tried to do this

4

Can you show for sure that MapSet.new normalizes to strings? I don’t see any reason that would be happening.

iex(1)> MapSet.new([Ash.CiString.new("FooBar"), Ash.CiString.new("foobar")])
MapSet.new([#Ash.CiString<"FooBar">, #Ash.CiString<"foobar">])

I’ve pushed a fix to ash main that should resolve this that special cases CiStrings in our internal type casting logic (something we had before but was missing a case).

The OR logic means if action.upsert? is true, runtime opts[:upsert?] = false is ignored.

The current semantics is that if the action says it’s an upset then it is always an upsert, you can’t override it to be false. So the correct logic is action.upsert? || opts[:upsert?]

Runtime upsert_fields option is never passed to data layer

This one is a bug, please open an issue and/or a PR :person_bowing:

5

ok, will do, thx for the clarifications.

next question: datalayers can document what features they support, and this seems to be the canonical list: Ash.DataLayer — ash v3.5.38

I am mostly guessing based on the name and reverse engineering other datalayers … is there a better way to know what each feature does / is supposed to do?

6

and yes you were right, the in filter thing had nothing to do with MapSet (and it was quite a stupid thing to write … I guess the vibe coding has started to rot my brain, sorry for wasting your time)

and yes, this fixes it: fix: special case ci_strings as strings in type casting · ash-project/ash@d81ccf0 · GitHub

7

I would love to say yes to this question, but no its really not made clear anywhere :cry:

8

hey @zachdaniel quick follow up on the data layer capabilities / features: in ash_postgres (ash_postgres/lib/data_layer.ex at main · ash-project/ash_postgres · GitHub) i see

  def can?(_, :calculate), do: true
  def can?(_, :expression_calculation), do: true
  def can?(_, :expression_calculation_sort), do: true

but in Ash.DataLayer — ash v3.5.40 I only see expression_calculation_sort

am I right in assuming that the other 2 are just missing from the doc?

9

Good call, fixed here: fix: add missing capabilities to data layer spec · ash-project/ash@8009bbe · GitHub

1 Like