moxley

moxley

Failing to get aggregation to work with `parent()` and 3-layer nesting

I want to calculate the number of unread Posts for each User in a list of User IDs. Read (verb, past tense) Posts are tracked with PostView, by recording the ID of the last read Post for a given User ID.

Here’s the SQL of what I need:

select
  u.id user_id,
  case
    when pv.user_id is not null then pv.count
    else p.count
  end unread_posts_count
from users u
left join (
  select pv.user_id, count(p2.id) "count"
  from post_views pv
  left join posts p1 on p1.id = pv.last_post_id
  left join posts p2 on p2.inserted_at > p1.inserted_at and p2.archived_at is null
  group by pv.member_id
) tp on tp.user_id = u.id
left join (
  select count(p.id), t.id tenant_id
  from tenants t
  left join posts p on p.tenant_id = t.id
  group by t.id
) p on p.tenant_id = u.tenant_id
where u.id in $1

It seems Ash wants you to use the aggregate DSL combined with relationships to do this. I modified test/actions/has_many_test.exs from the Ash code repo to approximate what my app is doing. Also, I added an attribute to the respective resources to represent a tenant ID, because my app is multitenant.

  defmodule Post do
  ...
      attribute :tenant_id, :string, public?: true
  ...
  end

  defmodule PostView do
    @moduledoc false
    use Ash.Resource,
      domain: Ash.Test.Actions.HasManyTest.Domain,
      data_layer: Ash.DataLayer.Ets

    ets do
      private?(true)
    end

    actions do
      default_accept :*
      defaults [:read, :destroy, create: :*, update: :*]
    end

    attributes do
      uuid_primary_key :id

      attribute :last_post_id, :uuid, public?: true
      attribute :user_id, :uuid, public?: true
    end

    relationships do
      belongs_to :user, Ash.Test.Actions.HasManyTest.User do
        source_attribute :user_id
        destination_attribute :id
        public?(true)
      end

      belongs_to :last_post, Post do
        source_attribute :last_post_id
        destination_attribute :id
        public?(true)
      end
    end
  end

  defmodule User do
    @moduledoc false
    use Ash.Resource,
      domain: Ash.Test.Actions.HasManyTest.Domain,
      data_layer: Ash.DataLayer.Ets

    ets do
      private?(true)
    end

    actions do
      default_accept :*
      defaults [:read, :destroy, create: :*, update: :*]
    end

    attributes do
      uuid_primary_key :id

      attribute :name, :string, public?: true
      attribute :tenant_id, :string, public?: true
    end

    relationships do
      has_one :post_view, PostView do
        source_attribute :id
        destination_attribute :user_id
        public?(true)
      end

      has_many :unread_posts, Post do
        source_attribute :tenant_id
        destination_attribute :tenant_id
        public?(true)

        # This is eventually what I want to do, but it always produces no matches
        # filter expr(inserted_at > parent(post_view.last_post.inserted_at))

        # After some experimentation, I found that this filters as expected
        filter expr(id != parent(post_view.last_post_id))
        # But this never matches, even though it seems equivalent
        filter expr(id != parent(post_view.last_post.id))
      end
    end
  end

So it seems that when the parent(...) expression reaches more than two layers deep, nothing matches, but if it reaches only two layers, it does. I also tried parent(...).id, but that gave an error.

Marked As Solved

zachdaniel

zachdaniel

Creator of Ash

Alright, this should be fully fixed in the latest release of ash_sql, with one small fix in ash_postgres which you should not need immediately. ash_postgres will be released next week.

Also Liked

zachdaniel

zachdaniel

Creator of Ash

Did some basic investigation, and it’s just a “good old fashioned bug”. The issue is pretty much what you described at the outset, and it stems from essentially a missing set of logic to add the required joins for parent expressions that are contained in relationships used by aggregate queries to the parent query itself.

I’ve pushed a fix to ash_sql main which you can try. It’s not complete but should cover your current case. There is still an issue specifically around if that parent expr itself contains another aggregate that will take me a bit longer to nail down.

Tomorrow or early next week I will transplant your tests into ash_postgres as regression tests and fix the other bug that I mentioned around aggregates.

zachdaniel

zachdaniel

Creator of Ash

Perfect thank you. I will investigate tomorrow morning and get back to you :bowing_man:

zachdaniel

zachdaniel

Creator of Ash

Alright, so, the issue in the test reproduction you provided is a bit insidious and I’m not sure if it relates to the actual issue you are seeing because it seems isolated to the way that the ETS data layer computes aggregates.

the id type of Post is :ci_string in that test.

      uuid_primary_key :id, type: :ci_string

But the attribute type of the relationship was :uuid.

      attribute :last_post_id, :uuid, public?: true

Changing that to :ci_string resolves the issue.

The reason this causes a bug is because we check the :source_attribute’s type to see if it supports simple_equality?/0, which means "can we compare values of this with ==. If we can, we use Elixir’s builtin maps for grouping and connecting related entities. This is actually a critical optimization, all Ash apps would be dramatically slower if we didn’t do this.

Due to this misconfiguration, we tried somewhere to compare a CiString struct with a string, and it was not equivalent.

There is supposed to be a compiler warning about types not matching here, not sure why there wasn’t, that can be investigated separately.

So the question is whether or not this has anything to do with your own issue as well, or if its just a red herring in the tests.

Last Post!

moxley

moxley

That appears to have worked. Thank you @zachdaniel!

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New

Other popular topics Top

baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49084 226
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 42533 114
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement