`not found in: nil` exception after Ash upgrade

Ash Forum Ash Questions / Help
,
1

I upgraded to the latest Ash (3.4.66 → 3.5.8), and one of my AshGraphql tests is failing with this error:

     08:24:51.258 request_id=GDwNw_x3uKJp7GcABxLB [error] cc719ddb-f52a-498e-97e7-def0d60a6290: Exception raised while resolving query.
     
     ** (Ash.Error.Unknown) 
     Bread Crumbs:
       > Exception raised in: GF.Events.Participation2.read
       > Exception raised in: GF.Events.Event2.read
     
     Unknown Error
     
     * ** (KeyError) key :to_tenant not found in: nil
     
     If you are using the dot syntax, such as map.field, make sure the left-hand side of the dot is a map
       (ash 3.5.8) lib/ash/policy/check/expression.ex:3: Ash.Policy.Check.Expression.try_strict_check/3
       (ash 3.5.8) lib/ash/policy/policy.ex:183: Ash.Policy.Policy.fetch_or_strict_check_fact/2
       (ash 3.5.8) lib/ash/policy/policy.ex:431: Ash.Policy.Policy.handle_constants/2
       (ash 3.5.8) lib/ash/policy/policy.ex:404: Ash.Policy.Policy.handle_constants/2

The line of code in Ash that raises the error is this: ash/lib/ash/policy/filter_check.ex at main · ash-project/ash · GitHub. subject is nil.

This is for an AshGraphql test.

This appears related to multitenancy. GF.Event.Event2 has a multitenancy block and a tenant database column, but GF.Events.Participation2 doesn’t.

In the error breadcrumbs, you’ll see two resources mentioned. The root resource being queried is GF.Events.Event2. It appears to be the has_many :attendees relationship to GF.Events.Participation2, because when I remove the attendees GraphQL field from the test, the error is gone. In this test, the actor should not have access to the attendees, but that wasn’t a problem before. It just returned an empty list for that case.

2

:thinking: interesting. subject shouldn’t ever be nil at that line (which is why we don’t defensively use something like subject[:to_tenant]. Is there more stacktrace available than just those lines? are your other Ash related dependencies updated to the latest as well?

3

Here’s the whole stacktrace:

       (ash 3.4.66) lib/ash/policy/check/expression.ex:3: Ash.Policy.Check.Expression.try_strict_check/3
       (ash 3.4.66) lib/ash/policy/policy.ex:183: Ash.Policy.Policy.fetch_or_strict_check_fact/2
       (ash 3.4.66) lib/ash/policy/policy.ex:431: Ash.Policy.Policy.handle_constants/2
       (ash 3.4.66) lib/ash/policy/policy.ex:404: Ash.Policy.Policy.handle_constants/2
       (ash 3.4.66) lib/ash/policy/policy.ex:390: Ash.Policy.Policy.handle_constants/2
       (ash 3.4.66) lib/ash/policy/policy.ex:404: Ash.Policy.Policy.handle_constants/2
       (ash 3.4.66) lib/ash/policy/policy.ex:390: Ash.Policy.Policy.handle_constants/2
       (ash 3.4.66) lib/ash/policy/policy.ex:29: Ash.Policy.Policy.solve/1
       (ash 3.4.66) lib/ash/policy/checker.ex:65: Ash.Policy.Checker.strict_check_scenarios/1
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1704: Ash.Policy.Authorizer.strict_check_result/2
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1127: Ash.Policy.Authorizer.field_condition/5
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1090: Ash.Policy.Authorizer.expression_for_ref/6
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1014: Ash.Policy.Authorizer.replace_refs/2
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:980: Ash.Policy.Authorizer.replace_refs/2
       (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:734: Ash.Policy.Authorizer.alter_filter/3
       (ash 3.4.66) lib/ash/can.ex:481: Ash.Can.alter_query/5
       (elixir 1.17.2) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
       (ash 3.4.66) lib/ash.ex:1390: Ash.can/3
       (ash 3.4.66) lib/ash/actions/read/read.ex:1580: Ash.Actions.Read.authorize_query/2
       (ash 3.4.66) lib/ash/actions/read/read.ex:472: Ash.Actions.Read.do_read/
     
         (ash 3.4.66) lib/ash/policy/check/expression.ex:3: Ash.Policy.Check.Expression.try_strict_check/3
         (ash 3.4.66) lib/ash/policy/policy.ex:183: Ash.Policy.Policy.fetch_or_strict_check_fact/2
         (ash 3.4.66) lib/ash/policy/policy.ex:431: Ash.Policy.Policy.handle_constants/2
         (ash 3.4.66) lib/ash/policy/policy.ex:404: Ash.Policy.Policy.handle_constants/2
         (ash 3.4.66) lib/ash/policy/policy.ex:390: Ash.Policy.Policy.handle_constants/2
         (ash 3.4.66) lib/ash/policy/policy.ex:404: Ash.Policy.Policy.handle_constants/2
         (ash 3.4.66) lib/ash/policy/policy.ex:390: Ash.Policy.Policy.handle_constants/2
         (ash 3.4.66) lib/ash/policy/policy.ex:29: Ash.Policy.Policy.solve/1
         (ash 3.4.66) lib/ash/policy/checker.ex:65: Ash.Policy.Checker.strict_check_scenarios/1
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1704: Ash.Policy.Authorizer.strict_check_result/2
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1127: Ash.Policy.Authorizer.field_condition/5
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1090: Ash.Policy.Authorizer.expression_for_ref/6
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:1014: Ash.Policy.Authorizer.replace_refs/2
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:980: Ash.Policy.Authorizer.replace_refs/2
         (ash 3.4.66) lib/ash/policy/authorizer/authorizer.ex:734: Ash.Policy.Authorizer.alter_filter/3
         (ash 3.4.66) lib/ash/can.ex:481: Ash.Can.alter_query/5
         (elixir 1.17.2) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
         (ash 3.4.66) lib/ash.ex:1390: Ash.can/3
         (ash 3.4.66) lib/ash/actions/read/read.ex:1580: Ash.Actions.Read.authorize_query/2
         (ash 3.4.66) lib/ash/actions/read/read.ex:472: Ash.Actions.Read.do_read/5

Yeah, all the Ash related dependencies are up to date:

ash                       3.5.8    3.5.8    Up-to-date           
ash_graphql               1.7.9    1.7.9    Up-to-date           
ash_paper_trail           0.5.3    0.5.3    Up-to-date           
ash_postgres              2.5.18   2.5.18   Up-to-date           
ash_sql                   0.2.74   0.2.74   Up-to-date
4

I also noticed that if I change the test so that the actor has access to the :attendees relationship, the error still happens.

5

I further isolated the conditions that cause the error. Here’s the GraphQL query now:

      query GetEvent2($id: ID!) {
        getEvent2(id: $id) {
          attendees(
            filter: {rsvpReply: {eq: YES}, waitlisted: {eq: false}, role: {notEq: HOST}, member: {status: {eq: ACTIVE}}}
          ) {
            id
          }
          id
        }
      }

When there’s a filter, the error happens. When I take it out, there’s no error.

6

And further still:

      query GetEvent2($id: ID!) {
        getEvent2(id: $id) {
          attendees(
            filter: {member: {status: {eq: ACTIVE}}}
          ) {
            id
          }
          id
        }
      }

It’s that filter on the member resource (Event2 → has_many :attendees Participation → belongs_to :member Member2)

7

Okay that helps. I think it’s a small plumbing error in the filter authorization code.

8

Then finally, in the test, the actor doesn’t have policy access to the member status attribute. When I update the actor to be able access it, the error goes away.

9

Okay, so the strange thing is why the existing field policy tests aren’t hitting that case :thinking: I’ve tried a similar formulation to what you have in various forms at the resource level. Can you reproduce this issue if you runt he corresponding Ash request? Something along the lines of query |> Ash.Query.load(attendees: Ash.Query.filter(Attendee, member.status == :active))

10

So I haven’t been able to reproduce this yet in my testing. If you have any way of reliably reproducing that would be great so we can of course get this covered by our authorization test suite. IIRC we have some field policy testing in AshGraphql, perhaps it could be reproduced there? I’ve got to step away for the rest of the day, but I’ve pushed up a change to main to normalize the underlying data structure in more of the callbacks which may resolve this issue. A release will go out next week or the week after (I’ve just merged combination query support which may need some time to bake).

11

Yes, that reproduces the error:

require Ash.Query
query = Ash.Query.filter(GF.Events.Event2, id: event.id)

query =
  query
  |> Ash.Query.load(
    attendees: Ash.Query.filter(GF.Events.Participation2, member.status == :active)
  )


Ash.read!(query, actor: ctx.session_member)
12

Oh wait. False alarm. That doesn’t reproduce the error. I’ll keep digging into this.

13

Did you have a chance to try the changes in main? We still really ought to reproduce as it would highlight a missing coverage area in field policies, but it would be a useful thing to know if the changes there resolve the issue. If they do, I’m not terribly worried about the impact of the issue as it’s essentially just a missing copy of the subject to the subject field.

14

I just tried it on main. It raised the same error. Here’s the abbreviated Ash.Policy.Authorizer:

%Ash.Policy.Authorizer{
  actor: %GF.Members.Member2{},
  resource: GF.Members.Member2,
  query: nil,
  changeset: nil,
  action_input: nil,
  data: nil,
  action: %Ash.Resource.Actions.Read{ },
  domain: GF.Domain,
  scenarios: nil,
  real_scenarios: nil,
  check_scenarios: nil,
  subject: nil,
  for_fields: [:status],
  solver_statement: {:and, ...},
  context: %{},
  policies: [...],
  facts: %{
    false => false,
    true => true,
    {Ash.Policy.Check.ActorAttributeEquals,
     [
       attribute: :__struct__,
       value: GF.Members.Member2,
       ash_field_policy?: true,
       access_type: :filter
     ]} => true,
    {GF.Members.ActiveMemberPolicy,
     [role: :members, ash_field_policy?: true, access_type: :filter]} => false
  },
  data_facts: %{}
}

The actor doesn’t have read access to the member’s status attribute, and the status attribute is being referenced in that GraphQL argument.

During the test, the debug statement was executed 3 times, and the last time raised the error.

15

Thank you, that authorizer dump is useful. So the problem wasn’t the copying from query → subject, it is that somehow all of query, changeset, and action_input keys are nil. That will effectively never work, something internally will always break if thats the case. I’m sure the answer lies in your stack trace somewhere, despite me not being able to reproduce this yet :cry:

16

Alright, I was able to snag a few minutes tonight to hopefully resolve the issue, but I’m out of time. If you can find any way to reproduce this I could probably fix it in like 5 minutes flat, but my tests have been unsuccessful so far for some reason. It doesn’t seem like a complex case to reproduce but its just not happening for me for some reason.

17

I created a PR that reproduces the error:

18

:fire: :fire: :fire: thank you. I will continue looking into this tonight or tomorrow at the latest.

19

@moxley this is fixed in main of ash. Sure enough, there was a place deep down in the field policy evaluator that was not putting the query into the subject field :slight_smile:

EDIT: thanks again for your diligence w/ the bug report and the reproduction :person_bowing:

4 Likes