AshGraphql filter for atom attribute stopped working in Ash 3

The add_calculations callback is where we determine the calculations, and then in add_field_level_auth function in the read action logic.

Would you mind giving me some hints which files/functions the field policies code path is run? I can take a look into it when I have some free time in my hands.

Yeah, I’d imagine this area is rife for optimizations

Ultimately, I think having fewer field policies is better than one policy per field from a conceptual standpoint, and we should come up with a way for you to write either a deny-list or an allow-list.

I’d be willing to bet that we could eliminate most if not all of that additional processing time via optimizing those code paths. If you could open an issue on Ash showing this info that would be great. I won’t have the time to optimize this in the near future, unfortunately, but perhaps an adventurous soul can investigate.

I just added some more benchmark numbers:

Name                       ips        average  deviation         median         99th %
disabled               4885.16        0.20 ms    ±16.71%       0.197 ms        0.34 ms
catch_all_always       1227.80        0.81 ms    ±22.45%        0.75 ms        1.61 ms
catch_all_roles         964.08        1.04 ms    ±13.55%        1.00 ms        1.57 ms
always                   50.60       19.76 ms    ±19.98%       19.15 ms       30.24 ms
bypass_roles             37.81       26.45 ms    ±13.31%       26.03 ms       35.65 ms
roles                    36.92       27.09 ms    ±11.88%       26.44 ms       38.14 ms

Comparison: 
disabled               4885.16
catch_all_always       1227.80 - 3.98x slower +0.61 ms
catch_all_roles         964.08 - 5.07x slower +0.83 ms
always                   50.60 - 96.55x slower +19.56 ms
bypass_roles             37.81 - 129.19x slower +26.24 ms
roles                    36.92 - 132.33x slower +26.88 ms

Memory usage statistics:

Name                     average  deviation         median         99th %
disabled               0.0254 MB     ±0.01%      0.0254 MB      0.0254 MB
catch_all_always        0.142 MB     ±0.00%       0.142 MB       0.142 MB
catch_all_roles          0.29 MB     ±0.01%        0.29 MB        0.29 MB
always                   5.22 MB     ±0.06%        5.22 MB        5.23 MB
bypass_roles            15.10 MB     ±0.13%       15.10 MB       15.15 MB
roles                   15.31 MB     ±0.06%       15.31 MB       15.34 MB

Comparison: 
disabled               0.0254 MB
catch_all_always        0.142 MB - 5.62x memory usage +0.117 MB
catch_all_roles          0.29 MB - 11.45x memory usage +0.27 MB
always                   5.22 MB - 205.84x memory usage +5.20 MB
bypass_roles            15.10 MB - 595.18x memory usage +15.07 MB
roles                   15.31 MB - 603.57x memory usage +15.29 MB

Legend:
disabled: field_policies are fully disabled
catch_all_always: there is only one field_policy with a catch_all that always authorize
catch_all_roles: there is only one field_policy with a catch_all that checks for the role
always: have a field_policy per attribute that always authorize
bypass_roles: have a field_policy_bypass per attribute that will check for the role
roles: have a field_policy per attribute that will check for the role

It is interesting to see that even having just one field_policy already makes the query 4x slower than not having any check at all

So, @zachdaniel , I did some benchmark testing here and I think I got some interesting numbers.

The test consists of a resource that contains around 255 attributes. I used the ETS data-layer and added one record to it.

Then, I run a read action to fetch it applying my field policies and another run not applying it.

The benchmark code is as follows:

defmodule Benchmark do
  alias Core.Pacman.Markets.Record
  
  def run() do
    actor = %{roles: [:blibs]}

    Benchee.run(
      %{
        "run" => fn ->
          Record
          |> Ash.Query.for_read(:read, %{}, actor: actor)
          |> Ash.read!()
        end,
      },
      time: 100,
      memory_time: 2
    )
  end
end

For the field policies, I added one per attribute, all of them have the same check, I did two types of checks to see if the check itself would be a bottleneck.

Using a check that search for a atom in the :roles field of an agent (based on the HasRole check from ash rbac):

      field_policy :external_id do
        authorize_if {HasRole, role: [:support, :admin, {:agent, :organization_roles}]}
      end

Using a always authorize check:

      field_policy :external_id do
        authorize_if always()
      end

Here are the results:

Name               ips        average  deviation         median         99th %
disabled       4885.16        0.20 ms    ±16.71%       0.197 ms        0.34 ms
always           50.60       19.76 ms    ±19.98%       19.15 ms       30.24 ms
roles            36.92       27.09 ms    ±11.88%       26.44 ms       38.14 ms

Comparison: 
disabled       4885.16
always           50.60 - 96.55x slower +19.56 ms
roles            36.92 - 132.33x slower +26.88 ms

Memory usage statistics:

Name             average  deviation         median         99th %
disabled       0.0254 MB     ±0.01%      0.0254 MB      0.0254 MB
always           5.22 MB     ±0.06%        5.22 MB        5.23 MB
roles           15.31 MB     ±0.06%       15.31 MB       15.34 MB

Comparison: 
disabled       0.0254 MB
always           5.22 MB - 205.84x memory usage +5.20 MB
roles           15.31 MB - 603.57x memory usage +15.29 MB

Seems like there is a very big overhead of creating field policies per field

I’m gonna try using the ets datalayer since I think that one would be more consistent on timing.

Ah, yeah you’re right Wasn’t thinking. I don’t think there is a good way to just apply field policies TBH. They are implemented as calculations.

Hmm, why would that not go to the database? Shouldn’t it fetch all these fields in the load call? At least that’s what I’m seeing here when I tried it.

Here is my code:

records = Record |> Ash.Query.for_read(:read, %{}) |> Ash.Query.select([]) |> Ash.Query.limit(100) |>  Ash.read!()

fields = [,,,]

Ash.load!(records, fields, actor: actor)

Sort of. You can fetch a record with select([]) and then Ash.load(thing, [:your, :big, :list, :of :attributes], actor: actor). That tests a bit more than just field policies, but shouldn’t go to the database or anything.

@zachdaniel is there some call that I can do to just run the field_policies for a resource action without running the full action (like we can use Ash.can? for normal policies)?

Basically I’m creating some benchmarks to test if there is any impact in performance, but it would be great if I can shrink the scope just to the field policies.