Using Aggregates and Calculations in read actions, possibly using Cachex

Hello friends!

So I’ve been ticking away at a project to showcase the possible use case for using Ash and Phoenix at my place of work.

I feel like I’m missing a few things about Ash (or maybe it’s just Elixir) to fully flesh out this example project.

For starters, here’s a more “”“real world”“” example of an Ash Resource that outlines what I’m trying to do:

defmodule MyProject.Entities.State do
  @moduledoc """
  """
  use Ash.Resource,
    data_layer: AshPostgres.DataLayer,
    extensions: AshJsonApi.Resource

  attributes do
    uuid_primary_key :id

    attribute :name, :string do
      allow_nil? false
    end

    attribute :opt_out, :boolean do
      allow_nil? false
      default false
    end
  end

  actions do
    # Exposes default built in actions to manage the resource
    defaults [:create, :read, :update, :destroy]

    # Defines custom read action which fetches post by id.
    read :by_id do
      # This action has one argument :id of type :uuid
      argument :id, :uuid, allow_nil?: false
      # Tells us we expect this action to return a single result
      get? true
      filter expr(id == ^arg(:id))
    end
      
  end

  code_interface do
    define_for MyProject.Entities
    define :create, action: :create
    define :read_all, action: :read
    define :update, action: :update
    define :destroy, action: :destroy
    define :get_by_id, args: [:id], action: :by_id
  end

  relationships do
    belongs_to :country, MyProject.Entities.Country
    has_many :counties, MyProject.Entities.County
    has_many :cities, MyProject.Entities.City
  end

  aggregates do
    count :number_of_counties, :counties do
      description "Nubmer of counties in a state."
      filter expr(opt_out == false)
    end

    list :counties_in_state, :counties, :id do
      description "List of counties in the given state."
      filter expr(opt_out == false)
    end

    count :number_of_cities, :cities do
      description "Number of cities in a state"
      filter expr(opt_out == false)
    end

    list :cities_in_state, :cities, :id do
      description "List of cities in the given state."
      filter expr(opt_out == false)
    end
  end

  postgres do
    table "states"
    repo MyProject.Repo
  end

  json_api do
    type "states"

    routes do
      base "/states"

      get :by_id, route: "/:id", action: :by_id
      index :read, route: "/all"
      post :create
    end
  end
end

So a couple of questions:

  1. How do I call an aggregate or calculation in a read action?

    list is very useful in a lot of situations that I’d be facing.

  2. Is it possible or even recommended to cache the results of read actions via Cachex?

    They will be used in the json_api section and the result of the some read actions can be quite large, some results might even be from the list type of aggregate or even more advanced calculations. Of course abusing MATERIALIZED VIEW is an option, but is that always recommended first before any advanced caching mechanisms?

  1. You can use load to load the aggregates/calculations. You can do it on a query: Ash.Query.load(query, [:cities_in_state]) or on records that you already have with Api.load(some_data, [:cities_in_state]).

  2. You can cache Ash responses, but all of the normal caveats/issues arise from doing that kind of thing :slight_smile: We actually do something similar for data that essentially never changes in ash_hq. https://github.com/ash-project/ash_hq/blob/main/lib/ash_hq/docs/resources/library/library.ex#L16

    read :read do
      primary? true

      argument :check_cache, :boolean do
        default true
      end

      prepare AshHq.Docs.Library.Preparations.CheckCache
    end
defmodule AshHq.Docs.Library.Preparations.CheckCache do
  @moduledoc """
  Checks a simple agent cache for libraries
  """
  use Ash.Resource.Preparation

  def prepare(query, _, _) when query.arguments.check_cache == true do
    Ash.Query.before_action(query, fn query ->
      AshHq.Docs
      |> Ash.Filter.Runtime.filter_matches(
        AshHq.Docs.Library.Agent.get(),
        query.filter
      )
      |> case do
        {:ok, results} ->
          results =
            results
            |> then(fn results ->
              if query.offset do
                Enum.drop(results, query.offset)
              else
                results
              end
            end)
            |> then(fn results ->
              if query.limit do
                Enum.take(results, query.limit)
              else
                results
              end
            end)
            |> Ash.Sort.runtime_sort(query.sort)

          Ash.Query.set_result(query, {:ok, results})

        {:error, _} ->
          query
      end
    end)
  end

  def prepare(query, _, _) do
    query
  end
end

Okay. I think I’m getting the flow of things, but let me verify you. I think Java and Hibernate have just poisoned my brain so it’s definitely a lot of new things to try and grasp.

  1. If getting an item/some items based on a single attribute of a defined resource, we can do

    read :by_attr do
       arguments :attr, :type, allow_nil?: false
       get? true
       filter expr(attr == ^arg(:attr))
    end
    
  2. If getting an item/some items involves something a bit more complex (i.e. checking API permissions before everything else) we can use prepare with a defined module that contains just a prepare function that can take up to 3 arguments.

  3. prepare is used for, well, preparing the results of a read action.

  4. If wanting to get or rather “load” an aggregate or calculated field, we can use Ash.Query.load.

    Ash.Query.load(query, [:some_aggregate_field])
    

    We can use this in another separately defined module and call it in the read action.

     # Inside of the State Resource
     code_interface do
       define_for MyProject.Resoruces.State
         
       define :cities_in_state, args: [:state_id]
     end
    
     read :cities_in_state do
       argument :state_id, :uuid, allow_nil?: false
       
       get? true
       
       pagination offset?: true, default_limit: 50, countable: true
       
       filter expr(id == ^arg(:id))
    
       # I would assume a more generic
       # MyProject.Utils.Resources.States.GetAggregate :aggregate_name
       # might be considered "better" in some cases, but
       # I don't recall how to pass in an argument in that context
       prepare MyProject.Utils.Resources.States.GetListOfCities
     end
    
     # Inside MyProject.Utils.Resources.States.GetListOfCities
     # Not entirely sure if doing a guard for permission checks 
     # here is "better" or idiomatic in Ash
     def prepare(query, _, _) when check_if_some_permission_passes do
       Ash.Query.load(query, [:cities_in_state])
         |> case do
             {:ok, results} ->
               # probably would transform the resulting
               # Resource in a way that doesn't include things like 
               # foreign keys that aren't necessary to see in a UI
               Ash.Query.set_result(query, {:ok, results})
             {:error, _} ->
               # probably would actually give a helpful message in a real world situation
               query
         end
     end
    

    We can then use this in as MyProject.Resources.State.cities_in_state(:some_state_id), and I believe we use it in a Api.read or Api.get! but I’m still trying to get that all straight in my head by going through the docs.

  5. In the json_api section, we can do the following:

    # Inside of MyProject.Resources.State
    json_api do
      type "states"
      
      routes do
        base "/states"
        
        get: :by_id, route: "/:id", action: :by_id
        get: :cities_in_state, route: "/cities/:state_id", action: :cities_in_state
      end
    end
    

Hopefully I’m not too far gone and I’m mostly following the flow of using Ash.

Yep, this is all good with one exception of #4, only slight semantics.

 
# checking if some permission passes, I would suggest using the built in policy authorizer
# and field policies. Field policies, for instance, will prevent viewing specific fields based on specific criteria, allowing you to just load what you want and things they can't see will come back with the value `%Ash.ForbiddenField{}`

 def prepare(query, _, _) when check_if_some_permission_passes do
  # Ash.Query.load doesn't do it in line. It happens when the query is run
   Ash.Query.load(query, [:cities_in_state])
     |> case do
         {:ok, results} ->
           Ash.Query.set_result(query, {:ok, results})
         {:error, _} ->
           # probably would actually give a helpful message in a real world situation
           query
     end

  If you want to some how transform the query results, it looks like this:

   Ash.Query.after_action(query, fn query, results -> 
     {:ok, do_something_with(results)}
   end)
 end

You won’t typically need to use Ash.Query.set_result unless you’re writing something that “pre-empts” the action (like a cache check plus result setter). But keep in mind if you want to use set_result you typically will want to use it in a before_action hook:

Ash.Query.before_action(query, fn query -> 
  case check_cache(query) do
    {:ok, cached_result} ->
      Ash.Query.set_result(query, cached_result)
    :error ->
      ...
  end
end)

Oh okay, that actually makes it a lot clearer to understand.

I guess my only question left would be is if there’s a recommended way to organize multiple preparations for a single Ash Resource.

In the real world case at work, we’d probably be using a lot of Ash.Query.load and Ash.Query.after_action for getting the results of many different aggregates or calculations and changing the shape of the data in order to properly display it in places, so keeping all of those functions group in the same module would be useful.

Why would you need to use after_action to display loaded data? Loading it automatically puts it in the results. I’d suggest that, if you want to do lots of transformation of the results, your better bet would be to use generic actions and not after action hooks.

action :some_name, {:array, :map} do
  constraints [fields: [....]]

  run fn input, _ -> 
    data = run_a_read_action(...)
    transform_the_data(data)
  end
end

Why would you need to use after_action to display loaded data?

Sorry, I worded that wrong.

after_action wouldn’t be for displaying data, we would use it for a read action to transform the data. So for certain API endpoints we can just called Ash.Api.get! or whichever function to call the read action, and that way we can limit the logic in endpoints.

Like a service in Spring that specifically gets data an endpoint will send out contains most of the logic to transform the data, as opposed to doing it in the endpoint itself.

EDIT: But I do see your point on generic actions