With Ash, how can i perform an optimized, costly DB preload like with a preload function in Exto

Ash Forum Ash Questions / Help
,
1

Hi everyone

I’ve recently started to give Ash a try with a greenfield, small project for a customer of mine. So far I really like the abstraction it adds to generating DB migrations, standardization of interfaces and code organization, and this is working fine for this new app (which is basically just a CRUD app again).

However while working on this, I’ve been thinking about how would I use this in the more complex projects I’ve got as well. One stumbling block that I’ve encountered is from an app where users can search their own assets. In that app, when a user lists or queries her assets, we need to run some rather expensive joins to first find the correct assets. Currently that is basically simply a long `Ecto.Query` with a lot of joins, if I see that correctly we can easily express this with Ash as well.

However after finding the assets, we need to preload a couple of related information. This preload query is also quite expensive, since we need to do multiple joins for it. To optimize the query execution, what we are doing is to use a custom preload function from Ecto, as described in the docs. Specifically we rely on the fact that we the function receives the id’s of all parents as a bundle for query optimization.

Now my question comes down to: How can we do that in Ash? I’ve searched through the documentation, but so far I’ve not been able to find a section on how to do this.

Thanks for any pointers and help already :smiley:

1 Like
2

Could you perhaps provide an example of the rules for your preload? This would typically be a job for a calculation, but hard to say for sure without an example :slight_smile:

3

Sure. After giving it a closer look (it has been some years since that code was written) I’ve realized that we actually no longer use a direct preload using a preload function, but instead call a custom function that will make another DB call, and then merges those second results back into the first search result. So roughly like this:

def search_assets(user_id)
  # first search for the assets this user can access
  # search result shall be a list of ecto structs
  search_result = Search.search_user_accessible_assets(user_id)

  # then call another module that loads the related information
  Search.load_shared_with_information(search_result, user_id)
end

defmodule Search do
  import Ecto.Query

  def search_user_accessible_assets(user_id) do
    query = from a in Asset,
       join: .......
    Repo.all(query)
  end

  def load_shared_with_information(search_result, user_id) do
    asset_ids = Enum.map(search_result, & &1.id)
    query = from r in Related,
       join: ....
       join: ....
       where: a.id in ^asset_ids and ....
       select: %{asset_id: a.id, related_id: r.id, info: r.more_info}
    
    related_by_asset_id = Repo.all(query) 
    |> Enum.group_by(& &.asset_id)

    Enum.map(search_result, fn asset -> 
       Map.put(asset, :related_association, Map.get(related_by_asset_id, asset.id))
    end)
  end
end

That is the rough idea. I cannot remember exactly why we replaced the function preload with this logic, but I think it was related to the fact that we actually split the loaded information into two separate association fields for performance optimization.

So probably a calculatoin as you mentioned @zachdaniel ?

Edit: Hit Submit way to early :slight_smile:

4

Sorry for the long delay, but yes calculations sound like exactly what you want :slight_smile: they also support depending on other calculations which allows for the splitting up you mentioned.

5

Hi Zach, thanks for your reply. I’ think I’ll give a try with calculations then.