sezaru

sezaru

How to create a "key" from a query/filter to enable caching?

Is there some way to make an key (can be a hash, a string, whatever, as long as it is consistent with other query with the same filters/params) from an Ash.Query struct?

Basically I want to add caching support for a resource, right now I have this (which is based on @zachdaniel cache in AshHq)

defmodule CacheAgent do
  @moduledoc false

  alias Core.Cnpj.LegalNature

  use Agent

  def start_link(_) do
    Agent.start_link(fn -> nil end, name: __MODULE__)
  end

  def get(query) do
    %{filter: filter, offset: offset, limit: limit, sort: sort} = query

    Agent.get_and_update(__MODULE__, fn state ->
      state = fetch_state!(state)

      output = 
        with {:ok, results} <- Ash.Filter.Runtime.filter_matches(LegalNature, state, filter) do
          results = results |> maybe_apply_offset(offset) |> maybe_apply_limit(limit) |> apply_sort(sort)

          {:ok, results}
        end

      {output, state}
    end)
  end

  defp maybe_apply_offset(results, nil), do: results
  defp maybe_apply_offset(results, offset), do: Enum.drop(results, offset)

  defp maybe_apply_limit(results, nil), do: results
  defp maybe_apply_limit(results, limit), do: Enum.take(results, limit)

  defp apply_sort(results, sort), do: Ash.Sort.runtime_sort(results, sort)

  defp fetch_state!(nil), do: Ash.read!(LegalNature)
  defp fetch_state!(state), do: state
end

Then I use like this in a preparation:

defmodule Core.Cnpj.LegalNature.Actions.CachedRead.Preparations.CheckCache do
  @moduledoc false

  use Ash.Resource.Preparation

  def prepare(query, _, _) do
    Ash.Query.before_action(query, fn query ->
      case CacheAgent.get(query) do
        {:ok, results} -> Ash.Query.set_result(query, {:ok, results})
        {:error, _} -> query
      end
    end)
  end
end

This works, but I don’t like the idea of having to filter the full list (Ash.Filter.Runtime.filter_matches) every time I ask an input from the agent (actually, doing this is slower than just getting the results directly from the DB without cache).

I would much prefer to be able to send the query as a key to a cache and just get the result back, something like this:

Cachex.fetch(:my_cache, query, fn query ->
  case query |> Ash.Query.set_argument(:use_cache?, false) |> Ash.read() do
    {:ok, _} = output -> {:commit, output}
    {:error, _} = error -> {:ignore, error}
  end
end)

This doesn’t work because each query, even if they have the same arguments/filters/etc, will have a different internal structure, so they wont match.

Any good way to make this work well?

First Post!

zachdaniel

zachdaniel

Creator of Ash

I would use the action and input arguments as the cache key as opposed to the query itself.

Last Post!

GrammAcc

GrammAcc

I wrote a custom per-function cache for a currency library in Python a long time ago, and what I found was that caching function calls in a general way is always kind of slow on the key computation step due to all the possible argument values. This is why Python’s built-in @lrucache decorator only works on hashables and doesn’t attempt to apply a stringification algorithm or even sort keyword arguments. Since this is a specific use-case in your application, it might be better to see if you can refactor the workflows to provide extra information that can be used as a simpler cache key for the enclosing function calls instead of trying to cache the query filtering itself.

Not sure if that’s feasible in your case, but if you haven’t explored it yet, it’s usually a lot simpler to cache function calls at the service layer than at the actual data access layer. Cheers!

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
Lily
In templates/appointment/index.html.eex: &lt;%= for appointment &lt;- @appointments do %&gt; &lt;tr&gt; &lt;td&gt;&lt;%= appoi...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement