Filtering an attribute based on whether it contains any string in a constant list

Ash Forum Ash Questions / Help
, ,
1

Hi, I’m trying to filter out records based on whether the file path contains any known filenames. Here is a working example of it:

  attributes do
    attribute :path, :string do
      public? true
      description "The path of the file."
    end

    attribute :repository_full_name, :string do
      public? true
      description "The full name of the repository (login/repo-name)."
    end
  end

  actions do
    read :read do
      argument :repository_full_name, :string do
        constraints match: ~r/^[a-zA-Z0-9-]+\/[a-zA-Z0-9-.]+$/
        allow_nil? false
      end

      filter expr(repository_full_name == ^arg(:repository_full_name))
      filter expr(
          contains(path, "application.properties")
      )
    end
  end

My question is, is it possible to use anything similar to Enum.any?(["application.properties"], &contains(path, &1) inside of the expr? I have attempted this but couldn’t get it to work, I’ve also tried to create a macro to recursively join multiple contains/2 with or/2 but I get some error about relationship paths. I’m new to Elixir and especially new to Ash so pointers on how to achieve something similar to Enum.any? would be greatly appreciated!

2

Right now you can’t do that with the expression syntax, but you can use arbitrary sql in fragments.

filter expr(fragment("""
EXISTS (
    SELECT 1 FROM unnest(?) AS part
    WHERE strpos(part, ?) > 0
)
""", path, "application.properties"))

Something like that ( just made it up off the top of my head you’d need to validate it)

2 Likes
3

Hey Zach, sorry for the late reply

I’m not too sure if this will work for my use case. Context is I’m working with Steampipe and using Postgres as data_layer to retrieve records. I intend to filter attributes which has a match on any string in a list, for example

expr(
  contains(path, "application.properties") or
  contains(path, ".tf") or
  contains(path, ".env") or ...
)

I foresee myself having more of these statements and don’t think it wise to repeat myself, so I tried to create a macro to try to achieve this:

defmodule Macrotest do
  defmacro expandList(list, path, f) do
    expandList(list, path, f, nil)
  end

  defp expandList([], _, _, block), do: block

  defp expandList([head | tail], path, f, nil) do
    expr = quote do: unquote(f)(unquote(path), unquote(head))
    expandList(tail, path, f, expr)
  end

  defp expandList([head | tail], path, f, quoted) do
    expr = quote do: unquote(quoted) or unquote(f)(unquote(path), unquote(head))
    expandList(tail, path, f, expr)
end

# somewhere inside the Ash Resource
filter(expr do
  expandList(["application.properties", ".tf", ".env"], path, :contains)
)

But of course it doesn’t work, I’ve tried to expand the expr macro to understand what it does but I’m not too sure if I’m overengineering this problem

4

You definitely don’t need all of that :smiley: Take a look at custom expressions. Inside of a custom expression you could reduce over the list and build an or expression.

express = [
  expr(a == b),
  expr(b == c),
  expr(c == d),
]

Enum.reduce(tl(exprs), hd(exprs), fn expr, base -> 
  expr(^base or ^expr)
end)
1 Like
5

Hi again, I found some spare time to work on this again and tried your recommendation with Ash.CustomExpression, below is the code I ended up writing:

defmodule Hachiware.Provider.Github.ContainsList do
  use Ash.CustomExpression,
    name: :contains_list,
    arguments: [[:term]],
    predicate?: true

  @filters ~w(.properties .tf)

  def expression(AshPostgres.DataLayer, [path]) do
    {:ok, expr(fragment("contains_list(?)", ^path))}
  end

  def expression(data_layer, [path])
      when data_layer in [
             Ash.DataLayer.Ets,
             Ash.DataLayer.Simple
           ] do
    {:ok, expr(fragment(&__MODULE__.contains_list/1, ^path))}
  end

  def expression(_data_layer, _arguments), do: :unknown

  def contains_list(path) do
    expressions =
      @filters
      |> Enum.map(fn entry -> expr(contains(^path, ^entry)) end)

    Enum.reduce(tl(expressions), hd(expressions), fn e, accum ->
      expr(^accum or ^e)
    end)
  end
end

And on config/config.exs:

config :ash, :custom_expressions, [Hachiware.Provider.Github.ContainsList]

But I end up with an undefined function error

 * ** (Postgrex.Error) ERROR 42883 (undefined_function) function contains_list(text) does not exist
     query: SELECT g0."path", g0."content", g0."repository_full_name", g0."commit_url" FROM "github"."github_repository_content" AS g0 WHERE ((contains_list(g0."path"::text))) AND (g0."repository_full_name"::text = $1::text)
     hint: No function matches the given name and argument types. You might need to add explicit type casts.

Thank you again for your help so far, and would appreciate your experience and guidance in this

6

Fragments allow you to write custom SQL. You’d have to find the SQL that corresponds to what you’re trying to do. I don’t think there is a contains_list function.

If you use main of ash, ash_postgres and ash_sql there is now a has and intersects function that may do what you want.