westbaystars

westbaystars

Filtering with JSON Path (Postgres)

Porting a multi-lingual site from using a native XML database to Postgres with JSONB fields, I’m running into problems when I want to filter by a value in the JSON array.

defmodule Myapp.Sports.Person do
  use Ash.Resource,
    domain: Myapp.Sports,
    data_layer: AshPostgres.DataLayer

  require Ash.Expr

  alias Myapp.Sports.{..., PersonName, ...}
  alias Myapp.Calculations.LangEmbeds

  ...
  actions do
    ...
    read :search_with_name do
      prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
      filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex \"^Maki\" flag \"i\") || ($[*].given_name like_regex \"^Maki\" flag \"i\")'")))

      argument :name, :string do
        allow_nil? false
      end

      argument :lang, Lang do
        allow_nil? false
        default :en
      end
    end
  end

  ...

  attributes do
    ...
    attribute :names, {:array, PersonName}, public?: true, allow_nil?: false
    ...
  end

The above read action works in finding all persons with either surname or given_name beginning with “Maki” in the array of PersonName. However, when I change the \"^Maki\" to a ? and pass in ^arg(:name) as both parameters, they don’t appear to be getting inserted into the final query.

      filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex ? flag \"i\") || ($[*].given_name like_regex ? flag \"i\")'", ^arg(:name), ^arg(:name))))
{:error,
 %Ash.Error.Unknown{
   bread_crumbs: ["Error returned from: Myapp.Sports.Person.search_with_name"], 
   query: "#Query<>", 
   errors: [
     %Ash.Error.Unknown.UnknownError{
       error: "** (Postgrex.Error) ERROR 42601 (syntax_error) syntax error at or near \"$1\" of jsonpath input\n\n    query: SELECT p0.\"attributes\", p0.\"id\", p0.\"links\", p0.\"aliases\", p0.\"names\", p0.\"updated_at\", p0.\"inserted_at\", p0.\"sid\", p0.\"xid\", p0.\"bridges\", p0.\"date_of_birth\", p0.\"date_of_death\", p0.\"drafts\", p0.\"gender\", p0.\"location_of_birth_id\", p0.\"location_of_death_id\", p0.\"stints\" FROM \"persons\" AS p0 WHERE ((to_json(names)::jsonb @@ '($[*].surname like_regex $1 flag \"i\") || ($[*].given_name like_regex $2 flag \"i\")'))",
       field: nil,
       value: nil,
       splode: Ash.Error,
       bread_crumbs: ["Error returned from: MyApp.Sports.Person.search_with_name"],
       vars: [],
       path: [],
       stacktrace: #Splode.Stacktrace<>,
       class: :unknown
     }
   ]
 }}

I’ve tried a number of variants of this approach, but none seem to pass the parameters to the fragment.

Any hints would be greatly appreciated.

Marked As Solved

westbaystars

westbaystars

This was the clue that I needed to get it to work. I just moved the query building to the Postgres side.

create or replace function person_name_search(person_names jsonb, name text)
  returns boolean as $$
  declare
    name text = coalesce(nullif(trim(name), ''), 'a');
    jsonpath jsonpath = concat(
      '($[*].surname like_regex "^', name, '" flag "i") || ',
      '($[*].given_name like_regex "^', name, '" flag "i")');
  begin
    return person_names @@ jsonpath;
  end; $$
  language plpgsql;

I added a check to make sure that the passed parameter isn’t NULL or empty, defaulting to searching for all names that begin with a. I’m doing that on the Elixir side as well, so it should never be NULL, but it seems like a good idea to verify either way.

Then I call the person_name_search function in a filter with fragment:

    read :search_with_name do
      prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
      filter (expr(
        fragment("person_name_search(to_json(names)::jsonb, ?)", ^arg(:search_name))
        ||
        fragment("person_name_search(to_json(aliases)::jsonb, ?)", ^arg(:search_name))
      ))

      argument :search_name, :string do
        allow_nil? false
        default "a"
      end

      argument :lang, Lang do
        allow_nil? false
        default :en
      end
    end

I believe that the search_name parameter is still being made safe before passing it on. Please correct me if someone thinks not.

This is expanded from my original question to include searching the :aliases field, which is also an array of PersonName embeds. Once I had this working for :names, oring that with calling it with :aliases worked as expected.

A query through the Sports domain yields:

Sports.search_persons("牧", :en)

{ok: [
   #Myapp.Sports.Person<
     name: #Myapp.Sports.PersonName<
       __meta__: #Ecto.Schema.Metadata<:built, "">,
       lang: :en,
       given_name: #Ash.CiString<"Haruki">,
       middle_names: nil,
       surname: #Ash.CiString<"Makino">,
       full_name: #Ash.CiString<"Haruki Makino">,
       registered_name: nil,
       given_name_ruby: nil,
       middle_names_ruby: nil,
       surname_ruby: nil,
       full_name_ruby: nil,
       registered_name_ruby: nil,
       aggregates: %{},
       calculations: %{},
       ...
     >,
     ...
     id: 47326,
     sid: "makino-haruki",
     ...
   >,
   #Myapp.Sports.Person<
     name: #Myapp.Sports.PersonName<
       __meta__: #Ecto.Schema.Metadata<:built, "">,
       lang: :en,
       given_name: #Ash.CiString<"Shoya">,
       middle_names: nil,
       surname: #Ash.CiString<"Makino">,
       full_name: #Ash.CiString<"Shoya Makino">,
       registered_name: nil,
       given_name_ruby: nil,
       middle_names_ruby: nil,
       surname_ruby: nil,
       full_name_ruby: nil,
       registered_name_ruby: nil,
       aggregates: %{},
       calculations: %{},
       ...
     >,
     ...
     id: 47353,
     sid: "makino-shoya",
     ...
   >,
   ...
 ]}

Thank you again for the hint to build the jsonpath portion separately. That did it.

Take care.

Also Liked

zachdaniel

zachdaniel

Creator of Ash

The primary issue here is that the whole value after @@ is a single string. The fragment logic is replacing ? with the parameter replacements, but postgres doesn’t accept parameters just inside of a string like that.

Try replacing your filter with this preparation. Note that I’ve also used the expression syntax to interpolate the names reference. You should do this instead of putting in names literally as that can cause ambiguous queries.

prepare fn query, _ -> 
  jsonpath = '($[*].surname like_regex \"^#{query.arguments.name}\" flag \"i\") || ($[*].given_name like_regex \"^#{query.arguments.name}\" flag \"i\")'

  Ash.Query.filter(query, fragment("to_json(?)::jsonb @@ ?", names, jsonpath)
end

Where Next?

Popular in Questions Top

hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New

Other popular topics Top

New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

We're in Beta

About us Mission Statement