westbaystars

westbaystars

Filtering with JSON Path (Postgres)

Porting a multi-lingual site from using a native XML database to Postgres with JSONB fields, I’m running into problems when I want to filter by a value in the JSON array.

defmodule Myapp.Sports.Person do
  use Ash.Resource,
    domain: Myapp.Sports,
    data_layer: AshPostgres.DataLayer

  require Ash.Expr

  alias Myapp.Sports.{..., PersonName, ...}
  alias Myapp.Calculations.LangEmbeds

  ...
  actions do
    ...
    read :search_with_name do
      prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
      filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex \"^Maki\" flag \"i\") || ($[*].given_name like_regex \"^Maki\" flag \"i\")'")))

      argument :name, :string do
        allow_nil? false
      end

      argument :lang, Lang do
        allow_nil? false
        default :en
      end
    end
  end

  ...

  attributes do
    ...
    attribute :names, {:array, PersonName}, public?: true, allow_nil?: false
    ...
  end

The above read action works in finding all persons with either surname or given_name beginning with “Maki” in the array of PersonName. However, when I change the \"^Maki\" to a ? and pass in ^arg(:name) as both parameters, they don’t appear to be getting inserted into the final query.

      filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex ? flag \"i\") || ($[*].given_name like_regex ? flag \"i\")'", ^arg(:name), ^arg(:name))))
{:error,
 %Ash.Error.Unknown{
   bread_crumbs: ["Error returned from: Myapp.Sports.Person.search_with_name"], 
   query: "#Query<>", 
   errors: [
     %Ash.Error.Unknown.UnknownError{
       error: "** (Postgrex.Error) ERROR 42601 (syntax_error) syntax error at or near \"$1\" of jsonpath input\n\n    query: SELECT p0.\"attributes\", p0.\"id\", p0.\"links\", p0.\"aliases\", p0.\"names\", p0.\"updated_at\", p0.\"inserted_at\", p0.\"sid\", p0.\"xid\", p0.\"bridges\", p0.\"date_of_birth\", p0.\"date_of_death\", p0.\"drafts\", p0.\"gender\", p0.\"location_of_birth_id\", p0.\"location_of_death_id\", p0.\"stints\" FROM \"persons\" AS p0 WHERE ((to_json(names)::jsonb @@ '($[*].surname like_regex $1 flag \"i\") || ($[*].given_name like_regex $2 flag \"i\")'))",
       field: nil,
       value: nil,
       splode: Ash.Error,
       bread_crumbs: ["Error returned from: MyApp.Sports.Person.search_with_name"],
       vars: [],
       path: [],
       stacktrace: #Splode.Stacktrace<>,
       class: :unknown
     }
   ]
 }}

I’ve tried a number of variants of this approach, but none seem to pass the parameters to the fragment.

Any hints would be greatly appreciated.

Marked As Solved

westbaystars

westbaystars

This was the clue that I needed to get it to work. I just moved the query building to the Postgres side.

create or replace function person_name_search(person_names jsonb, name text)
  returns boolean as $$
  declare
    name text = coalesce(nullif(trim(name), ''), 'a');
    jsonpath jsonpath = concat(
      '($[*].surname like_regex "^', name, '" flag "i") || ',
      '($[*].given_name like_regex "^', name, '" flag "i")');
  begin
    return person_names @@ jsonpath;
  end; $$
  language plpgsql;

I added a check to make sure that the passed parameter isn’t NULL or empty, defaulting to searching for all names that begin with a. I’m doing that on the Elixir side as well, so it should never be NULL, but it seems like a good idea to verify either way.

Then I call the person_name_search function in a filter with fragment:

    read :search_with_name do
      prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
      filter (expr(
        fragment("person_name_search(to_json(names)::jsonb, ?)", ^arg(:search_name))
        ||
        fragment("person_name_search(to_json(aliases)::jsonb, ?)", ^arg(:search_name))
      ))

      argument :search_name, :string do
        allow_nil? false
        default "a"
      end

      argument :lang, Lang do
        allow_nil? false
        default :en
      end
    end

I believe that the search_name parameter is still being made safe before passing it on. Please correct me if someone thinks not.

This is expanded from my original question to include searching the :aliases field, which is also an array of PersonName embeds. Once I had this working for :names, oring that with calling it with :aliases worked as expected.

A query through the Sports domain yields:

Sports.search_persons("牧", :en)

{ok: [
   #Myapp.Sports.Person<
     name: #Myapp.Sports.PersonName<
       __meta__: #Ecto.Schema.Metadata<:built, "">,
       lang: :en,
       given_name: #Ash.CiString<"Haruki">,
       middle_names: nil,
       surname: #Ash.CiString<"Makino">,
       full_name: #Ash.CiString<"Haruki Makino">,
       registered_name: nil,
       given_name_ruby: nil,
       middle_names_ruby: nil,
       surname_ruby: nil,
       full_name_ruby: nil,
       registered_name_ruby: nil,
       aggregates: %{},
       calculations: %{},
       ...
     >,
     ...
     id: 47326,
     sid: "makino-haruki",
     ...
   >,
   #Myapp.Sports.Person<
     name: #Myapp.Sports.PersonName<
       __meta__: #Ecto.Schema.Metadata<:built, "">,
       lang: :en,
       given_name: #Ash.CiString<"Shoya">,
       middle_names: nil,
       surname: #Ash.CiString<"Makino">,
       full_name: #Ash.CiString<"Shoya Makino">,
       registered_name: nil,
       given_name_ruby: nil,
       middle_names_ruby: nil,
       surname_ruby: nil,
       full_name_ruby: nil,
       registered_name_ruby: nil,
       aggregates: %{},
       calculations: %{},
       ...
     >,
     ...
     id: 47353,
     sid: "makino-shoya",
     ...
   >,
   ...
 ]}

Thank you again for the hint to build the jsonpath portion separately. That did it.

Take care.

Also Liked

zachdaniel

zachdaniel

Creator of Ash

The primary issue here is that the whole value after @@ is a single string. The fragment logic is replacing ? with the parameter replacements, but postgres doesn’t accept parameters just inside of a string like that.

Try replacing your filter with this preparation. Note that I’ve also used the expression syntax to interpolate the names reference. You should do this instead of putting in names literally as that can cause ambiguous queries.

prepare fn query, _ -> 
  jsonpath = '($[*].surname like_regex \"^#{query.arguments.name}\" flag \"i\") || ($[*].given_name like_regex \"^#{query.arguments.name}\" flag \"i\")'

  Ash.Query.filter(query, fragment("to_json(?)::jsonb @@ ?", names, jsonpath)
end

Where Next?

Popular in Questions Top

jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

Other popular topics Top

openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44139 214
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement