westbaystars
Filtering with JSON Path (Postgres)
Porting a multi-lingual site from using a native XML database to Postgres with JSONB fields, I’m running into problems when I want to filter by a value in the JSON array.
defmodule Myapp.Sports.Person do
use Ash.Resource,
domain: Myapp.Sports,
data_layer: AshPostgres.DataLayer
require Ash.Expr
alias Myapp.Sports.{..., PersonName, ...}
alias Myapp.Calculations.LangEmbeds
...
actions do
...
read :search_with_name do
prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex \"^Maki\" flag \"i\") || ($[*].given_name like_regex \"^Maki\" flag \"i\")'")))
argument :name, :string do
allow_nil? false
end
argument :lang, Lang do
allow_nil? false
default :en
end
end
end
...
attributes do
...
attribute :names, {:array, PersonName}, public?: true, allow_nil?: false
...
end
The above read action works in finding all persons with either surname or given_name beginning with “Maki” in the array of PersonName. However, when I change the \"^Maki\" to a ? and pass in ^arg(:name) as both parameters, they don’t appear to be getting inserted into the final query.
filter (expr(fragment("to_json(names)::jsonb @@ '($[*].surname like_regex ? flag \"i\") || ($[*].given_name like_regex ? flag \"i\")'", ^arg(:name), ^arg(:name))))
{:error,
%Ash.Error.Unknown{
bread_crumbs: ["Error returned from: Myapp.Sports.Person.search_with_name"],
query: "#Query<>",
errors: [
%Ash.Error.Unknown.UnknownError{
error: "** (Postgrex.Error) ERROR 42601 (syntax_error) syntax error at or near \"$1\" of jsonpath input\n\n query: SELECT p0.\"attributes\", p0.\"id\", p0.\"links\", p0.\"aliases\", p0.\"names\", p0.\"updated_at\", p0.\"inserted_at\", p0.\"sid\", p0.\"xid\", p0.\"bridges\", p0.\"date_of_birth\", p0.\"date_of_death\", p0.\"drafts\", p0.\"gender\", p0.\"location_of_birth_id\", p0.\"location_of_death_id\", p0.\"stints\" FROM \"persons\" AS p0 WHERE ((to_json(names)::jsonb @@ '($[*].surname like_regex $1 flag \"i\") || ($[*].given_name like_regex $2 flag \"i\")'))",
field: nil,
value: nil,
splode: Ash.Error,
bread_crumbs: ["Error returned from: MyApp.Sports.Person.search_with_name"],
vars: [],
path: [],
stacktrace: #Splode.Stacktrace<>,
class: :unknown
}
]
}}
I’ve tried a number of variants of this approach, but none seem to pass the parameters to the fragment.
Any hints would be greatly appreciated.
Marked As Solved
westbaystars
This was the clue that I needed to get it to work. I just moved the query building to the Postgres side.
create or replace function person_name_search(person_names jsonb, name text)
returns boolean as $$
declare
name text = coalesce(nullif(trim(name), ''), 'a');
jsonpath jsonpath = concat(
'($[*].surname like_regex "^', name, '" flag "i") || ',
'($[*].given_name like_regex "^', name, '" flag "i")');
begin
return person_names @@ jsonpath;
end; $$
language plpgsql;
I added a check to make sure that the passed parameter isn’t NULL or empty, defaulting to searching for all names that begin with a. I’m doing that on the Elixir side as well, so it should never be NULL, but it seems like a good idea to verify either way.
Then I call the person_name_search function in a filter with fragment:
read :search_with_name do
prepare build(load: [name: expr(%{lang: ^arg(:lang)})])
filter (expr(
fragment("person_name_search(to_json(names)::jsonb, ?)", ^arg(:search_name))
||
fragment("person_name_search(to_json(aliases)::jsonb, ?)", ^arg(:search_name))
))
argument :search_name, :string do
allow_nil? false
default "a"
end
argument :lang, Lang do
allow_nil? false
default :en
end
end
I believe that the search_name parameter is still being made safe before passing it on. Please correct me if someone thinks not.
This is expanded from my original question to include searching the :aliases field, which is also an array of PersonName embeds. Once I had this working for :names, oring that with calling it with :aliases worked as expected.
A query through the Sports domain yields:
Sports.search_persons("牧", :en)
{ok: [
#Myapp.Sports.Person<
name: #Myapp.Sports.PersonName<
__meta__: #Ecto.Schema.Metadata<:built, "">,
lang: :en,
given_name: #Ash.CiString<"Haruki">,
middle_names: nil,
surname: #Ash.CiString<"Makino">,
full_name: #Ash.CiString<"Haruki Makino">,
registered_name: nil,
given_name_ruby: nil,
middle_names_ruby: nil,
surname_ruby: nil,
full_name_ruby: nil,
registered_name_ruby: nil,
aggregates: %{},
calculations: %{},
...
>,
...
id: 47326,
sid: "makino-haruki",
...
>,
#Myapp.Sports.Person<
name: #Myapp.Sports.PersonName<
__meta__: #Ecto.Schema.Metadata<:built, "">,
lang: :en,
given_name: #Ash.CiString<"Shoya">,
middle_names: nil,
surname: #Ash.CiString<"Makino">,
full_name: #Ash.CiString<"Shoya Makino">,
registered_name: nil,
given_name_ruby: nil,
middle_names_ruby: nil,
surname_ruby: nil,
full_name_ruby: nil,
registered_name_ruby: nil,
aggregates: %{},
calculations: %{},
...
>,
...
id: 47353,
sid: "makino-shoya",
...
>,
...
]}
Thank you again for the hint to build the jsonpath portion separately. That did it.
Take care.
Also Liked
zachdaniel
The primary issue here is that the whole value after @@ is a single string. The fragment logic is replacing ? with the parameter replacements, but postgres doesn’t accept parameters just inside of a string like that.
Try replacing your filter with this preparation. Note that I’ve also used the expression syntax to interpolate the names reference. You should do this instead of putting in names literally as that can cause ambiguous queries.
prepare fn query, _ ->
jsonpath = '($[*].surname like_regex \"^#{query.arguments.name}\" flag \"i\") || ($[*].given_name like_regex \"^#{query.arguments.name}\" flag \"i\")'
Ash.Query.filter(query, fragment("to_json(?)::jsonb @@ ?", names, jsonpath)
end
Popular in Questions
Other popular topics
Latest Ash Threads
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security









