Flop - Filtering, sorting and pagination for Ecto

[0.20.3] - 2023-06-23

Changed

  • Flop.count/3 will now wrap queries that have GROUP BY clauses in a
    subquery.

Fixed

  • Fixed cursor-based pagination on composite types.
4 Likes

Hi, I’m currently giving your library a try, and I’m really impressed so far, so a big thank-you!

Currently, I’m having a bit of an issue trying to get compound keys to work, because for some reason it tries to query the database directly. I’m not sure where I’m branching off from the documentation.

Update

It seems to work fine, if I call Flop.validate_and_run/3

Code
# In the schema
  @derive {
    Flop.Schema,
    sortable: [:first_name, :last_name],
    filterable: [:dairy_id, :search],
    default_order: %{
      order_by: [:last_name, :first_name],
      order_directions: [:asc, :asc]
    },
    compound_fields: [search: [:first_name, :last_name, :email, :phone_number, :id]]
  }

# Flop (created by Flop.validate/2)
%Flop{
  after: nil,
  before: nil,
  first: nil,
  last: nil,
  limit: 50,
  offset: nil,
  order_by: [:last_name, :first_name],
  order_directions: [:asc, :asc],
  page: nil,
  page_size: nil,
  filters: [%Flop.Filter{field: :search, op: :ilike, value: "elm"}]
}
# Query error
** (Postgrex.Error) ERROR 42703 (undefined_column) column m0.search does not exist

    query: SELECT ... FROM "members" AS m0 WHERE (m0."search" ILIKE $1) ORDER BY m0."last_name", m0."first_name" LIMIT $2

You didn’t post the actual function call you make, but usually this means that you forgot to pass the for option to the function that makes the query:

opts = [for: Pet]
flop = Flop.validate!(params, opts)
Flop.run(query, flop, opts)

And similarly with any other function that makes a query. Without this option, Flop doesn’t know in which schema to look up the field configuration, and it will just use the field name from the filter as it is.

Another issue I’m spotting in your code is that you added the id column to the compound field config. like only works on string (text, varchar) columns. Flop does not cast integer or binary columns as a string automatically, which means you’ll get a cast error here. If you want to allow partial matches on a binary ID column, the best way to do this at the moment would be a custom field.

1 Like

I see, I was looking at the docs for Flop.run/3 and thought this was handled automatically. Thank you for the prompt reply! :smile:

I can see how that can be confusing. I updated the example in the documentation and added some notes about the for option here and there: docs/run for by woylie · Pull Request #352 · woylie/flop · GitHub. I hope this will make it clearer for the next one. Thanks for bringing this up!

3 Likes

Flop version 0.21.0 is now available. This release brings significant enhancements to the casting and validation of filter values. These changes might require some adjustments to your schema configuration and to code that directly handles filter values. Please carefully review the upgrade notes linked below.

Added

  • Introduced operators as a new option for restricting acceptable operators
    for a custom field.
  • Added bindings option for custom fields, allowing required named bindings to
    be added via Flop.with_named_bindings/4.
  • The ecto_type option on join and custom fields now supports
    references: {:from_schema, MySchema, :some_field}.
  • The ecto_type option now supports a convenient syntax for adhoc enums:
    {:ecto_enum, [:one, :two]}.
  • Improved documentation with added type definitions: t:Flop.Schema.option/0,
    t:Flop.Schema.join_field_option/0, t:Flop.Schema.custom_field_option/0,
    and t:Flop.Schema.ecto_type/0, describing options available when deriving
    the Flop.Schema protocol.

Changed

  • Breaking change: Filter values are now dynamically cast based on the
    field type and operator, instead of allowing any arbitrary filter value. This
    change ensures that invalid filter values cause validation errors instead of
    cast errors.
  • The options for deriving the Flop.Schema protocol and for use Flop
    now undergo stricter validation with NimbleOptions.
  • Flop.Cursor.encode/1 now explicitly sets the minor version option for
    :erlang.term_to_binary/2 to 2, aligning with the new default in OTP 26.
    Before, this option was not set at all.
  • Added a decoded_cursor field to the Flop struct. This field temporarily
    stores the decoded cursor between validation and querying and is
    discarded when generating the meta data.

Deprecated

  • The tuple syntax for defining join fields has been deprecated in favor of a
    keyword list.

Fixed

  • Resolved an issue where setting replace_invalid_params to true still
    caused validation errors for pagination and sorting parameters due to cast
    errors, instead of defaulting to valid parameters.
  • Fixed the type specification for Flop.Filter.allowed_operators/1.
4 Likes

This looks amazing! However, I feel like I’m missing something: is there any way of giving a class atribute to the table generated by Flop.Phoenix.table() at the call site, without having to use global configuration to do it?

Have a look here.

You can pass any of these options to the opts attribute of any call to Flop.Phoenix.table/1. For example, you can do:

<.Flop.Phoenix.table opts={table_attrs: [class: "some-class"]}>

It’s obviously easier if you extract a private table_opts function in whichever view you’re going to using it in. It could also be merged with a default set of options.

appreciation post for such an amazing library, Thank you

1 Like