Qry - Query your domain

Your Libraries & OS Mentoring Libraries
, , ,
1

I created a project named Qry that allows you to query your domain with a nice syntax, returning results as nested maps – a concept similar to GraphQL or Ecto’s preload.

This is a personal need, but wanted to share it in case others find it useful. It’s in a working alpha state, but lots of ideas to come like improved error handling and async data loading (like dataloader).

There’s an example at the top of the docs: qry v0.3.0 — Documentation

Would love feedback.

10 Likes
2

As feedback - a few more introductory paragraphs about:

  • what part of the “domain querying” problem domain this library is intended to solve
  • what alternatives (if any) are available, and why they might not be quite suitable for this
  • what use cases this has

would be very helpful. I think it certainly covers a gap in most libraries, especially in backends with complicated ephemeral and internal state (a lot of ORM systems assume a backing database), but a couple of paragraphs would market that a lot better.

2 Likes
3

The main benefits of this library are the same benefits of GraphQL – declaratively fetching data, frees you from thinking about the implementation details of how that data is cobbled together. The exact shape of our database almost never matches what the end user sees. As you said, sometimes data is ephemeral, derived, excluded, pulled from other sources/APIs, etc.

I don’t know of any other solutions for this. Ecto has preload for the database layer, but it doesn’t solve the problem described above. And of course there’s the absinthe library, but that’s if you’re using GraphQL.

A lot of what I see goes like this: A requirement comes along to ensure deleted users aren’t removed from the database. So instead of using Ecto directly, a function is created:

def fetch_user(id) do
  User |> where([u], u.id == ^id and not u.deleted) |> Repo.one()
end

Devs can use this function and not have to worry about how a user is fetched. There are a lot of growing pains with this style. For example, consider another function that gets added:

def fetch_org(id) do
  Org |> where([o], o.id) |> preload(:users) |> Repo.one()
end

Uh oh, we forgot to exclude deleted users. Also, the caller of fetch_org/1 might not need the org’s users.

With Qry, you’ll write the implementation functions for how data is loaded, then you’ll just use the main query interface everywhere:

# an org with users
Domain.query({:org, %{id: 1}, [:users]})

# just an org
Domain.query({:org, %{id: 1}})

# all users
Domain.query(:users)

In the Domain.fetch functions for fetching users, you’ll exclude deleted users, and know that everywhere Domain.query is called, and no matter what part of the graph users are attached, it’ll exclude the deleted ones.

That’s just one example, but in general bigger/older apps tend to struggle with data loading growing pains.

4 Likes
4

I really like the idea of dataloader for internal use within a project. I’m wondering if this library does anything in regards to preventing n+1 queries, say you’re indeed loading thing from the db eventually. The docs doesn’t really show how this would eventually interact with a database.

5

Glad you like the idea! :slight_smile:

I would say preventing N+1s isn’t the job of Qry, but they’ll likely be avoided as a side-effect.

I’ve been on the fence of a design decision that is related to N+1s: whether to make the fetch functions alway fetch in batch. There are pros/cons here which I should enumerate. One of which (I think) would require you to create a schema-like file. Right now, Qry is a very light-weight wrapper.

You’re right that I should show a good example of how you might use this with Ecto. I was trying to do the simplest thing (hard-coded data) to emphasize that Qry is unconcerned with how your data is loaded.

Thanks for the feedback. I’ll put an example together.

6

Yeah, I totally get the idea that it should be agnostic to the actual data store. But on the other hand querying a database is the common case and that needs to be doable reasonably efficient. In the end it’s also not just a problem with databases, but anything where fetching items in certain batches is benefitial.

7

For the org/user queries above, the fetch implementations for Ecto could look something like this:

defmodule Db do
  use Ecto.Repo, ...
end

defmodule Db.Org do
  use Ecto.Schema

  schema "orgs" do
    has_many(:users, Db.User)
  end
end

defmodule Db.User do
  use Ecto.Schema

  schema "users" do
    field(:deleted, :boolean)
    belongs_to(:org, Db.Org)
  end
end

defmodule Domain do
  use Qry.Repo

  import Ecto.Query

  def fetch(:org, args, %{}) do
    query = Db.Org
    query = if args[:id], do: query |> where([o], o.id == ^args.id), else: query
    query |> Db.one()
  end

  def fetch(:users, args, %{}) do
    query = Db.User |> where([u], not u.deleted)
    query = if args[:org_id], do: query |> where([u], u.org_id == ^args.org_id), else: query
    query |> Db.all()
  end

  def fetch(%Db.Org{} = org, :users, args, %{}) do
    args = args |> Map.put(:org_id, org.id)
    fetch(:users, args, context)
  end
end

Fetching users as a subfield of org ultimately runs the same fetch(:users, args, context) function, but with an org_id arg. That function can now be the source-of-truth for fetching users, regardless of where users are in the graph.