Inspect behaviour seems to have changed with more recent versions of Elixir

Questions / Help
1

As part of my our test suites, we’ve designed small helpers such as:

  defmacro assert_query_equal(q, right) do
    quote do
      unquote(q)
      |> inspect(pretty: false, width: :infinity, limit: :infinity, printable_limit: :infinity)
      |> assert_equal(unquote(right))
    end
  end

Within our tests, we use it like this:

      pattern
      |> Query.with_email_like()
      |> assert_query_equal(
        "#Ecto.Query<from u0 in EAICore.Schema.User, as: :users, " <>
          "where: fragment(\"lower(f_unaccent(?)) ILIKE lower(f_unaccent(?))\", u0.email, ^\"#{similar_pattern}\"), " <>
          "group_by: [u0.id], " <>
          "order_by: [asc: u0.inserted_at]>"
      )

We really like this way of testing, because we’re testing what our queries look like rather than hitting a database - and tests are really fast.

However, it seems that inspect might have recently changed, because it is inserting sequences such as \n.
It could be fine, the problem being that we’re using Faker to generate inputs for these query tests, and they’re not necessarily the same lengths - so the inspect output changes not only on these parameters, but also on how it builds these \n sequences…

I’ve tried forcing something a bit more “deterministic” an output by using options such as pretty: false, width: :infinity, limit: :infinity, printable_limit: :infinity - my first intuition was that in particular pretty: false would do the trick. But no.

It could be that the problem is in the implementation of the inspect behaviour of Ecto Query, Fragment, etc. And I am not entirely sure how to approach this. In particular, how Algebraic documents are constructed, and whether or not there is a way to pass a custom_options to inspect that would propagate to whatever Ecto is doing…

Any ideas on how to force inspect never to add new lines, and instead simply build a long string?

Thanks.

2

You could strip newlines before comparing. Inspect is not really meant to a stable representation of the inspected value, especially custom inspect implementations like the one you see with Ecto.Query.

But I’d argue this is not a great way to test queries. There are various ways the underlying query could be edited without changing how it works, but changing how inspect would print it – that’s brittle and will limit how you can refactor the tested code.

You could go with MyApp.Repo.to_sql(:all, query) to generate SQL to compare, which would include less irrelevant details. But again there can by many permutations of an SQL query, which result in the same data to be queried from a db (given same db state).

E.g. consider the following queries:
SELECT id FROM table WHERE inserted_at > ? AND inserted_at = updated_at
SELECT id FROM table WHERE inserted_at = updated_at AND inserted_at > ?

The difference between the two could easily be the result of reordering of a few lines of code, with no change in behaviour.

6 Likes
3

I too think you should just drop testing queries in this way. Just go wild and assert on their struct fields.

2 Likes
4

Great suggestions @LostKobrakai .

Dropping new lines isn’t an option unfortunately, as some parameters passed to queries might have newline sequences that we really don’t want to touch. Also, it seems that the inspect behaviors not only add new lines, but space characters based on nesting…

The problem is that to_sql requires Repo to be ready - and in the application where these tests live, there’s no repo. Schemas are separated in different apps.

I do agree with @dimitarvp that testing the structs would be the solution here - we’re really as close to unit tests as possible…

I’ll have a look at how difficult it would be to bring the Repo in the app that has all the Schemas, as converting it to SQL still seems to be the most readable form to see in a test…

But back on the original topic - do you know if there’s a way to prevent Ecto from “prettifying” anything?

Thanks!

5

It isn’t Ecto doing the prettifying, it’s Inspect. In fact, the Inspect.Algebra docs directly cite a paper called “Strictly Pretty” (2000, Lindig), which should be a pretty good indication that this representation is not meant to be un-prettifiable. :sweat_smile:

I second the misgivings about this approach in general in this thread—inspect is a developer tool with no formal specification; indeed, engineered to flexibly change its output based on context and upstream implementation in structs, and relying on it for automated correctness checks is probably not great.

That being said, I think you could look into doing something like setting a custom inspect function via Inspect.Opts.default_inspect_fun/1 to force limit: :infinity, width: infinity, pretty: false, printable_limit: :infinity, structs: false everywhere. It’s hacky and brittle, but so is this approach to unit testing, and may work for your situation.

3 Likes
6

Unfortunately despite Inspect being a protocol most libraries implement it for their own data structures. If you try to implement the inspect protocol for Ecto data structures you’ll get a compiler warning for redefining the inspect protocol for that struct.

On the one hand this makes sense because Ecto knows best about what is private to the implementation and likely knows a lot more about the edge cases around building an inspectable representation of the data.

But as mentioned by a lot of the people here already the approach is a little sketchy. The Ecto.Query struct should essentially be considered as an opaque data structure that ecto uses to produce SQL queries. If they change the representation you’d be in a bit of trouble. And more importantly you aren’t testing that the query is correct in any way that matters… if you forget a not in the where, what highlights that for you? Doesn’t really matter that they are quick if they aren’t helpful.

But you know more about your use case than we do! If all bets are off you could consider File.read! ing the source code in your tests and asserting that the function is written how you expect. As that seems to basically be what you are testing

1 Like
7

@doodloo I’m wondering if you found a solution for this. I’m kinda in the same situation. I agree with the folks that using inspect is not the best way, but we do have some old tests using this approach. Thanks!

8

Using inspect isn’t the best way, but that’s the only easy way for checking against the structure… We didn’t find another way and resorted to refactoring.

1 Like
9

I used your method once or twice but as per above I found asserting on the fields to be better because it’s easier to encode expectations.