Looking for projects that use Ecto. Purpose: to improve and document a testing library

[marick]

I’m currently a fair way into a project to produce a flexible domain-specific language for testing code that uses Ecto. The “elevator pitch” is:

It’s easier to write product code that uses Ecto than it is to test that code. It should be the other way around.

EctoTestDSL (catchy, huh?) was originally designed for my own (idiosyncratic) use of Ecto, but I’ve been generalizing it to support other styles. To do that, I need more examples of other styles. So:

Do you know of a well-tested project that uses Ecto? I would like to fork it and rewrite the tests. Note: I don’t expect project owners to switch over to EctoTestDSL. I do expect to use the forks as examples in the documentation about how to adopt/adapt the library.

---

If you’re curious about the library, see WHY.md. It’s 1000 words, with this structure:

Why an Ecto Test DSL?

• A minimal example
• Concise and repeated assertions
• Workflows define what gets done and what gets checked
• Associations and foreign keys
• Generating test data for ordinary ExUnit tests
• Summary

Here’s an example that shows off some features without explaining them.

  use EctoTestDSL.Variants.PhoenixGranular.Insert
  
  def create_test_data do 
    start(
      module_under_test: Schemas.Animal,
      repo: App.Repo
    ) |>

    field_transformations(
      as_cast: [:name, :notes, :lock_uuid, :species_id]
    ) |>

    workflow(                                 :success,
      note_free: [params(name: "Bossie",
                         notes: "",
                         species_id: id_of(bovine: Insert.Species))
                 ],

      with_notes: [params_like(:note_free, except: [notes: "gelding"])]
    ) |>

    workflow(                                :constraint_error,
       duplicate_name: [
         insert_twice(:note_free),
         constraint_changeset(error: [name: "has already been taken"])
       ]
    )

[benwilson512]

On the subject of Ecto being hard to test: I think people often make the mistake of trying to create records in the database independent of their context modules, and so they are constantly needing to create complex “factory like” utilities to create the desired records.

Instead, I’ve found that having APIs in my contexts that allow me to create what I want not only improves the clarity and utility of my tests, but also improves the general usefulness of my context functions going forward. If a context is hard to test, it’s going to be hard to use generally.

That isn’t to say I’m against what you have here, to be frank I don’t fully grok what you have here. But, basically, I’ve moved away from the notion of “testing ecto” and moved towards testing my contexts. Testing that directly relates to ecto can still be sometimes useful in certain complex cases where I want some sort of unit test, but in those cases it’s often the details of ecto that matter. I’d rather have the details of ecto laid bare than hidden behind a DSL.

[marick]

I think we are not necessarily disagreeing.

But when it comes to such questions, I think examples clarify. That’s why I’d like to write tests for your code. Is that possible? I’d be willing to sign an NDA if it were understood I could then write a disguised example using the same style for public demonstration.

[al2o3cr]

I agree that setup can be a hassle, but I’m not convinced that this approach is a good one. At first read, it feels like I’d end up with every file in the ecto_test_dsl package open in my editor when something goes wrong, because the workflows etc make it hard to follow control flow.

Formatting note: you’ll want to figure out how to make mix format tolerate the patterns you’re building, especially things like this hanging indent on workflow declarations:

marick/ecto_test_dsl/blob/cdaaf72eef53687fa5745bc60fe5553e0c2c111b/test/variants/phoenix_granular_insert/validation_order_test.exs#L46

One thing that is currently a major pain point for my team with ex_machina is associations that refer to other parts of the object graph. For instance, let’s imagine an ecommerce store:

• Store, Customer, and Order are our key schemas
• Customer belongs to a Store
• Order belongs to a Store
• Order belongs to a Customer

Right now, if I’m trying to build an Order in ex_machina I have a problem - attaching an unsaved Store to both an unsaved Customer and Order works fine, but triggers a validation error when Ecto tries to insert the Store a second time (because saving the one attached to Customer doesn’t mutate the struct attached to Order).

I’d be curious to see how your library would solve this problem.

[marick]

@al2o3cr

I agree that setup can be a hassle, but I’m not convinced that this approach is a good one. At first read, it feels like I’d end up with every file in the ecto_test_dsl package open in my editor when something goes wrong, because the workflows etc make it hard to follow control flow.

It is my hope that problem is small given good error messages and a tracing facility that shows you control flow. Note: workflows are few and not branchy.

Tracing is crude as of yet, as shown below. You can cap the maximum depth, but that’s all the abbreviation right now.

[marick]

@al2o3cr

I’d be curious to see how your library would solve this problem.

Can you send me the test you want to see pass? I’m not sure of the steps in the test. You can use marick@exampler.com for email.

[marick]

@al2o3cr

Formatting note: you’ll want to figure out how to make mix format tolerate the patterns you’re building, especially things like this hanging indent on workflow declarations:

  field_transformations(as_cast: [:age],
    age_plus: on_success(&(&1+1), applied_to: [:age])
  ) |>

You’ll be glad or horrified to hear that I’ve removed the pipelining:

Well, partly that and partly because parsing was starting to too multi-pass-y.