linusdm

linusdm

Property-based testing slow when hitting the database

I’m trying to test an Ecto query that joins a few tables, and that a lot of where conditions that have to satisfy before selecting rows. I was thinking about how property based testing would be a nice way to generate a lot of permutations to make sure the filtering works as expected. The setup involves creating some entities, and calling context functions that also hit the database. Generating the data works, but I see the test takes quite some time. I could conclude that property based testing is just not suited when hitting the database, because it’s too slow to generate all the required state. But I’m not sure if that’s really true. Is this still a viable option somehow (for example, by turning down the :max_runs setting for these kinds of tests)?

I’m new to property based testing, so I appreciate all feedback.

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

We have a few property based tests that hit the Database, and they are indeed slow. My general tips would be:

  1. use async false and a shared sandbox mode so that you can insert common records ahead of all individual property interactions.

  2. benchmark individual iterations. Maybe your code is actually just slower than it has to be and you can make it faster.

  3. accept the slowness and set a time limit. Have this time limit be configurable and periodically test it with larger time limits if you are changing related code and want to be extra thorough.

  4. consider setting up a Postgres database optimized for testing. This is a larger topic but a lot of postgreses data integrity features are unimportant for running tests.

dimitarvp

dimitarvp

PosgtreSQL and SQLite differ too much for those tests to be objective.

LostKobrakai

LostKobrakai

Depends on how you set it up. The sandbox is checked out and in where you or some code owned by you calls the APIs to do so. You can do both: Sandbox per generated value(s) or sandbox for the whole test.

I’d also add a completely different angle here. It sounds like you have a bunch of business logic, which at the end should result in an appropriate selection of records. You generally cannot really test Ecto.Query structs, as their content is mostly private API. So most often we default to executing the query as well to test the query.

Another option would be separating the “business logic to build up query details” from “turning query details into Ecto.Querys”. You can use a custom struct to aggregate all the information you need to build the final query. That struct can be fully (unit) testable with non-private fields. Also implement the Ecto.Queryable protocol and you can use that custom struct wherever Ecto.Queryable is accepted. The protocol implementation can then be a simpler (not as complex to test) implementation of turning the data in your struct into an Ecto.Query.

Where Next?

Popular in Questions Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

Other popular topics Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement