Property Testing a CRUD API

Questions / Help
,
1

I am sketching out how to test a CRUD API.

My idea is to create a simplified model / FSM, generate different states and test for properties based on the model.

E.g. assuming everything’s successful,

  1. GET /users
  2. POST /create/id/9
  3. DELETE /delete/id/9
  4. GET /users

users from 1. and 4. should be the same

Then do the a similar test when

  1. GET /users
  2. POST /create/id/8
  3. GET /users

3.'s users should have one more entry etc.

or when

  1. GET /users
  2. POST /create/id/11
  3. DELETE /delete/id/54 //:error
  4. GET /users

Again 4’s users should have one more entry.

basically, setting properties for as many cases I can, and then testing with random test-cases.

Problem is, I cannot find much material online, I am sure it is pretty standard esp. as CRUD APIs are pretty standard… (?)

2

Problem is, I cannot find much material online

I think https://propertesting.com/ is pretty good.

2 Likes
3

I really like what I have read also - just wondering whether there is a canonical “this is how you property test a CRUD API” - these are the properties to look out for, no matter the language etc.

4

For a server-side rendered CRUD website I’d concentrate on making StreamData generators of controller parameters – omit mandatory fields, send fields with invalid data in them, try and update or delete non-existing IDs, send fields that your app isn’t at all expecting. StreamData can be really alien at start but when you experiment with it for a weekend you can do some seriously powerful data generators with it!

1 Like
5

I ended up following the book’s recommendations (https://propertesting.com/) e.g. create an ‘oracle’ state machine.

Basically, you create random states by interacting with your API and keep a record in your oracle.

You compare both oracle state and results from your API, and make sure they are always the same.

You’d be surprised the bugs you catch… I really need to try and bundle this into a library, am sure you could test lots of apps pretty easily with one library…

1 Like
6

Can you explain more about how you create the oracle state machine.

7

From What is a Property (online version of the book, prior to having been edited):

Finally, there’s a rare but great type of modeling that may be available, called “the oracle”. An oracle is a reference implementation, possibly coming from a different language or software package, and can therefore act as a pre-written model. The only thing required of the programmer to test the system is to compare their implementation with the oracle and ensure the same results are given.

4 Likes