How to ensure sequential test execution?

what I want to achieve is:
execute a test to registry a record using a given payload
execute another test with the same payload to throw a proposital error (the db doesnt accept duplicates)
but even cleaning the db before tests, sometimes the first request fails, sometimes it doesn’t, basically it seems like they occur randomly, sometimes all the tests pass, sometimes a few etc. I dont have idea on how to assure this flow

defmodule TestesPay.MyRouterTest do
  use ExUnit.Case, async: false
  use Plug.Test
  alias MyApp.Repo
  require Logger
  @create_qr_url "https://sandbox.asaas.com/api/v3/pix/qrCodes/static"
  @pix_key "ab09d852-14bd-43ac-91eb-d38b717e6aea"
  @test_price 22
  @payload "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF"

  defp clean_contracts(_) do
  
    MyApp.Repo.delete_all(MyApp.Schemas.Contract)
    Logger.info("Contratos limpos com sucesso!")
    
    :ok
  end

  setup_all do
    access_token = System.get_env("ACCESS_TOKEN")
    Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
    MyApp.Repo.delete_all(MyApp.Schemas.Contract)
    :ok
    {:ok, %{access_token: access_token}}
  end

  describe "database cleaning verification" do
    test "ensure contracts table is empty before tests" do

    all_records = MyApp.Repo.all(MyApp.Schemas.Contract)
    Logger.info("Registros antes da limpeza: #{inspect(all_records)}")
    MyApp.Repo.delete_all(MyApp.Schemas.Contract)

    contracts_count = MyApp.Repo.aggregate(MyApp.Schemas.Contract, :count, :id)
    Logger.info("Número de registros na tabela contracts: #{contracts_count}")

      assert contracts_count == 0, "A tabela contracts não está vazia!"
    end
  end

  describe "create contract teste1" do
    Logger.info("[TESTE1] INIT")
    Logger.info("[TESTE1] PAYLOAD: #{@payload}")
    test "POST /api/create_contract" do

      body = Jason.encode!(%{
        "chain" => "eth",
        "coin" => "eth",
        "ref" => "my_referral",
        "payload" => @payload
      })

      conn =
        conn(:post, "/api/create_contract", body)
        |> put_req_header("content-type", "application/json")
        |> TestesPay.MyRouter.call(%{})

        actual_response = Jason.decode!(conn.resp_body)
        Logger.info("[TESTE1] Body: #{inspect(actual_response)}")
        all_records = MyApp.Repo.all(MyApp.Schemas.Contract)
        Logger.info("Registros depois do [TESTE1]: #{inspect(all_records)}")
      assert conn.status == 203
    end
  end

  describe "retorna erro pois registro ja existe" do
    Logger.info("[TESTE2] INIT")
    Logger.info("[TESTE2] PAYLOAD: #{@payload}")

    all_records = MyApp.Repo.all(MyApp.Schemas.Contract)
    Logger.info("[TESTE2] Registros no banco: #{inspect(all_records)}")
    test "POST /api/create_contract com payload já existente retorna erro" do
      
      initial_body = Jason.encode!(%{
        "chain" => "eth",
        "coin" => "eth",
        "ref" => "my_referral",
        "payload" => @payload
      })
      conn =
      conn(:post, "/api/create_contract", initial_body)
      |> put_req_header("content-type", "application/json")
      |> TestesPay.MyRouter.call(%{})

      actual_response = Jason.decode!(conn.resp_body)
      Logger.info("[TESTE2] body: #{inspect(actual_response)}")
      all_records = MyApp.Repo.all(MyApp.Schemas.Contract)
      Logger.info("Registros depois do [TESTE2]: #{inspect(all_records)}")
    assert conn.status == 400

    end
  end

  describe "assure database state after payload test" do
    test "database" do
      Logger.info("database estado final")
      all_records = MyApp.Repo.all(MyApp.Schemas.Contract)
      contracts_count = MyApp.Repo.aggregate(MyApp.Schemas.Contract, :count, :id)
      Logger.info("registro final: #{inspect(contracts_count)}")
      Logger.info("numero final de contratos: #{inspect(contracts_count)}")
      assert contracts_count == 1
    end
    
  end

end
15:32:31.867 [info] [TESTE1] INIT

15:32:31.867 [info] [TESTE1] PAYLOAD: 00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF

15:32:31.867 [info] [TESTE2] INIT

15:32:31.867 [info] [TESTE2] PAYLOAD: 00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF

15:32:32.025 [info] [TESTE2] Registros no banco: []

15:32:32.073 [info] database estado final

15:32:32.084 [info] registros finais: []

15:32:32.092 [info] numero final de contratos: 0


  1) test assure database state after payload test database (TestesPay.MyRouterTest)
     test/integration_test.exs:87
     Assertion with == failed
     code:  assert contracts_count == 1
     left:  0
     right: 1
     stacktrace:
       test/integration_test.exs:93: (test)

"/api/create_contract"

15:32:33.345 [info] [TESTE2] body: %{"chain" => "eth", "coin" => "eth", "payload" => "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF", "private" => "3jRrp8orhspK4yngX5vGv1JH3pSauZuwQPBgDpFcf8DwM4GtmdNpcjAB1XqGpwsVhp4yDZGttJaLrmfa2GJNBW8b", "public" => "7wyExYgwuRv8BS8qTZDi7XnayoFHZf8t2AUFXHUBVUDh", "ref" => "my_referral", "ref_fee" => 0.01, "transaction_charged" => 26.26, "transaction_value" => 26.0}

15:32:33.360 [info] Registros depois do [TESTE2]: [%MyApp.Schemas.Contract{__meta__: #Ecto.Schema.Metadata<:loaded, "contracts">, id: 258, chain: "eth", coin: "eth", transaction_value: Decimal.new("26.000"), transaction_charged: Decimal.new("26.260"), ref: "my_referral", ref_fee: Decimal.new("0.01"), payload: "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF", public: "7wyExYgwuRv8BS8qTZDi7XnayoFHZf8t2AUFXHUBVUDh", private: "3jRrp8orhspK4yngX5vGv1JH3pSauZuwQPBgDpFcf8DwM4GtmdNpcjAB1XqGpwsVhp4yDZGttJaLrmfa2GJNBW8b", status: "pending"}]


  2) test retorna erro pois registro ja existe POST /api/create_contract com payload já existente retorna erro (TestesPay.MyRouterTest)
     test/integration_test.exs:64
     Assertion with == failed
     code:  assert conn.status == 400
     left:  203
     right: 400
     stacktrace:
       test/integration_test.exs:81: (test)


15:32:33.361 [info] Registros antes da limpeza: [%MyApp.Schemas.Contract{__meta__: #Ecto.Schema.Metadata<:loaded, "contracts">, id: 258, chain: "eth", coin: "eth", transaction_value: Decimal.new("26.000"), transaction_charged: Decimal.new("26.260"), ref: "my_referral", ref_fee: Decimal.new("0.01"), payload: "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF", public: "7wyExYgwuRv8BS8qTZDi7XnayoFHZf8t2AUFXHUBVUDh", private: "3jRrp8orhspK4yngX5vGv1JH3pSauZuwQPBgDpFcf8DwM4GtmdNpcjAB1XqGpwsVhp4yDZGttJaLrmfa2GJNBW8b", status: "pending"}]

15:32:33.366 [info] Número de registros na tabela contracts: 0
."/api/create_contract"

15:32:33.767 [info] [TESTE1] Body: %{"chain" => "eth", "coin" => "eth", "payload" => "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF", "private" => "31owKnxfK2LuLjdTPDpJJFpnxXyED8d7LiZM7hnXyqSpQ8s5zBT5NoouGc8mBP5cJ6WA58TqUuJXTXqkG4qT9VEa", "public" => "DJZQv35VqtrKfHQ9MPx7Dy36uCG3VJqXkRLYsJHK7X3g", "ref" => "my_referral", "ref_fee" => 0.01, "transaction_charged" => 26.26, "transaction_value" => 26.0}

15:32:33.768 [info] Registros depois do [TESTE1]: [%MyApp.Schemas.Contract{__meta__: #Ecto.Schema.Metadata<:loaded, "contracts">, id: 259, chain: "eth", coin: "eth", transaction_value: Decimal.new("26.000"), transaction_charged: Decimal.new("26.260"), ref: "my_referral", ref_fee: Decimal.new("0.01"), payload: "00020126580014br.gov.bcb.pix0136ab09d852-14bd-43ac-91eb-d38b717e6aea520400005303986540526.005802BR5914Pedro Henrique6005Macae62290525PEDROHEN00000000554994ASA6304AEBF", public: "DJZQv35VqtrKfHQ9MPx7Dy36uCG3VJqXkRLYsJHK7X3g", private: "31owKnxfK2LuLjdTPDpJJFpnxXyED8d7LiZM7hnXyqSpQ8s5zBT5NoouGc8mBP5cJ6WA58TqUuJXTXqkG4qT9VEa", status: "pending"}]
.
Finished in 1.9 seconds (0.00s async, 1.9s sync)
4 tests, 2 failures

Randomized with seed 796590

reading the logs I have the feeling the tests run in random order

All tests inside a singular test case file are still ran sequentially so that’s not your problem.

IMO check out the Sandbox example on how to achieve better isolation. I have the exact same setup in my latest apps and never had trouble with leftover data between tests.

the problem was that ecto db sandbox automatically clean all registrys from db after each test! but thanks :slight_smile: