Running all tests in test folder

You can do:

mix test test/some/path

and it will run all tests in that path. Is that not working for you?

No because the file are executed in random order. The option seed: 0 disables the random order only inside the file.

We are running integration tests for a system for which we have written a simulator to communicate with. Therefore we have dependencies like StartUp test has to run first.
I guess at the end of day we will have hundreds of tests where some of them will have strict dependencies. It will be very messy to write all tests in one single file.
Also we would like to sort the tests into categories to be able to activate or deactivate a special type of test.
I know you can do that also with @tag :skipped but this is also not very elegant in our case.

Do you have some example code that we could look at? Talking about it using just words is not as good as using words+code. I really think it’ll help our understanding of this issue.

I don’t get your point.
It has nothing to do with my code because the test are running if I put them in one file with the option seed: 0 and async: false. I “just” want to be able to run them in a given order.

file a_test.exs has test a1, a2, a3
file b_test.exs has test b1, b2, b3
file c_test.exs has test c1, c2, c3

The test should run for example in the following order.

a1, a2, a3, c1, c2, c3, b1, b2, b3 

You’re asking us to give you the solution for Y but you may need the X solution. I don’t think I have enough info on your problem to know for certain. This is why I asked for code.

If something seems hard to do then it can be because how you’re attempting it isn’t the right way.

We are not testing elixir code.
We have written a simulator in elixir which is communicating via serial interface with a device. The code of this divece is c++ -code.
The simulator simulates a motor control.
Now I would like to test this device via the serial interface.
If that device sends a message an event is triggered and we are checking if this was the right event.
It can not be compared to elixir unit tests.
Till now we have just some dummy tests. First we have to solve the above-mentioned problem.

If you are not writing tests for your elixir code, why then do you try to use mix test for it instead of an elixir script that you would call serial port tester´or something like that?

Agreeing with @bitboxer here, it sounds like you are using the wrong tool for the job. mix test is for testing against Elixir code. As a rule it is very bad practice to have order dependent tests because such tests are extremely brittle as they rely on quite a lot of invisible implicit state.

I fully agree that tests should run independently. But in our cases we have start up test which have to run first.
Nevertheless thank you for the discussion.

I’ve to learn the ropes of scripting in elixir.

You can also write a regular Elixir application, where the application start function orchestrates the test execution. And if helpful, you can use the supervision structure to achieve more reliability.