Should not be hard as long as only one test is run at a time in regards to the filesystem side.

What exactly were you having trouble with, what specific code did you get stuck on?

Well, actually that’s more about the right way to do it.
Should I create a fake/temporary files tree (eg on /tmp) and play with it, or should I just run the GenServer on the app itself and touch a file (feels wrong to do it during a test).
My other question is how can I assert the GenServer did its job ? (in verbose mode i have some output, but I’m not sure that’s the best way).
The server send itself a cast when it detects a file change, which run Mix.Tasks.Compile.Elixir

Rather than hardcoding the “Mix.Tasks.Compile.Elixir” function, you could just inject it in the init function on the GenServer. Then for testing you can just inject a function that sends the parameters back to the test for comparison.

defmodule Demo do
  def init([notify]) do
    {:ok, notify}
  end

  def terminate(reason, notify),
    do: notify.(:terminate, reason, self())

  def handle_call(request, {pid, _}, notify) do
    notify.(:call, request, pid)
    {:reply, :ok, notify}
  end

  def handle_cast(request, notify) do
    notify.(:cast, request, self())
    {:noreply, notify}
  end

  def handle_info(msg, notify) do
    notify.(:info, msg, self())
    {:noreply, notify}
  end

  def default_fun(t,r,p),
    do: IO.puts "#{t} #{inspect r} #{inspect p}"

end

run = fn demo, wait ->
  Process.sleep wait
  GenServer.call demo, {:hello,"there"}
  Process.sleep wait
  GenServer.cast demo, "For You Information"
  Process.sleep wait
  Kernel.send demo, 42
  Process.sleep wait
  GenServer.stop demo
  Process.sleep wait
end

fetch = fn wait ->
  receive do
    msg ->
      IO.inspect msg
  after
    wait ->
      :ok
  end
end

fun = &Demo.default_fun/3
{:ok, demo} = GenServer.start_link Demo, [fun]
run.(demo, 200)

dest = self()
test_fun = fn (t, r, p) -> Kernel.send dest, {t,r,p} end
{:ok, test} = GenServer.start_link Demo, [test_fun]
run.(test, 10)
fetch.(10)
fetch.(10)
fetch.(10)
fetch.(10)
{:message_queue_len, count} = Process.info self(), :message_queue_len
IO.puts "Message queue empty: #{count == 0}"

$ elixir demo.exs
call {:hello, "there"} #PID<0.80.0>
cast "For You Information" #PID<0.86.0>
info 42 #PID<0.86.0>
terminate :normal #PID<0.86.0>
{:call, {:hello, "there"}, #PID<0.80.0>}
{:cast, "For You Information", #PID<0.88.0>}
{:info, 42, #PID<0.88.0>}
{:terminate, :normal, #PID<0.88.0>}
Message queue empty: true
$

An excerpt how that might be used in ExUnit:

defmodule PhoneSupTest do
  use ExUnit.Case

  # ... setup and stuff ...

  # function that returns the function to be injected
  def notify_callback do
    pid = self()
    fn (reason, other, ms) ->
      Kernel.send pid, {reason, other, ms} # send tuple back to test process
      :ok
    end
  end


  test "PhoneSup - demo" do
    notify = notify_callback() # 1. Create function to inject
    ms1 = 1
    ms2 = 2

    {:ok, p1} = PhoneSup.attach_phone ms1, notify # 2. inject function
    {:ok, p2} = PhoneSup.attach_phone ms2, notify # 2. inject function

    assert (PhoneFsm.action {:outbound, ms1}, p2) == :ok # 3. test something
    assert_receive {:outbound, ^ms1, ^ms2}               # 4. verify contents
    assert_receive {:inbound, ^p2, ^ms1}                 #    of generated
                                                         #    messages

    # ... more assertions, etc ...

  end
end