How to test whether an optional dependency is truly optional?

Qqwy · October 4, 2021, 11:02am

I am writing the library TypeCheck which has an optional dependency on StreamData.

From time to time, part of the code (wrongly) ends up calling StreamData directly, rather than (correctly) being wrapped in a Code.ensure_compiled?(StreamData) do ... end check.

In the code and test suite of the library itself this never results in any errors, as optional dependencies will be installed in a library which mentions them in its top level mix.deps.

However, when someone uses a version of the library with this problem, then they will get compiler-warnings unless they install the optional dependency themselves.

This means that twice now this mistake has only been seen after a version of the library was published.

To make sure that this does not happen again, I would like to create a regression test as part of the test suite of TypeCheck itself. But how can this be done?
Is there a way to wrap an extra ‘sample mix project’ in a test? Or is another approach possible?

LostKobrakai · October 4, 2021, 11:06am

There are ways, but they need quite a bit of boilerplate:

github.com

sasa1977/boundary/blob/master/test/support/test_project.ex#L5-L32


      
          def in_project(opts \\ [], fun) do
            loaded_apps_before = Enum.into(Application.loaded_applications(), MapSet.new(), fn {app, _, _} -> app end)
            %{name: name, file: file} = Mix.Project.pop()
            tmp_path = Path.absname("tmp/#{:erlang.unique_integer(~w/positive monotonic/a)}")
          
            app_name = "test_project_#{:erlang.unique_integer(~w/positive monotonic/a)}"
            app = String.to_atom(app_name)
            project = %{app: app, path: Path.join(tmp_path, app_name)}
          
            try do
              File.rm_rf(project.path)
          
              Mix.Task.clear()
              :ok = Mix.Tasks.New.run([project.path])
              reinitialize(project, opts)
          
              Mix.Project.in_project(app, project.path, [], fn _module -> fun.(project) end)
            after
              Mix.Project.push(name, file)

This file has been truncated. show original

Qqwy · October 4, 2021, 12:58pm

Thank you!

With help of that code example, I was able to tackle the task.
In the end, I made it a bit simpler as we won’t need multiple of these tests to be run side-by-side (/at the same time), so it was OK to just create the actual project files rather than creating it dynamically.

If someone is interested, the resulting test became:

github.com

Qqwy/elixir-type_check/blob/master/test/type_check_test.exs#L97-L105

    
      
          describe "StreamData is optional" do
            {stdout, 0} = System.cmd("mix", ["deps.compile", "--force"],
              cd: "./test/support/depending_project",
              stderr_to_stdout: true
            )
          
          
  refute stdout =~ "warning: StreamData"
            refute stdout =~ "warning: "
          end

LostKobrakai · October 4, 2021, 12:59pm

There`s a test missing within the describe though

Qqwy · October 4, 2021, 1:34pm

Thank you for noticing! This has been fixed .
(Interesting how assert works correctly even outside of a test, except that errors will not be highlighted as nicely.)

hauleth · October 4, 2021, 1:47pm

I think that there should be a way to do such magic via compiler tracing. This probably would be cleanest approach.

Qqwy · October 4, 2021, 2:31pm

How would this work?