Testing Only Code and Dead-Code Elimination

Chat / Discussions
,
1

Something I really appreciate from our ecosystem is that we trying to include “Test Support” files as part of the dependency packages.

Recently I have had a dilemma,

  1. Create two packages (a normal package, and a testing package) and publish them independently, for example: one_piece_commanded and one_piece_commanded_testing.
  2. Check for Mix.env() at compile-time and more the code
if Mix.env() == "test" do
  defmodule MyPackage.TestSupport do
    def hello do
      :hello_world
    end
  end
end

The first idea, is way too much work maintaining the packages. Incrementally becoming worst over time the more packages I have to maintain and whatnot.

The second idea, I feel dirty even suggesting it but I am not sure of any other solution that wouldn’t compile into production code that was only meant to be used in testing. Increasing the binary side (ideally, shouldn’t be the case).

Ideally, Elixir would help in the subject matter somehow, not sure what it would look like.

Any thoughts on the topic? How do you deal with the situation?

P.S: Kind of related to the trick of doing if Code.ensure_loaded?(Absinthe) do: .... to load extra code or not based on deps being installed.

2

I am not sure if I understand you right. But you can setup compile path per environment.

defmodule MyApp.Mixfile do
  use Mix.Project

  def project do
    [
      ...
      elixirc_paths: elixirc_paths(Mix.env()),
      ...
    ]
  end

  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(_), do: ["lib"]
...
3

Ah, is this a discussion on what step elixir ecosystem/community should do to help reduce Testing Package code into production / ensure these Testing Support Package code not compiled in production? in that case i have no real comment

Original post:

Is including Test Support package/module directly on the library without any custom processing a viable option?
`oban` library include testing module (https://hexdocs.pm/oban/Oban.Testing.html#content) directly without any flag and rely on developer to not misuse it for production code
4

oban is not the only library doing that. The same applies to Plug.Test, Phoenix.ConnTest or Phoenix.LiveViewTest.

If test helpers are not meant to be bundled in prod code I’d consider a compile time config like config :my_app, compile_test_helpers: false to disable them. Every other toggle will be a heuristic and therefore just waiting to not work for some user.

Generally I’d really consider hard, why including test helpers in prod code would be problematic though. What if the usecase is a test runner and prod actually runs tests?

2 Likes
5

I went for a little bit of an ugly route: https://github.com/straw-hat-team/beam-monorepo/blob/master/apps/one_piece_commanded/docs/explanations/enhancing-testing-environments.md

I rather do what Oban.Testing is doing, except that it is being compiled for production.

That is why I wish Elixir come up with some shared solution since we all doing whatever we think is correct trying to have a solution to the same problem :person_shrugging: not sure what is the right answer.

In that case, you would compile the test directory or whatever using elixirc_paths properly. In case of dependency packages, then I wouldn’t consider such code as “testing only”.

Sorry for the confusion @Marcus I can’t simply do that because the testing code lives in dependency packages, not my package.

6

Imo the issue here is the assumption that test helpers are code only used in tests. For a library I don’t think that’s an assumption, that should be made and therefore I actually support the existing approaches of having those modules be handled just like any other.

There are some valid concerns like file size bloat, but those are optimizations and also likely most relevant in really constraint environments, where you’d have the same concern not only about modules holding test helpers, but rather any module. So for the file size concern it makes sense to use a wholistic approach of dealing with it, rather than thinking about those handful of modules specific to test helpers.

1 Like
7

Fair enough,

Just for the sake of exercise to anticipate problems, what would be a good RFC to tackle the problem from Elixir itself?