Best way to pass in external parameters to tests (e.g. from the console)?

What is the best way to pass in custom values to a test (e.g. when running the tests from the console)?

I am currently using environment variables to do this:

@debug_params [
  some_custom_value: System.get_env("SOME_CUSTOM_VALUE")

test "some test" do
  some_value = if @debug_params[:some_custom_value] do
    # Use a hardcoded value to save time or to test some specific thing
    # If no environment variable is set, get some generic value, or wait a long
    # time for an endpoint to return a result, or some other default behaviour


I feel like there is a better way to do this. I figured that I would be able to use test tags for this somehow, but those seem to be for static values only, and can’t be modified from the outside world.


EDIT: No XY Problem answers please, I’m just curious if this specific thing is possible. :slight_smile:

This may come off as a bit of a non-answer but based on your description in the comments of what you’re trying to accomplish, my solution to this is to have separate tests. One with mocks and one without and use a tag (like @external or something) to skip the slow ones (though in your situation you would just be running the quick one on its own).

As an idea to try and give you more what you are looking for, you could could pass in the slow operations as a fun which is conditionally defined in the setup:

setup do
  long_running_fun =
    if custom_value = System.get_env("SOME_CUSTOM_VALUE") do
      fn -> custom_value end

  %{long_running_fun: long_running_fun}

test "some test", %{long_running_fun: long_running_fun} do
  some_value = long_running_fun.()

EDIT: I’m realizing this only let’s you override one long running function :thinking:

Part of the reason I asked the question is to satisfy my burning desire to know if it’s possible. Whether or not it’s practical, it’s a question that I keep running into, just for the sake of having another tool in the toolbox.

The main reason I finally got around to asking the question is that I’m working on an implementation of a service that integrates with a workflow from a third-party API, and each of the values in the workflow is unique and arbitrary, and one of them (in the middle) expires very quickly. And each one in the chain depends on the result of the values returned from the previous endpoints.

I find that it’s easier to 1) edit and save a dotenv file, then 2) source and run the tests, while 3) having a default fallback which fetches the values the default (slow) way. All this as opposed to 1) hardcoding a bunch temporary values in the test itself, each one of which I would have to 2) remember to remove, plus 3) if I have to come back to debug one of the endpoints (they’re all part of one big workflow and the values all need to be fresh), I have this one reference I can use for my “debug params”.

Is there a better way? Probably. Do I still want to know if there’s a better way to do this specific thing? Most definitely.

Hey, wasn’t trying to XY you, just sharing how I would do it based on the info you shared because AFAIK you can’t pass values into a test other than through ENV vars. This has been discussed before but can’t find the thread.

You could still achieve what you were saying through my second suggestion there, just conditionally stub out each api call based on the presence of an ENV var from a dotenv file. It’s a bit of work but it’s all I got! Otherwise, sorry I couldn’t be of help, hopefully someone else can!

1 Like

No problem, I just had this feeling that there is some obvious answer under my nose… but I think I was actually just misunderstanding the purpose of test “tags”. Also, I’m just trying to make sure that my solution isn’t too far off the rails.

And I was more concerned with the XY problem thing in general, not specifically your answer. The first part of your post (“This may come off as a bit of a non-answer…”) did trigger the alarm bells in my head though. :smiley:

1 Like

Looking through the docs I don’t see anything. I don’t see anything like that ever making it in because as soon as you have logic in your tests they kind of cease to become tests in a way, ie, “What’s testing the logic in the tests?” In special cases like yours it might make sense so of course there are ways to make it happen, but providing a sanctioned API would normalize it as a practice which doesn’t seem prudent to me. Just my take!

I’m wondering if you’d be better off with snapshot testing tool. That way you push the concern of retaining some prev. value to the library and not something handrolled.