Approach for numerous edge cases, HTTP GET, exvcr

I’m debating between using exvcr stubs or just generating lots of custom cassettes to exercise edge cases of an http get request against an external endpoint. I’m curious about the forum’s thoughts on the pros and cons of each.

Was just reading this article, might be helpful for you: Testing External Web Requests in Elixir? Roll Your Own Mock Server | by Sophie DeBenedetto | Flatiron Labs | Medium

1 Like

Interesting but too heavyweight for my tastes.

That article actually re-convinced me to use mock and not to roll my own mock server. :003:

I have already used all of them: exvcr, mock, mox and bypass. For testing code that relies on underlying HTTP client libraries I liked mock the most with bypass a close second.

mox is amazing for a lot of testing but it doesn’t help me mock the underlying HTTP client library. I’ll use mox when I write higher level code that tests working with my API client itself e.g. I will make it return {:error, :too_many_retries} and see if the higher-level code reacts properly to that.

exvcr I tried to use just today but it was too opaque for me and even though the docs / GitHub page are rich on details I was unable to achieve a simple scenario – namely have two use_cassette blocks one inside the other (as I was testing pagination and wanted two different requests mocked, and yes I used its query parameter distinction parameter). Plus it wasn’t obvious how to use recorded (custom) cassettes. I simply gave up after 10 minutes and reached for mock which took the same amount of time and worked right away with zero surprises (NOTE: I only used mock twice before and the last time was more than a year ago).


Not to incline future readers too much in any particular direction as libraries evolve and get better (though some don’t) but so far mock was the best for testing code that relies on a lower level library (by mocking the said lower-level library seamlessly) and mox for testing everything in your own code where you have full control.

Folks should really give patch v0.12.0 — Documentation try(also see the cheatsheet)
The ergonomics and guarantees you get with Patch are far better than Mock or Mox in many cases. There is a comparison if you want to know how each mocking library behaves under different scenarios

5 Likes

I am liking exvcr a lot now. For error cases, I started with stubs, but I am going to switch to a combination of stubs and recorded cassettes with shorten timeouts in the code under test.


Once you get in direct call, delete, and record workflow, it gets pretty efficient doing happy path tests with exvcr. The only thing that can be tricky is remembering to not use random data generation when creating input data, so that your requests match fixed data in cassettes. (I wish there was a way to specify only generating new random data when exvcr is recording new cassettes.)

Workflow:

  • Edit code under test
  • Comment out use_cassette calls
  • Adjust tests and confirm updates work with live calls
  • Delete cassette, enable use_cassette again
  • Run tests triggering recording of new cassettes for deleted cassettes

Well I still have no clue how to pre-record cassettes so there’s that. :smiley:

1 Like

You can use one cassette and record multiple requests in that cassette.


For custom cassettes, you just specify the location to the custom cassette. It’s usually easier to record a cassette and then modify it into a custom cassette.

Maybe I can’t talk properly but again, I seriously don’t know what to do to record a cassette EXPLICITLY – not implicitly by running a test and allowing ExVCR to store it, not that – so I can reuse it later with the custom cassettes path.

Can you give me a working example?

The workflow I am interested in:

  1. Run $mysterious_command to record an HTTP response
  2. Store file under custom/cassettes/path
  3. Specify the above path in my tests

I don’t know how to do #1.

You select an adapter from the exvcr tutorial. Then add the appropriate setup to an exunit test for your adapter. Then wrap your call in “use_casette” with your desired cassette name. Then run the test once. After the test has run, you will find the recorded cassette in your cassettes directory.

When I am at my computer tomorrow, I will try to detail out every step above with code snippets. We use hackney/httpoison at work, in case you are using a different adapter which may have issues that I am unaware of.


For the explicit case, there is an iex helper, but I haven’t used it.

OK so it’s how I thought – you have to run the test once for that. And then just point ExVCR to the place where you moved the file afterwards (the so-called custom cassettes path). Alright.

Haven’t looked up the IEx helper but you might be right that it could do explicit cassette recording. Hm, I might give it a try.

First time I hear about patch, looks very good actually, thanks!

1 Like

(also, I recommend matching on req body and params, because it is easy to overlook needing to re-record cassettes if you don’t use precise matching)

test config:

config :exvcr,
  vcr_cassette_library_dir: "test/fixture/vcr_cassettes",
  custom_cassette_library_dir: "test/fixture/custom_cassettes",
  filter_sensitive_data: [
    [pattern: "<PASSWORD>.+</PASSWORD>", placeholder: "PASSWORD_PLACEHOLDER"]
  ],
  filter_request_headers: ["Authorization"]

mix.exs

      {:exvcr, "~> 0.13.3", [only: :test]}

test

  use ExVCR.Mock, adapter: ExVCR.Adapter.Hackney
 ...

  setup_all do
    HTTPoison.start()
    :ok
  end

    test "..." do
      use_cassette "..." do
           # do the api calls
      end
  ...

After reading this thread, I finally decided to give ex_vcr a try. So far I was using Bypass to test interactions with external APIs.

I’m using Tesla with Finch as my HTTP client.

So I have old test modules using Bypass and I’m writing a new test module that uses ex_vcr. What I noticed is that, if I run the new test in async mode (which ideally I would like to do), the test interferes with the other async tests that use Bypass. Basically, some tests fail because they don’t get the expected response.

I haven’t dug too much into it yet, but my initial assumption is that this is due to ex_vcr setting up global mocks for Finch. Could this be the issue?

Just checking if any of you has ever encountered this problem before or can give me a hint. Thanks!

2 Likes

Looks like exvcr uses meck and meck notes the modules have to replaced globally - GitHub - eproxus/meck: A mocking library for Erlang - so I don’t think you can use bypass and exvcr in the same test suite while using async.

1 Like

Thanks. Yes that’s what I figured too. meck - unlike Mox - apparently doesn’t support mocks local to a single process.

My workaround is to run the exvcr test synchronously - I have only one for now. As soon as I have more than one exvcr test, I will run them asynchronously in their own separate mix test execution. Best I can do for now I guess.

1 Like

Not just apparently. meck replaces the modules bytecode and reloads the module, which can only be done globally to the whole VM.

2 Likes