ReqCassette v0.2.0 Update

Quick update on ReqCassette! v0.2.0 is now available with some nice
improvements:

What’s New:

• with_cassette/3 function API for cleaner test code
• Human-readable cassette filenames (github_user.json instead of
  a1b2c3d4.json)
• Multiple interactions per cassette file
• Four recording modes: :replay, :record, :record_missing, :bypass
• Sensitive data filtering (headers, regex-based redaction)
• Configurable request matching
• Pretty-printed JSON with native JSON objects (40% smaller)

Example:

import ReqCassette

test "API integration" do
  with_cassette "github_user", [mode: :replay], fn plug ->
    response = Req.get!("https://api.github.com/users/wojtekmach", plug: plug)
    assert response.status == 200
  end
end

First run records, subsequent runs replay instantly (no network, async-safe).

Links:

• Hex: req_cassette | Hex
• Docs: ReqCassette v0.6.1 — Documentation
• Migration guide: https://hexdocs.pm/req_cassette/migration_v0.1_to_v0.2.html

Breaking changes from v0.1 - see migration guide.

Nice work! Thanks for putting this together!!!

ReqCassette v0.3.0 – Safer Recording & Better Filtering

What’s New

Safer :record mode (breaking change)

• Fixed dangerous behavior where multi-request tests would lose interactions
• Simplified API: 4 recording modes → 3
• Migration: just replace :record_missing with :record

Better filtering

• New filter_request and filter_response options
• Comprehensive filtering guide for protecting API keys

Quick Example

with_cassette "api_test",
  [filter_request_headers: ["authorization"]],
  fn plug ->
    Req.get!("https://api.example.com", plug: plug)
  end

I am using ReqCassette to test a wrapper I’ve written for the Google Gemini API.I am not very happy with the wrappers I’ve found, so I’ve written my own - actually most of the code and the module docs are automatically generated from the JSON API spec.

One thing I want to be able to test is polling the same URL with the same query params to see if the batch has completed. This is clearly a stateful interaction. Even though this is a bit of a dumb thing to test, it’s actually really useful to be able to replay this interaction so that I can test/prototype new ways of displaying progress (progress bar, spinner, whatever).

Apparently, ReqCassette will not store the sequence of requests, but instead will save the request/response pair once, and will continue to return that result forever (I got this idea from inspecting the cassettes, not from inspecting the code).

Is this interpretation right? Is there any way of using ReqCassette for propertly stateful interactions?

ReqCassette v0.4.0 - Templating & Better Debugging

Quick update on ReqCassette since 0.3.1

What’s New

Templating - One cassette can now handle multiple requests with different dynamic values. Great for APIs that return unique IDs, timestamps, or when testing with varying inputs.

with_cassette "product_lookup",
  [template: [patterns: [sku: ~r/\d{4}-\d{4}/]]],
  fn plug ->
    # First run records with SKU 1234-5678
    # Future runs work with ANY SKU - cassette substitutes automatically
    Req.get!("https://api.example.com/products/9999-0000", plug: plug)
  end

LLM Presets - Built-in patterns for Anthropic and OpenAI APIs. No more cassette mismatches from msg_* or chatcmpl-* IDs:

template: [preset: :anthropic]  # handles msg_*, toolu_*, req_*
template: [preset: :openai]     # handles chatcmpl-*, call_*
template: [preset: :llm]        # all providers

Mismatch Diagnostics - When replay fails, you now get detailed info showing exactly which fields didn’t match:

🔴 :uri NO match
🟢 :method match

🔬 :uri details
Record 1:
stored: "https://api.example.com/v1/users"
value:  "https://api.example.com/v2/users"

Debug Tools - New mix req_cassette.inspect task to inspect cassette contents, and debug: true option for template troubleshooting.

Links

• Templating Guide
• LLM Testing Guide
• GitHub

Thanks for giving it a try @tmbb!

I have been able to test a multi-turn conversation with LLM APIs, where there is a chain of messages one after another that runs, and the cassette can store the sequence.

It’s not completely clear to me about what you mean by stateful interaction, but maybe the feature that I just released about templates could be useful here. Where we can parametrize the recorded cassettes, so we can pattern match against that and update the response.

Again, the use cases I have been using are related to LLMs. But in my case, I have some information on the input request that changes over time, but that’s predictable, so I can create a template and have it respond accordingly.

Let me know if that goes in the direction that helps you in your use case. I am definitely open to exploring ways to make the library more powerful.

In pseudocode, I am doing the following:

def wait(batch, poll_interval) do
  # Note the following URL, query params, etc are all deterministic;
  # The response may be equal the first couple times, but eventually
  # it will change when some responses in the batch are processed
  new_batch = update_state(batch, Req.get("google_api.com/#{batch.name}"))
  if new_batch.done do
    new_batch
  else
    # Wait a bit more
    :timer.sleep(poll_interval)
    wait(new_batch, poll_interval)
  end
end

When I call this with a cassette (I have my custom logic to get the plug into the request from the Process dictionary, assume that part works), it loops forever because apparently it always sees the same response in which the batch is not done.

ReqCassette v0.5.0 - Sequential Matching for Stateful API Testing

What’s New in v0.5.0

Sequential Matching

By default, ReqCassette uses first-match: the same request always returns the
same response. This works for most tests, but not for polling scenarios
where identical requests should return different responses over time.

Now you can enable sequential matching:

# Polling API that returns different states
with_cassette "job_polling", [sequential: true], fn plug ->
  Req.get!("/job/status", plug: plug)  # -> {"status": "pending"}
  Req.get!("/job/status", plug: plug)  # -> {"status": "running"}
  Req.get!("/job/status", plug: plug)  # -> {"status": "completed"}
end

Requests match interactions in order (request 1 → interaction 0, request 2 →
interaction 1, etc.).

Cross-Process Support

If you’re making HTTP requests from spawned processes (Task.async,
GenServer, etc.) with sequential matching, use a shared session:

session = ReqCassette.start_shared_session()
try do
  with_cassette "parallel_test", [session: session, sequential: true], fn plug ->
    tasks = for i <- 1..3 do
      Task.async(fn -> Req.get!("/item/#{i}", plug: plug) end)
    end
    Task.await_many(tasks)
  end
after
  ReqCassette.end_shared_session(session)
end

Templates Auto-Enable Sequential

If you’re using templating (template: [...]), sequential matching is now
automatically enabled - no need to add sequential: true.

Backward Compatibility

Default behavior is unchanged. Existing tests work without modification.

Links

• Hex
• GitHub
• Migration Guide (v0.4 → v0.5)

Feedback welcome!

Ah! I got it and I just released 0.5.0 adds exactly what I think you need - sequential matching.

Enable it with sequential: true:

with_cassette "batch_polling", [sequential: true], fn plug ->
  # Each call to the same URL now returns the NEXT recorded response
  wait(batch, poll_interval, plug)
end

When recording, each request hits the real API and stores a new interaction.
When replaying, requests match interactions in order (request 1 →
interaction 0, request 2 → interaction 1, etc.) instead of first-match.

Your cassette will look something like:

{
  "interactions": [
    {"request": {...}, "response": {"status": "RUNNING", "progress": 0.2}},
    {"request": {...}, "response": {"status": "RUNNING", "progress": 0.5}},
    {"request": {...}, "response": {"status": "RUNNING", "progress": 0.8}},
    {"request": {...}, "response": {"status": "SUCCEEDED", "done": true}}
  ]
}

On replay, each poll gets the next response in sequence until it sees
done: true.

One caveat: If your polling code runs in spawned processes (Task.async,
GenServer), you’ll need a shared session:

session = ReqCassette.start_shared_session()
try do
  with_cassette "batch_polling", [session: session, sequential: true], fn plug ->
    wait(batch, poll_interval, plug)
  end
after
  ReqCassette.end_shared_session(session)
end

But if everything runs in a single process (which sounds like your case), just
sequential: true should work.

Let know how it goes!

Thank you this is exactly what I was looking for and it works really well!