Testing Broadway best practices

Questions / Help
,
1

I’ve been working through a few Broadway SQS pipelines and I’ve made some good progress in getting all aspects of it tested. In previous projects, we didn’t put the Broadway pipeline into our application’s supervisor. Now that I’m working on an app that does put the Broadway pipeline into the supervisor, I’ve noticed that the SQS queue that’s being pointed at is queried during test runs. Is that correct? It seems like that’s the expected behavior, and it makes sense that things would fire off once the app starts. But it brings up a few other questions…

What are other people doing in their test environment – are you creating an SQS dedicated to testing? Or are you setting a config value for a fake URL so that SQS queries will never resolve? Or something else? I’ve gotten different feedback, so I’m wondering what the best practices are here?

So far, the strategy that makes the most sense to me is to provide my test config with a DummyProducer and/or fake SQS URL so that any test runs do not make actual calls to the outside world when I wasn’t expecting them. If needed (e.g. for integration testing), I can (re)start a new pipeline process and point it at a real SQS queue.

Thoughts?

2

On a project I’m working on, we conditionally start a few Broadway pipelines in Application. There is another supervisor that starts all of the Broadways, so that at least one test can start everything, feed data in one end of the application, and see outputs after the various pipelines crunch through data. ExUnit start_supervised and stop_supervised work well for this.

This also allows us to start smaller parts of the application in other tests.

Right now the data comes in one end via reading a file, so we don’t have the problem you’re describing. We’re about to add an SQS component, so I’m curious where you ended up with this. My plan is to either swap out a dummy producer in tests.