Hi,
How do you load test your Phoenix LiveView applications? I haven’t found anything that works well out of the box. I want to simulate user behavior so I know where the application falls over.
Rationale
If the database server is on max load when I have 1k users - I want to know that. If I run out of connections because my application nodes scales up from 2 to 10 – I want to know that.
Solution
My current idea is to write a custom tool that I can package as a deployment to simulate real user traffic which contains some common functions. I will also deploy a version of my app that does no CSRF checks etc - I don’t think any “out of the box” tools will work.
I’ve assessed a few “out of the box tools” but they’re mostly enterprise or won’t easily run from a container inside of e.g. Kubernetes or lack other properties.
Any other options? What’s your experiences with load testing LiveView applications?
Thanks.
1 Like
Hi @Dangercoder,
First of all: If the availability is of importance for business then I advise to hire a company to do the testing. Their simulations will be more realistic and they have the proper infrastructure to perform the test so the results are not contaminated. (One of the most common mistakes is running the benchmark software on the same machine as the software being tested.)
Second: When you want to simulate user behavior, there are plenty (non-Elixir) software suits that feature it. All they need is a way to simulate network traffic, which is quite a basic feature. For LiveView it might be beneficial if they can also handle Javascript and WebSockets; then you can measure the whole stack.
Example load testing software: NeoLoad
Browser simulation: Selenium
Their simulations will be more realistic
I appreciate your perspective but I disagree.
It depends on the skill-level of the engineers in your team/org. If you have no experience in-house or a bunch of engineers that can’t learn new skills/make good decisions external help can indeed be a good strategy.
For LiveView it might be beneficial if they can also handle Javascript and WebSockets; then you can measure the whole stack.
I agree, I won’t be simulating the browser because it’s too heavy-weight. There are some more light-weight options (with a better DSL) than Selenium but they’re still heavy to run.
When I made the decision It was between https://k6.io/ or writing something myself.
I’ve tried other tools before. It’s just that they’re either “Enterprise” like the one you’ve just linked, or you’re forced to write the tests in javascript, java and maybe use XML to configure them.
Hi, related to tool choice, we used stressgrid in one of our projects.
It has support for websockets:
We checked in chrome dev tools for our liveview websocket messages, then played with stressgrid elixir scripts various web socket traffic scenarios.
1 Like
I have been building an in-house tool using :flame and :wallaby.
I use flame to spin up machines and spin up chromedriver sessions with wallaby.
I am not really load-testing, I am rather on a “find and vanquish high-traffic bugs” mission
That is, I have not done much work on the benchmarking side of things because I was specifically trying to recreate a product-specific bug that happens when there are lots of concurrent connected users getting rapid live updates. I was able to recreate the bug on our staging site and I got a good feel for how international latency affects some of our flows, thanks to Fly+Flame.
It is not a very sophisticated setup but it’s nice that everything is still in Elixir and it’s nice to be able to write familiar wallaby user flows. They are easy to read and modify.
3 Likes
