Is this a good use case for elixir?

Questions / Help
, ,
1

Hi,

I need to run an js script 500 times at once and collect the outputs from each execution. Is this something that can easily be done with elixir, at least in comparison to other languages?

I’m also interested (if the above is true) if It is possible to have a separate process track the execution of each of the scripts and update the results on a drab page like in the async tasks example?

Just in case it is important the script that I plan to run is a automated e2e test using selenium.

Thanks in advance

2

Will you run this 500 js in the same browser? I assume you want to use the browser, as you’re refering to Drab.

Yes, exactly like in the async example - run the separate task for each process and collect results as it finishes.

1 Like
3

Will you run this 500 js in the same browser? I assume you want to use the browser, as you’re refering to Drab.

This is how I was imagining the workflow:

  1. User opens the drab page with the different tests listed on their comp.
  2. User presses ‘Run Tests’ button
  3. A script runs on the server where the drab page is hosted, firing of 500 seperate headless chrome instances.
  4. The users gets a list of reports from each test instance as the tests finish.
4

This part might be a little bit tricky, if you want to do it in parallel :wink:

1 Like
5

Any suggestions?

6

I have never measured it, but I believe RAM would be a bottleneck for 500 chromes. You may want to run it on the multiple machines, and yes - this is a case for Elixir.

Also, be prepared to have a small number of random bugs occasionally. I am using chromedriver with hound in Drab’s tests, and from time to time it just fails with the different browser errors. Re-running the same test fixes the issue.

1 Like
7

There is a service for this, https://www.browserless.io, but they are charging something like 5$ per browser, so probably it would be cheaper to provide your own infrastructure.

8

I’ll try and work that out somehow. Maybe I’ll just have to rent a beefy server.

I’m quite familiar with those :stuck_out_tongue:

9

Use Task.async_stream/3 and toy with the limit until you get a maximum that fits the hardware you’re on. Once you get that functioning your test will run (which is valuable in itself). Then if that number is unacceptable and runtime is too long, get a bigger box and see what it will give you. Selenium is slow and uses a lot of CPU as well. So its best to get something working as is first before springing to another box so you’ll have something to compare against when you do decide to increase the hardware (not to mention being able to justify or not justify the cost of doing so).

2 Likes
10

Also, consider starting with number of cores. Don’t try 100 or something crazy like that at first because it will never complete. Start with something with few resource constraints and inefficient and move up to efficient and max resources because once you go past that its just likely to waste your time.

2 Likes