I’m trying to build a low-cost website serving a catalogue of items, with standard options of viewing/filtering said items. It is expected that a catalogue itself will be queried often, but rarely updated. It is also a requirement that the website is resilient against DDoS attacks and other script-kiddie-level threats.
Given the requirements (and, let’s be honest, my shallow level of Erlang/Elixir ecosystem knowledge), I was wondering if I can use Agent module to serve as an in-memory cache:
Load whole db table into agent’s state as a list (or map of id → item);
Whenever a user requests a page, get the table from the Agent’s state, and filter by whichever items are needed;
In case of an update - reload table, replace the Agent’s state.
I read about immutability in Elixir and assumed that this would be an improvement compared to making calls to the DB each time, since copied/filtered lists and maps will still point to the same immutable list.
Then I tested both approaches locally by holding down the “F5” button and saw that memory usage goes up either way.
By now, I have read about “share-nothing concurrency” and realized that copying the whole table with the response between processes is probably not the best way to go. And while I can change Agent module to accept a filter and return only requested items (and spawn a bunch of them for good measure) - I wonder if the whole setup even makes sense in practical terms?
Maybe there’s already a module/library that does what I need, or maybe I should stop overthinking, put my server behind DDoS protection by something like Cloudflare, and call it a day?
I would be grateful to hear a hint from someone who has experience in actually deploying something to a live audience.
Making your app fast is good, and things like cachex will help if you have cachable items.
For true DDOS protection you really want to shield from that at a higher layer like the load balancer or CDN. If you have highly cacheable items a CDN can be quite helpful too, in particular if you have highly cacheable items consider just having the CDN front that too.
As Ben mentioned you will generally want multi-layered protection to be effective against DDoS attacks. So enabling your firewall and installing something like sshguard or fail2ban, using rate-limiting on the server level so requests are blocked before they touch your app (with something like HAProxy) and then rate limiting in your app and caching if possible.
The most efficient way to block attacks will be at the firewall level, followed by the proxy/load balancer (HAProxy), followed by your app (most expensive in terms of resources required - so block attacks before they hit your app whenever possible!)
You’ll then need to monitor the server and if you are getting attacked employ measures to mitigate them… but all that can be a lot so as you mentioned yourself many will opt to use a WAF like Cloudfare.
Whether you use Cloudfare or not you’ll want to ensure you harden your server, but again, it is a fairly big topic. The firewall with something like sshguard or fail2ban, changing SSH ports, removing root-login with password, etc, is a good place to start. Alternatively (and if this is commercial project) you could just get a managed server* and they should take care of a lot of it, or if this is a personal/hobby project you could start with a cloud provider like Digital Ocean because they have lots of set-up and administration guides that should help.
*When I started out we first had managed servers for a few years, then after I felt comfortable we moved to un-managed. So don’t feel like you have to dive straight into the deep end.
