ElixirConf 2022 - Michael Lubas - Classifying Bot IP Addresses in Phoenix

ElixirConf: ElixirConf 2022 - Michael Lubas - Classifying Bot IP Addresses in Phoenix

Comments welcome! View the #elixirconf tag for more ElixirConf talks!

7 Likes

Very interesting!

During my time at Papa we just banned India and China IPs as a whole and that solved the vast majority of our problems with bots.

If you’re a startup I highly recommend going that route (at least initially) since it’s cheap and easy. Just make sure you don’t actually do business with the users in those countries. Use common sense

3 Likes

What about Bloom Filters?

Many IP address lookup approaches employ Bloom filters to obtain a high-speed search performance. Especially, the search performance of trie-based algorithms can be significantly improved by adding Bloom filters, because Bloom filters can determine whether a node exists in a trie without accessing the trie. The false positive rate of a Bloom filter must be reduced to enhance the lookup performance. One important characteristic of a trie is that all the ancestors of a node are also stored. The proposed IP lookup algorithm utilizes this characteristic in reducing the false positive rate of a Bloom filter without increasing the Bloom filter size.

Bloom Filter is memory efficient than a Hash Map with the same performance. The only thing to note is that this is a probabilistic data structure so for a small number of cases, it may give wrong results (which can be limited).

The applications of Bloom Filter are:

  • Weak password detection
  • Internet Cache Protocol
  • Safe browsing in Google Chrome
  • Wallet synchronization in Bitcoin
  • Hash based IP Traceback
  • Cyber security like virus scanning

Bloom Filters by Example



Some Elixir Libs: Blex && Bloomex


P.S. For blocking some more bots, we can use ua_inspector & device-detector:

3 Likes

Thank you! There’s also a blog post, Classifying Data Center IP Addresses in Phoenix Web Applications with Radix Trees, to go with the talk.

4 Likes