Locus: Geolocation and ASN lookup of IP addresses

geolocation

#1

Hi forum,

I’m pleased to announce the release of locus 1.0.0, an Erlang/OTP library that allows you to pinpoint the country, city or ASN of IPv4 and IPv6 addresses, using MaxMind’s GeoLite2 databases.

The MaxMind databases you choose are downloaded on-demand, cached on the filesystem and updated automatically.

The databases are loaded into memory (mostly) as-is; reference counted binaries are shared with the application callers using ETS tables, and the original binary search tree is used to lookup addresses. The data for each entry is decoded on the fly upon successful lookups.


#2

Hi forum,

Locus 1.1.1 was released today.

Added:

  • OTP 18, 19.0, 19.1 and 19.2 support (version 1.0.x required 19.3 or higher)
  • ability of consulting database metadata, source and version through :get_info
  • ability of subscribing database loader events
  • ability of specifying connect, download start and idle download timeouts
  • ability of turning off caching

Documentation was moved to HexDocs and test coverage was substantially increased.


#3

Great work! That’s a hugely useful dataset to have easy access too.

Without having looked heavily at what you’ve done, what is the size of the database?


#4

Thanks! :slight_smile:

Without having looked heavily at what you’ve done, what is the size of the database?

It depends on the database you pick. MaxMind provides three databases for free:

  • Country database (~3 MiB when uncompressed)
  • City database (~60 MiB when uncompressed; conceptually similar to country database but with more precision)
  • ASN database (~6 MiB when uncompressed)

As the databases are, for the most part, used in their raw form, the expected memory consumption should correspond roughly one-to-one.

They’re not, however, included in the package; they’re downloaded on demand, and caching is leveraged to minimize likelihood of unavailability upon subsequent application boots. My main motivation for this was to avoid some of the shortcomings of egeoip, where database updates were committed to the repository, which grew ever larger.

I do plan to support explicit loading from local filesystem, with or without support for updating from network after having that as a starting point. But because the semantics of this, and all its edge cases, aren’t still quite clear in my head, I’ve been delaying it, as I want to keep things as simple as possible for now.


#5

Whooo awesome there!


#6

Hi forum,

Locus 1.3.0 was released today.

Added:

  • ability of loading databases from local file system
  • type spec of database entries

Fixed:

  • wrong handling of timezones on cached tarballs
  • wrong handling of daylight saving time on conditional HTTP requests

The timezone / DST fixes mentioned above were also backported to earlier versions and tagged under:

  • 1.0.1
  • 1.1.4
  • 1.2.2

Links:


#7

Hello forum,

Locus 1.6.0 was released today.

Added:

  • new API method for validating loaded databases (locus:analyze/1)
  • new command line tool supporting database validation

Changed:

  • safety of database HTTPS downloads was substantially improved by now rejecting expired certificates, mismatched hostnames, self-signed certificates or unknown certificate authorities
  • test coverage using MaxMind’s test data was greatly extended
  • database decoder was thoroughly optimized
  • documentation was mildly improved

Fixed:

  • misguided rejection of UTF-8 strings with non-printable (but valid) codepoints
  • unnecessarily strict refusal to load 2.x database formats succeeding 2.0
  • infinite recursion in maliciously crafted databases due to circular paths

Links: