I don’t really know how to answer to this in a reasonable way, but why would modules with 10,000,000 functions be the appropriate tool for the job?

Particularly if you need special build machines just to accommodate potentially compiling the thing and keep running into limitations of the runtime environment you’re using, I would be led to believe that you need a different tool than the one that keeps breaking and failing to do what you want it to do.

Is there a different approach that you could take, or maybe you found a problem that’s just not a good fit for the VM? This sounds like a nightmare.

This line here clouds what you’re trying to prove. This first generates a list of 10 million atoms. This will blow out the atom table long before you try to actually make any functions.

iex(1)> defmodule PocManyClauses do
...(1)>   @list (1..10000000) |> Enum.map(fn n -> :"fn_#{n}" end)
...(1)> end
no more index entries in atom_tab (max=1048576)

Crash dump is being written to: erl_crash.dump...done

Regardless, it still probably isn’t possible to generate a function with 10 million clauses even if you could get past the atom thing. In Absinthe we generate a LOT of functions and compile time starts becoming an issue orders of magnitude below 10 million.

Is this true? The O() behaviour of pattern matching depends on the complexity and kind of pattern. For integer literals and atom literals (which boil down to integers) it’s my understanding that the beam can produce O(1) lookup tables. You’re trying to generate range check clauses though and I’m pretty sure that the theoretical best for that is O(log(n)).

More to the point, the O here doesn’t really matter cause you have a fixed size lookup space. What you need to determine is how fast a given lookup is, and how many lookups the system can handle concurrently. If that meets your needs then the O doesn’t really matter. If it’s too slow then yeah, looking for something with a better O can be a good guide, but I’d make sure your problem space actually permits O(1).