aquamat

aquamat OP

Hello!

For my first post to the forum that has helped me a great deal as I have become involved with this language and ecosystem.

I have some code that allows me to test if an email address looks like it is from a free/consumer email provider, e.g. gmail.com or hotmail.com. Following what I believe to be an idiomatic approach I wanted to use meta coding to take input from a list of nearly 5000 domains for such providers and make a module with a function head of is_free() for each… e.g.:

def is_free("gmail.com"), do: true
def is_free("hotmail.com"), do: true
...
def is_free(_), do: false

The code that I ended up with and which works is as follows:

defmodule FreemailCode do
  @moduledoc """
  helper to provide __using__ that generates an is_free(domain) function 
  for each domain listed in the input file as a being a free/consumer 
  email provider such as gmail.com, hotmail.com etc.

  The source file is free-domains-2.csv downloaded from hubspot at: 
  https://knowledge.hubspot.com/forms/what-domains-are-blocked-when-using-the-forms-email-domains-to-block-feature
  """
  defmacro __using__(_) do
    domains = File.read!("/path/to/free-domains-2.csv")
      |> String.split("\r\n")

    IO.puts("Freemail loading domains")
    for domain <- domains do
      if domain == "" do
        quote do
          def is_free(_), do: false
        end
      else
        quote do
          def is_free("#{unquote(domain)}"), do: true
        end
      end
    end

    ## the following simply doesn't seem to work or get added in the 
    ## right place, always matching all domains if we do it here. Hence the if 
    ## statement above.
    # quote do
    #   def is_free(_something), do: false
    # end
  end
end

defmodule Freemail do
  @moduledoc """
  `is_free_email("[[user]@]domain")` will return true if given either a
  complete email address or just the domain of a free/consumer email 
  provider.

  Underlying are the module functions is_free(<email domain>) used to determine
  whether an email address domain is from a free/consumer mail provider or not.
  These are generated by the macro in the above FreemailCode module.
  """
  use FreemailCode

  def is_free_email(email_addr) do
    case email_addr |> String.split("@") do
      [_user, domain] -> is_free(domain)
      [domain]        -> is_free(domain)
    end
  end
end

And that’s great, but my first attempt was to avoid the need to have __using__, after-all, I wouldn’t be injecting these functions into many modules. I just wanted to build one module. So my first attempt was:

defmodule FreemailTwo do
  domains = File.read!("/path/to/free-domains-2.csv")
    |> String.split("\r\n")
    |> Enum.filter(fn x -> x != "" end)

  IO.puts("FreemailTwo loading domains")
  for domain <- domains do
    quote do
      def is_free("#{unquote(domain)}"), do: true
    end
  end
  # quote do
  #   def is_free(_), do: false
  # end
end

When compiling, this does read the domains and iterate over them - IO.puts statements will prove this. However, if I try to import FreemailTwo and call an is_free/1:

** (UndefinedFunctionError) function FreemailTwo.is_free/1 is undefined or private                                            

I can require this into a module and it doesn’t work, import it… nada. I believe this is valid, and compilation doesn’t complain, but the generated functions simply don’t appear to be there. What is it that I am missing!?

The second mystery, as touched upon in the code comments above, is this: I thought I would generate all the headers for actual domains and then, in the next code section, generate a catch-all function to return false. You can see this commented out. However, doing so also failed and I found that this function would always match. Instead I detect end of file (in this case by testing for the blank line at the end - a very ugly approach - and inserting the catch-all when I do so. This is the final working solution. It’s great - does what I want and works well. But… why was there a problem coding it in the first way!?

I guess the final question is broader: I believe that this is the idiomatic Elixir way to solve this problem of testing to see if a value is in one of many. Alternatives would be to read in or generate code to create a large list and test if candidate values are in that list. I believe this is probably slower and the general wisdom seems to be that exploiting Elixir/Erlang pattern matching is the more powerful and faster approach. The most inelegant (to my eyes) approach would be to have a DB table and query that each time (or Redis cache…). Am I going about this in the best way?

I hope this is of interest to someone. I am happy to accept that I have a working solution and move on, but would be fascinated to learn as to why my other approaches failed.

Regards
Matthew

First 2 of 2 Posts Switch mode

al2o3cr

al2o3cr

A macro needs to return an AST - writing code like:

for x <- xs do
  quote do
    ...
  end
end

works when it’s the LAST expression in the function because the result is a list of AST fragments (the output of quote).

Adding another piece of code to the macro causes the return value of for to be discarded, the same way that this:

def foo do
  4 + 5
  :ok
end

discards the return value of +.

You’ll need to instead combine the results of the for loop and the quote block for the default case, into a list etc.


This code has the opposite problem - the quote isn’t needed when writing in a module definition. Using it results in the same thing as writing a tuple at the top-level:

defmodule Foo do
  {:def, [context: Elixir, import: Kernel], [{:is_foo, [context: Elixir], ["a"]}, [do: true]]}
end

which compiles but doesn’t do anything.

For the in-a-module-definition case, you can use unquote directly:

defmodule Foo do
  for x <- ~w(a b c) do
    def is_foo(unquote(x)), do: true
  end
  def is_foo(_), do: false
end

One additional note: check out the @external_resource module attribute, for making sure FreemailTwo always gets recompiled when the CSV changes.

aquamat

aquamat OP

:man_facepalming: - so obvious when you point it out - thank you very much for your thorough reply!

— All posts loaded —

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
roeland
Kia ora, We have been using elixir-google-api to connect to Google Drive. However, with the updates to Tesla due to CVEs this is now bro...
New
rahultumpala
Hello, I have an Elixir backend that implements a custom protocol over TCP. I want to load test the backend and assess the performance o...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New

We're in Beta

About us Mission Statement