Reducing repetition of strings via macro?!

[woolfred]

Hey everyone ,

we (me and my team) are currently struggling with the idea of reducing the repetition of the same strings in several places within our application. I guess in other languages we would rely on enums.

tl;dr

In the end we introduced a macro to include module attributes with the strings we needed to match on. But aren’t fully convinced, if this is the right approach. Therefore we would like to hear what you all think about this approach

First I will try to give a context and then try to show some examples of the problem, which gave us the feeling that we would want to introduce a “single point of truth”. At the end I show you “the solution” Here we go…

Context & Problem

Our backend is dealing with payment data it receives from an app and handles the communication with an external payment service.
As some requests take a bit longer and we wanted to deal with bad reception, the basic architecture is asynchronous and relies on Oban.

So basically: When the App makes a payment request we create an %PaymentProcess{status: "initialized", payment_provider: "paypal", ....}, put it in the database, enque an Oban job to deal with the external service interaction and return a 200. The app is then listening to state changes of the payment.
Based on the provider the job handles the interaction with the external service differently and in the end depending on the result it will update the PaymentProcess like this:

• %PaymentProcess{status: "paid", payment_provider: "paypal", ....} or
• %PaymentProcess{status: "declined", payment_provider: "paypal", error_code: 123}

After that, it sends the app a notification to fetch the new state of the payment process (the app also pulls on a regular base as a fallback).

When the app now fetches the current PaymentProcess, we fetch it from the DB and return a 200 if everything went smoothly (status: "paid") or 422 when the payment was declined. Part of the error response is also a text the app should display, which is specific to the used payment_provider. In some cases it is also a mix of the used payment_provider, the status and a error_code.
(We decided to let the backend provide the error text, as it is way easier to change texts, add new cases etc. with a new backend deployment, instead of releasing an app update through the app stores)

Examples of repetition

This flow and setup led to code where we need to pattern match on the payment_provider in several places.
E.g. does Request look like this:

defp payment_authorization(%{
    payment_authorization_provider: "paypal",
    payment_authorization_attributes: %{"fetching" => some, "provider" => specific_data}
  }) do
	...
end
 
defp payment_authorization(%{
    payment_authorization_provider: "google_pay",
    payment_authorization_attributes: %{"other" => specifc_attributes}
  }) do
  ... 
end 

And the part where we build a failure (lets call it AppFailure), we do something like this:

defp authorization_failed(payment_provider) do
  case payment_provider do
    "paypal" ->
      {:unprocessable_entity,
       "Please do this and that to fix the problem."}

    "google_pay" ->
      {:unprocessable_entity,
       "Open the Google Pay App to update your card details."}
  end
end

Especially in this module the repetition in the case statements matching over and over again on "paypal" ,"google_pay" etc. got pretty obvious.
The first idea was using module attributes at the top:

@paypal "paypal"
@google_pay "google_pay"
...

This helped to reduce repetition and error-proneness, while providing auto-completion with VSCode.

But we still have the same strings in our Request module and basically wanted a way to share the module attributes. The first idea (we already knew wouldn’t work) looked like this:

defmodule PaymentProviders do
  @paypal "paypal"
  @google_pay "google_pay"
	
	def paypal, do: @paypal
	def google_pay, do: @google_pay
end

But you can’t use functions for pattern matching.

Our solution

Now what we came up with, but maybe shouldn’t do: Macros

defmodule PaymentProvider do
  defmacro __using__(_opts) do
    quote 
      @google_pay "google_pay"
      @paypal "paypal"
    end
  end
end

With this we could improve our AppFailure like this:

defmodule AppFailure do
  use PaymentProvider
	
  ...
	
  defp authorization_failed(payment_provider) do
    case payment_provider do
      @paypal ->
	    {:unprocessable_entity,
	     "Please do this and that to fix the problem."}

      @google_pay ->
        {:unprocessable_entity,
          "Open the Google Pay App to update your card details."}
    end
  end
end

And also use it to pattern match in the function signature:

defmodule Request do
  use PaymentProvider
	
  defp payment_authorization(%{
      payment_authorization_provider: @paypal,
      payment_authorization_attributes: %{"fetching" => some, "provider" => specific_data}
    }) do
        ...
  end
end

The refactoring the code with the macro felt better than the initial version with the same strings throughout the code. But we would really like to hear, what you all think about this approach, as we are really hesitant as soon as we try to achieve something with macros and start wondering where we took a wrong turn.

Looking forward to read your thoughts and thank you for reading anyway!

[LostKobrakai]

Usually one would use atoms in places you’re using the strings. For db level support there’s Ecto.Enum.

[woolfred]

Oh, cool. Good point.
The first time we thought about Enum-like things, was before Ecto 3.5 which introduced Ecto.Enum. At that time there was ecto_enum and we somehow had in mind, that we will need to deal with Postgres Enums, which we didn’t wanted to do at that time.

Than let me rephrase my question: Is reducing repetition of atoms via a macro a valid idea?

(We are also going to explore if we could leverage Typespecs better to help us avoid typos)

[bartblast]

It feels a little too over-engineered in my (subjective) opinion.

[IvanR]

If the idea is to make the IDE to help you to select a provider from the list, then it may be useful to add a scope to attribute like @payment_provider_paypal, @payment_provider_google_pay etc.

Then in VSCode when you start typing @payment you get the scoped autocompletion list of providers

[woolfred]

How did you get access to our source code?!
This is exactly how we named the attributes, but I wanted to keep the example code short. I guess not the best idea, as it is better to have such a scope/prefix.

Edit:
I can’t edit my initial post, but for others the version with improved naming might be useful:

defmodule PaymentProvider do
  defmacro __using__(_opts) do
    quote 
      @payment_provider_google_pay "google_pay"
      @payment_provider_paypal "paypal"
    end
  end
end

[dimitarvp]

I would go much further and just make a macro that injects most of the boilerplate, especially the failed clauses which seem pretty easy to generate.

You can also make your own DSL (domain-specific language) with macros and have stuff like:

defmodule Request do
  use PaymentAuthorization, by: @paypal
end

…which would generate your above function clause:

defp payment_authorization(%{
    payment_authorization_provider: "paypal",
    payment_authorization_attributes: %{"fetching" => some, "provider" => specific_data}
  }) do
	...
end

I’ve been in projects where such repetition exists and it’s mega-annoying and very error-prone. DSLs in this case can save your sanity (and eliminate bugs by subtle typos).

[opsb]

If each of the payment providers has to handle the same scenarios perhaps a behaviour would make more sense?

defmodule PaymentProvider do
  @callback payment_authorization(..)
  @callback authorization_failed(...)
end

[LostKobrakai]

Or a protocol given data might also be provider specific.

[eksperimental]

@woolfred

defp payment_authorization(%{
    payment_authorization_provider: "paypal",
    payment_authorization_attributes: %{"fetching" => some, "provider" => specific_data}
  }) do
	...
end
 
defp payment_authorization(%{
    payment_authorization_provider: "google_pay",
    payment_authorization_attributes: %{"other" => specifc_attributes}
  }) do
  ... 
end 

I would reduce the verbosity of your code a bit.

defp payment_authorization(%{
    provider: :paypal,
    attributes: %{fetching: "some", provider: "specific_data"}
  }) do
	...
end
 
defp payment_authorization(%{
    provider: :google_play,
    attributes: %{other: "specifc_attributes"}
  }) do
  ... 
end 

---

UPDATE:
This could be simplified with pattern function matching

@woolfred

defmodule AppFailure do
  use PaymentProvider
  
  ...
  
  defp authorization_failed(payment_provider) do
    case payment_provider do
      @paypal ->
      {:unprocessable_entity,
       "Please do this and that to fix the problem."}

      @google_pay ->
        {:unprocessable_entity,
          "Open the Google Pay App to update your card details."}
    end
  end
end

like this:

defmodule AppFailure do
  use PaymentProvider
  
  ...
  
  defp authorization_failed(:paypal),
    do: {:unprocessable_entity, "Please do this and that to fix the problem."}

  defp authorization_failed(:google_pay),
    do: {:unprocessable_entity, "Open the Google Pay App to update your card details."}
end

[eksperimental]

@woolfred

defmodule PaymentProvider do
  defmacro __using__(_opts) do
    quote 
      @payment_provider_google_pay "google_pay"
      @payment_provider_paypal "paypal"
    end
  end
end

Sorry, but why don’t you just refer to them by atoms. I think you are overlyengineering stuff.

One pattern that I commonly use is in your main module, when you manually add your providers.

defmodule YourApp do
  # Edit only this line when a new payment provider is added
  @payment_providers [:google_pay, :paypal]

  defguard is_payment_provider(term) when term in @payment_providers

  def payment_providers(), do: @payment_providers
end

defmodule YourApp.PaymentProviders do
  import YourApp, only [is_payment_provider: 1]

  # Now apply this guard to your function, so you don't have to worry about passing the an invalid provider

  defp payment_authorization(%{provider: provider}) when is_payment_provider(provider),
    do: beautiful_stuff()
end

[lud]

This is what we use:

defmodule MyApp.Enums.EnumCompiler do
  @moduledoc false

  defmacro defenum(mod_alias, property) do
    providing_code = Macro.to_string(property)

    quote location: :keep, bind_quoted: binding() do
      module = Module.concat(MyApp.Enums, mod_alias)

      values =
        case property do
          %Xema.Schema{enum: enum} -> enum
          %Xema.Schema{const: const} -> [const]
          other -> raise "invalid values: #{inspect(other)}, given as #{providing_code}"
        end

      IO.puts("defining enum #{inspect(module)} as #{inspect(values)}")

      defmodule module do
        Enum.each(values, fn
          val when is_binary(val) and val != "" ->
            :ok

          value ->
            raise "Invalid value for enum #{inspect(__MODULE__)}: #{inspect(value)}, given as #{providing_code}"
        end)

        [first | _] = values
        first_fun = first |> String.downcase() |> String.to_atom()

        @moduledoc """
        Generated enum with the following values:

        #{MyApp.Enums.EnumCompiler.format_values_markdown(values)}

        A function named after the lowercase version of the value is defined for
        each enum value, returning the enum key as a `t:String.t()`.

            IO.inspect #{__MODULE__ |> Module.split() |> List.last()}.#{first_fun}(), label: "value"
            # value: "#{first}"

        Macros are also defined to provide values access through named calls in
        guards and matches. They use the same names as the functions, with a 
        `__` prefix to denote compile-time work. You must require the module in 
        order to be able to use those macros.

            require #{inspect(__MODULE__)}, as: #{__MODULE__ |> Module.split() |> List.last()}
            #{__MODULE__ |> Module.split() |> List.last()}.__#{first_fun}() = "#{first}"
            # "#{first}"
        """

        @doc """
        Returns all values as a list.
        """
        def _values, do: unquote(values)

        @doc """
        A macro that injects the full value list.
        """
        @doc guard: true
        defmacro __values do
          values = unquote(values)

          quote do
            unquote(values)
          end
        end

        values
        |> Enum.each(fn value ->
          fun = value |> String.downcase() |> String.to_atom()

          @doc "Returns the enum value: `#{inspect(value)}`"
          @doc section: :enum_value
          def unquote(fun)(), do: unquote(value)
        end)

        values
        |> Enum.each(fn value ->
          fun = value |> String.downcase()

          @doc "Macro injecting AST for the enum value: `#{inspect(value)}`"
          @doc section: :enum_value_macro
          defmacro unquote(:"__#{fun}")(), do: unquote(value)
        end)
      end
    end
  end

  def format_values_markdown(values) do
    values
    |> Enum.map(&[?`, inspect(&1), ?`])
    |> Enum.intersperse(", ")
    |> to_string
  end
end

Given a value like "HELLO", the generated module will have the hello/0 function and the __hello/0 macro (the macro can be used in pattern matching).

We use Xema. If you do not, you will have to accept a bare list of strings in the following case:

      values =
        case property do
          %Xema.Schema{enum: enum} -> enum
          %Xema.Schema{const: const} -> [const]
          other -> raise "invalid values: #{inspect(other)}, given as #{providing_code}"
        end

[dimitarvp]

Extremely high quality demonstration of how macros reduce boilerplate. Bravo!