abitdodgy

abitdodgy

Better design pattern for creating a polymorphic API

The other day I wrote a small lib to interact with AWS IAM (on top of ExAws). I wanted the internal API to have one Parser.parse/2 function that would handle any response. ExAws executes the request and calls this function with the response and the action name as the two arguments. For example:

parse.call(response, action)

Where response is {:ok, %{body: xml, status_code: status}} and action is the AWS IAM operation (“CreateUser”, “DeleteUser”, etc…).

I created a single module with a parse/2 action that dispatched to a function, which in turn called the right parser in the right module (pattern matching on the action name).

defmodule ExAws.Iam.Parser do
  alias ExAws.Iam.Parsers.{AccessKey, User}

  def parse({:ok, %{body: xml, status_code: status} = resp}, action) when status in 200..299 do
    parsed_body = dispatch(xml, action)
    {:ok, %{resp | body: parsed_body}}
  end
  def parse(resp, _), do: resp

  @user_actions ~w[ListUsers CreateUser]
  defp dispatch(xml, action) when action in @user_actions do
    User.parse(xml, action)
  end

  @access_key_actions ~w[ListAccessKeys GetAccessKeyLastUsed]
  defp dispatch(xml, action) when action in @access_key_actions do
    AccessKey.parse(xml, action)
  end
end

I then created individual parsing modules for each entity:

defmodule ExAws.Iam.Parsers.User do
  def parse(xml, "ListUsers") do
    # parse the xml
  end

  def parse(xml, "GetUser") do    
    # parse the xml
  end

  # etc...
end

defmodule ExAws.Iam.Parsers.AccessKey do
  # ...
end

I’m not a fan, though. The Parsers.AccessKey and Parsers.User should be private (which we don’t have in Elixir). I can’t import the functions because the names would clash. I also feel like I’m abusing pattern matching.

Protocols won’t work because the data types don’t change. What other patterns can I use to achieve the same API?

Most Liked

dimitarvp

dimitarvp

The best way in Elixir I could see is a mix of your original approach, and behaviours.

  1. Behaviour:
defmodule ExAws.Iam.Consumer do
  @callback consume_response(String.t, String.t) :: any # (xml, action)
end
  1. Implementers:
defmodule User do
  @behaviour ExAws.Iam.Consumer
  def consume_response(xml, action), do: #...
end
  1. Refactor dispatch to use router functions like so:
defp consumer(action) when action in @user_actions, do: User
defp consumer(action) when action in @access_key_actions, do: AccessKey

#...

defp dispatch(xml, action), do: consumer(action).consume_response(xml, action)

Code is not tested. But IMO this brings you close enough to some practical polymorphism while remaining terse enough.

(Another alternative would be to have a big module where you match on all possible combos but I don’t think it would be an improvement over your current approach – except maybe for having less source files.)

peerreynders

peerreynders

def SomeModule do

  @action_parsers %{
    "ListUsers" => User,
    "CreateUser" => User,
    "ListAccessKeys" => AccessKey,
    "GetAccessKeyLastUsed" => AccessKey
  }
 
  def parse({:ok, %{body: xml, status_code: status} = resp}, action) when status in 200..299 do
    with {:ok, mod} <- Map.fetch(@action_parsers, action) do
       parsed_body = apply(mod, :parse, [xml, action])
       {:ok, %{resp | body: parsed_body}}
    else
      _ ->
        # BOOM
    end
  end

end

But as it is you’ll probably be best off by pursuing the behaviour angle further if just to capture the implementation commonalities between your various parsers.

abitdodgy

abitdodgy

In the end, I stuck with my approach of having a Parser module pattern-match on the action name and delegate to the correct module for parsing. This way I only have to specify a parser once.

  def parse({:ok, %{body: xml, status_code: status} = resp}, action) when status in 200..299 do
    parsed_body = dispatch(xml, action)
    {:ok, %{resp | body: parsed_body}}
  end
  def parse(resp, _), do: resp

  @user_actions ~w[ListUsers GetUser]  # etc ...

  defp dispatch(xml, action) when action in @user_actions do
    User.parse(xml, action)
  end

I also realise that the API I wrote was contrived. A single generic operation/2 function would have allowed me to interact with the entire IAM API without having to write a function for each IAM action/endpoint, as I was doing. It can be used by the internal API for convenience functions too.

  def operation(action, params, opts \\ []) do
    {parser, params} = Keyword.pop(params, :parser, &Parser.parse/2)
    opts = Keyword.put_new(opts, :parser, parser)

    @shared_opts
    |> Keyword.merge(params)
    |> Keyword.put(:action, camelize(action))
    |> list_to_camelized_map()
    |> to_operation(opts)
  end

operation(:create_user, user_name: "foo")

# or for internal use

def create_user(username, opts \\ []) do
  operation([user_name: username] ++ opts)
end

Below is what I had until now. Those functions (create_user/2, list_users, etc…) are now relegated being to convenience functions only.

  def create_user(username, opts \\ []) do
    operation(:create_user, [user_name: username] ++ opts)
  end

  defp to_operation(params, opts) do
    %ExAws.Operation.Query{
      action: params["Action"],
      params: params,
      parser: Keyword.get(opts, :parser),
      path: params["Path"] || "/",
      service: :iam
    }
  end

So, what good are they? Well, maybe I can convert them to execute the operation on AWS instead of returning an ExAws op. For example:

  def create_user(username, opts \\ []) do
    :create_user
    |> operation([user_name: username] ++ opts)
    |> ExAws.request()
    |> to_user_struct()
  end
%User{
 arn: ...,
 create_date: ...,
 path: ...,
 user_name: ...,
 user_id: ...
}

Finally, it would be nice to have a parser for all those actions. But it’s hard work to write one for 60 or so actions. I wrote my first macro ever (be very afraid) to define DSL for parsers:

  defparser(:get_user,
    fields: [
      get_user_result: [
        ~x"//GetUserResult",
        user: [
          ~x"./User",
          :path,
          :user_name,
          :arn,
          :user_id,
          :create_date
        ]
      ],
      response_metadata: [
        ~x"//ResponseMetadata",
        :request_id
      ]
    ]
  )

The macro itself, below, is still not optimal. I would rather do away with passing the XML paths (~x"//GetUserResult") and handle that internally in the macro, but I have no way passing the type.

defmodule ExAws.Iam.TestMacro do
  import SweetXml, only: [sigil_x: 2]

  defmacro defparser(action, opts) do
    action_name = to_camel(action)

    fields =
      opts
      |> Keyword.get(:fields)
      |> Enum.map(fn field ->
        compile(field)
      end)

    quote do
      def parse(xml, unquote(action_name)) do
        SweetXml.xpath(xml, ~x"//#{unquote(xml_path(action_name))}", [
          {
            unquote(xml_node(action_name)),
            [~x"//#{unquote(xml_path(action_name))}" | unquote(fields)]
          }
        ])
      end
    end
  end

  defp xml_path(action), do: action <> "Response"
  defp xml_node(action), do: xml_path(action) |> to_snake()

  defp compile(field) when is_atom(field) do
    quote do
      {unquote(field), ~x"./#{unquote(to_camel(field))}/text()"s}
    end
  end

  defp compile({:sigil_x, _, _} = field), do: field

  defp compile({key, value}) do
    quote do
      {unquote(key), unquote(compile(value))}
    end
  end

  defp compile(list) when is_list(list) do
    Enum.map(list, fn field ->
      compile(field)
    end)
  end

  defp to_camel(atom), do: atom |> Atom.to_string() |> Macro.camelize()
  defp to_snake(string), do: string |> Macro.underscore() |> String.to_atom()
end

Here’s the code on a separate branch.

Thoughts?

Where Next?

Popular in Questions Top

jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New

Other popular topics Top

joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement