kccarter

kccarter

Looking for some advice on how a seasoned Elixir developer might go about implementing the creation of a map/struct that is dependent on several other maps.

Let’s say we’re modelling something like a Freight Shipping Order, aka “Bill of Lading”. Generally speaking, these have a lot of fields. In our case, we need to make several API calls to just get the right information to assemble one.

Consider this pseudo code, typos are to be ignored.

def fetch_freight_order(id) do 
  # These all just return a Map, constructed from a JSON response.
  shipping_customer = call_api(...)
  receiving_customer = call_api(...)
  line_items = call_api(...)
  freight_company = call_api(...)
  
 # ... and maybe 4-5 more calls to an api we don't control ...

  # To assemble a freight_order, we need all of the parts above.
  freight_order = new_freight_order(
    shipping_customer,
    receiving_customer,
    line_items,
    freight_company,
   # ... 4-5 more arguments
  )
end

Option 1
Would Elixir developers pass each argument or would they be more included to pass a keyword list, map, or use some other pattern?

# Easy to get confused at the call site which order these are all in.
def new_freight_order(
  shipping_customer, 
  receiving_customer,
  line_items,
  freight_company,
   # ... 4-5 more arguments
)

%{
  # Populate shipping_customer fields
  shipping_name: shipping_customer["Name"],
  shipping_street: shipping_customer["Street"],

  # Populate receiving_customer fields
  receiving_name: receiving_customer["Name"],
  receiving_street: receiving_customer["Street"],

  # ... maybe 50+ more fields ... 
}

end

Option 2
If we pass a single map, then we can pattern match in the function’s signature but it gets rather unwieldy pretty quickly and the call site also gets a lot busier:

def new_freight_order(%{
  shipping_customer: shipping_customer,
  receiving_customer: receiving_customer,
  line_items: line_items,
  # a bunch more matches...
}) do 

# Merge into a single freight_order
end

Option 3
Or perhaps use single pattern matches:

def new_freight_order(freight_order, %{shipping_customer: shipping_customer}) do 
  # Merge in just the shipping fields
end

def new_freight_order(freight_order, %{receiving_customer: receiving_customer}) do 
  # Merge in just the receiving fields
end

Option 4
Alternatively, would anyone create separate functions for each section?

def populate_shipping_customer(freight_order, shipping_customer) do 
end

def populate_receiving_customer(freight_order, receiving_customer) do 
end

I appreciate that style is subjective, but as new Elixir devs, we’re curious what patterns might be preferred over others. This use-case comes up quite a bit for us. A lot of the data we’re working with within our Elixir app (and showing to the user), requires a considerable number of discrete API calls just to gather the required data before we can assemble it into some sort of local representation.

Thanks in advance.

Showing Posts 1 to 10

D4no0

D4no0

The rule of thumb is to not have a lot of arguments for a function (for me personally 4 is the upper limit). The most dangerous thing is as you mentioned, should you mess up the order and you can be in trouble.

In this case going with a map argument is the way to go, however I would advise against Option 2, as it is generally discouraged to have a match where you literally match on all the fields.

What you can do instead is to use the . map operator, for example freight_order.shipping_customer, in this way you will have an exception if that key is missing.

If you want to ensure that the structure of the map is respected at compile-time please use Typespecs.

sodapopcan

sodapopcan

A couple of recommendations:

Because APIs are volatile, perhaps you want to give your users a message about it and NOT LetItCrash™, you should have your API calls return error tuples and use them with with.

with {:ok, shipping_customer} <- call_api(...),
     {:ok, receiving_customer} <- call_api(...),
     {:ok, line_items} <- call_api(...),
     {:ok, freight_company} <- call_api(...) do
  # Put them all together
else
  {:error, api_error} ->
    # Handle error
end

In terms of responses, I would have a mapper to rename keys and then you could even use Ecto to validate the final result (via an embedded schema).

def map_keys(shipping_customer, receiving_customer, line_items, freight_company) do
  %{
    shipping_name: shipping_customer["Name"],
    shipping_street: shipping_customer["Street"],
    receiving_name: receiving_customer["Name"],
    receiving_street: receiving_customer["Street"],
    # ...
  }
end

Here I actually wouldn’t use . in this case (even though it’s generally a good suggestion) to keep it simple and leave the validation for the final Schema.

Then you could make an Ecto schema for the final structure and validate it:

defmodule Thing do
  use Ecto.Schema
  import Ecto.Changeset

  embedded_schema do
    field :shipping_name, :string
    field, :shipping_street: :string
    # ...
  end

  def build(attrs) do
    %__MODULE__{}
    |> change()
    |> validate_shipping()
    |> validate_receiving()
    |> # ...
    |> apply_action(:update)
  end
end

Then the whole with would look like:

with {:ok, shipping_customer} <- call_api(...),
     {:ok, receiving_customer} <- call_api(...),
     {:ok, line_items} <- call_api(...),
     {:ok, freight_company} <- call_api(...),
     mapped_attrs = map_keys(shipping_customer, receiving_customer, line_items, freight_company),
     {:ok,  thing} <- Thing.build(mapped_attrs) do
  # Do stuff with `thing`!
else
  {:error, %Ecto.Changeset{} = changeset} ->
    # Handle data error

  {:error, %SomeHttpLibError{} = api_error} ->
    # Handle network error
end

In terms of number of params I definitely like to keep them low, but sometimes rules need to be broken! Not sure how many API calls you need to make, though, so as always, YMMV.

derek-zhou

derek-zhou

I do not have a hard limit for how many arguments for a function call; though I do agree that too many is a code smell. I use keyword list for options only. I’d also argue against making up map to reduce argument count. Use structs, or even Ecto. Yes, Ecto schemas can be used without a database.

D4no0

D4no0

Using ecto for validation of incoming data was a good advice, but using ecto for arguments is certainly an overkill, let’s not transform this topic into talking about types.

derek-zhou

derek-zhou

I was suggesting using structs instead of maps for arguments. Ecto might be useful to construct the structs, because there could be a need to validate the date from the (3rd party) API calls something like:

    %FreightOrder{}
    |> FreightOrder.changeset(%{
        shipping_customer: shipping_customer,
        receiving_customer: receiving_customer,
        line_items: line_items,
        ...
        })
    |>  submit!()

EDIT: seems like @sodapopcan already suggested that.

D4no0

D4no0

Structs are a great idea, I always use them, but totally forgot about them now!

Speaking about this question in general, it was not about validating data, but passing arguments around, so a map is a perfect wrapper for this kind of business, as are structs. There are cases where structs are better than maps and vice-versa, but for a beginner I think that presenting ecto structs as a good pattern to write code is a misguidance.

If the suggestion is around trying to clone strict typing in elixir, then that is a 100% anti-pattern in itself.

kccarter

kccarter OP

For a bit more context, many of our API calls return pretty large JSON responses. For now, we’ve tended to just work with string based keys because we’re worried of the atom limit, though we haven’t truly stressed tested that.

The example above would tend to produce a more deterministic set of keys for each API call, but we definitely have other parts of the codebase where the JSON response is much less deterministic.

sodapopcan

sodapopcan

You should definitely use string keys for JSON responses AFAIC pretty much exactly for the reason you stated. I also find it’s good signal to mean “untrusted.”

The idea of using Ecto is Schemas is to bring that under control. Convert the JSON into a known shape before it’s fed through the rest of your code. So you could end up with code something like: shipping_name: response["shipping_custoer_name"] || response["shipping_name"] || response["some_other_key"] or whatever logic you need. I made the mistake of only showing example of validation. Validation would only fail if something crucial is missing, but otherwise use the changeset to wrangle keys into something known, even if some are blank.

Of course, if you are blindly picking off keys and doing something with them without ever knowning what they are called then this isn’t possible. It does depend on use-case, of course.

rvirding

rvirding

Creator of Erlang

A struct has one definite advantage over using plain maps is that you define what should be in that structure and what should not be there. It gives you much better control over the data. And as structs are implemented using maps it is just as efficient as using maps directly.

D4no0

D4no0

Agree, but defining structs with 2-4 keys can be too verbose in some cases. I don’t say I’m against it, however using structs excessively can lead to obscure code that is hard to read, after all code clarity is as important as compile-time guarantees.

Where Next? Top

Trending in Questions Top

water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New

Other Trending Topics Top

budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews