fceruti

fceruti

How to normalize function params

Hey everyone!

Saša Jurić on his talk called Clarity @ ElixirConf EU 2021, mentions a way of architecting your code that uses a function that he names normalize/2, but sadly, he says he doesn’t have time to share it.

I want to implement this programming style, but I’m missing this key ingredient. Do you know any libraries or gist that accomplish this? Thanks :folded_hands:

def register(conn, params) do
  schema = [
    email: {:string, required: true},
    password: {:string, required: true}
    date_of_birth: :datetime,
    # ...
  ]

  with {:ok, params} <- normalize(params, schema),
    {:ok, user} <- MySystem.register(params) do
    # respond success
  else
  {:error, reason} ->
  # respond error
  end
end

update: if it’s not clear, I’m looking for a way of checking the existence and type of all the fields specified in schema.

Marked As Solved

riebeekn

riebeekn

I’ve not used it myself, but the tarams library looks like it might handle your use case if you want to go with a library instead of a custom solution GitHub - bluzky/tarams: Cast and validate external data and request parameters for Elixir and Phoenix · GitHub

Also Liked

sasajuric

sasajuric

Author of Elixir In Action

Yes, on both accounts. I can’t share the code (it’s owned by the clients), but the basic take is something like

def normalize(data, types) do
  {%{}, types}
  |> Ecto.Changeset.cast(data, Map.keys(types))
  |> Ecto.Changeset.apply_action(:insert)
end

This is probably not enough (e.g. you may want to handle nils and missing keys), but it’s a solid start.

tomekowal

tomekowal

I wanted to reply on Twitter but I saw there is already a solution in here. I’d like to share my anyway :slight_smile:
Since we need to traverse the schema many times, I normalize it first and then use list comprehensions like this:

  def parse(params, schema) do
    # First I want to have entire schema in one format [{key, {type, opts}}]
    normalized_schema = for {key, type_spec} <- schema, do: {key, apply_default_opts(type_spec)}
    keys = for {key, _} <- normalized_schema, do: key
    types = for {key, {type, _opts}} <- normalized_schema, into: %{}, do: {key, type}
    required_fields = for {key, {_type, opts}} <- normalized_schema, Keyword.get(opts, :required), do: key
    defaults = for {key, {_type, opts}} <- normalized_schema, default = Keyword.get(opts, :default), into: %{}, do: {key, default}

    {defaults, types}
    |> cast(params, keys)
    |> validate_required(required_fields)
    |> apply_action(:normalize)
  end

  @default_opts [required: false]
  defp apply_default_opts(type) when is_atom(type), do: {type, @default_opts}
  defp apply_default_opts({type, opts}), do: {type, Keyword.merge(@default_opts, opts)}

The line computing defaults might be tricky to understand because it is long and introduces a variable inside the comprehension filter.
Also, I’d vote for naming that function “parse” instead of normalize. Parsing is an action of taking a bunch of data and trying to transform it into a structure that is understandable. In the spirit of Parse, don’t validate

It would be possible to push the schema specification even more to introduce validations like:
email: {:string, format: ~r/@/}
and then call Changeset.validate_format(changeset, key, format) for each. But that would require another option for each Ecto.Changeset validatior. It might be an overkill but the schema format is very readable so it is tempting :smiley:

stefanchrobot

stefanchrobot

Just a few remarks if you don’t mind:

Enum.reject(schema, fn {_, v}

I’d go with something longer and less generic than k (key?) and v (value?). Maybe {field, schema}? Plus in this case I think it makes sense to use {_k, v} to explain the meaning of the first element.

cond do
  is_tuple(v) -> {k, elem(v, 0)}
  is_atom(v) -> {k, v}
  true -> raise(ArgumentError, "Bad formed schema")
end

Using more pattern matching would be more idiomatic:

case v do
  {type, _opts} when is_atom(type) -> {k, type}
  type when is_atom(type) -> {k, type}
  _ -> raise ArgumentError, "bad schema: #{inspect(v)}"
end

And one more thing:

schema |> Enum.map(fn {k, v} -> ... end) |> Enum.into(%{})

can be replaced with:

Map.new(schema, fn {k, v} -> ... end)

Last Post!

IvanR

IvanR

If you are up to a declarative approach and dependency inversion towards your application core’s data types, you can use Domo library. That generates validators and constructor functions from a t() type spec of the struct, and the type spec is validated for syntax correctness by elixir during the compilation.

So the request can be accepted into the struct that looks like the following:

defmodule Request do
  use Domo, ensure_struct_defaults: false

  defstruct [:email, :password, :date_of_birth]

  @type t :: %__MODULE__{
    email: email(),
    password: password(),
    date_of_birth: Date.t() | nil
  }

  @type email :: String.t()
  precond email: &String.match?(&1, ~r|.+\@.+\..+|)

  @type password :: String.t()
  precond password: &String.length(&1) > 7

  # Domo adds new/1, new!/1, ensure_type/1, ensure_type!/1 here automatically
end

And the validation can be done like that:

iex(1)> Request.new(%{email: "user@test.com", password: "some_password"})
{:ok, %Request{date_of_birth: nil, email: "user@test.com", password: "some_password"}}

iex(2)> Request.new(%{email: "usertestcom", date_of_birth: "none"})      
{:error,
 [
   password: "Invalid value nil for field :password of %Request{}. Expected the value matching the <<_::_*8>> type.",
   email: "Invalid value \"usertestcom\" for field :email of %Request{}. Expected the value matching the <<_::_*8>> type. And a true value from the precondition function \"&String.match?(&1, ~r|.+\\\\@.+\\\\..+|)\" defined for Request.email() type.",
   date_of_birth: "Invalid value \"none\" for field :date_of_birth of %Request{}. Expected the value matching the %Date{} | nil type."
 ]}

The dependency inversion can be done by sharing email() and password() in the shared module to have the same rules for emails and passwords across the whole app.

Domo plays nicely with Ecto schemas because they are structs too. See Domo.Changeset for this kind of integration.

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 44608 311
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New