Peri - a comprehensive schema validation library for Elixir

Hello forum!

I am excited to share Peri, a schema validation library for Elixir, designed to simplify and enhance your data validation processes. Inspired by Clojure’s Plumatic Schema, Peri brings a robust and flexible solution to the Elixir ecosystem.

What is Peri?

Peri is a library that focuses on validating raw maps and supports nested schemas, optional fields, custom validations, and a variety of data types. It provides detailed error reporting to help you debug and handle invalid data effectively.

Key Features:

  • Simple and Nested Schema Validation: Easily validate both flat and deeply nested schemas. Define your data structures in a clear and concise manner.
  • Optional and Required Fields: Specify which fields are optional and which are required. Ensure your data meets the expected criteria.
  • Custom Validations: Implement custom validation functions for complex rules specific to your application.
  • Support for Various Data Types: Validate strings, integers, floats, booleans, atoms, tuples, lists, and more.
  • Detailed Error Reporting: Receive clear and informative error messages to quickly identify and resolve issues in your data.

Example Usage:

Here’s a quick example to showcase how easy it is to define and validate a schema using Peri:

defmodule MySchemas do
  import Peri

  defschema :user, %{
    name: :string,
    age: :integer,
    email: {:required, :string},
    address: %{
      street: :string,
      city: :string
    },
    tags: {:list, :string},
    role: {:enum, [:admin, :user, :guest]},
    geolocation: {:tuple, [:float, :float]},
    rating: {:custom, &validate_rating/1}
  }

  defp validate_rating(n) when n < 10, do: :ok
  defp validate_rating(_), do: {:error, "invalid rating"}
end

user_data = %{name: "John", age: 30, email: "john@example.com", address: %{street: "123 Main St", city: "Somewhere"}, tags: ["science", "funky"], role: :admin, geolocation: {12.2, 34.2}, rating: 9}
case MySchemas.user(user_data) do
  {:ok, valid_data} -> IO.puts("Data is valid!")
  {:error, errors} -> IO.inspect(errors, label: "Validation errors")
end

In this example, we define a user schema with various fields, including nested structures, required fields, and custom validations. Peri makes it straightforward to ensure that your data conforms to these definitions.

Getting Started:

To start using Peri, add it to your mix.exs dependencies:

def deps do
  [
    {:peri, "~> 0.2"}
  ]
end

Then run mix deps.get to fetch and compile the dependency.

Documentation and Resources:

Contributing:

I welcome feedback, suggestions, and contributions from the community. If you find any issues or have ideas for improvements, please check out the contributing guidelines on GitHub.

Why Use Peri?

Peri is designed to integrate seamlessly into your Elixir projects, providing a powerful and flexible tool for schema validation. Whether you are dealing with simple data structures or complex nested schemas, Peri offers a clean and efficient solution.

I hope you find Peri useful for your Elixir applications. Feel free to share your experiences, ask questions, and contribute to the project.

Happy coding!

Peri GitHub Repository | HexDocs

25 Likes

Peri looks interesting, and I may consider using it instead of Norm to define my Typed Contracts in the Elixir Scribe tool.

In your opinion what are the pros and cons of Peri when compared with Norm?

1 Like

Thanks for the comment! I’m following the Elixir Scribe project, so it would be a pleasure to contribute it, even indirectly.

So Peri and Norm want to tackle the same problem, but the way each one achieve this is very different.

Norm focus in DSL based on macros for type safe guarantee, and go beyond that, provided native custom validations and generation for your already defined schema.

Peri on the other hand, os very recent and the main focus is to define schemas based on raw elixir data structures. No macros, no structs, no magical stuff, only the old good way to recurse into a schema and enforce structure, type casting or custom validations.

I was thought to be less complex in comparison of Ecto schemaless changesets and the cost is that the library is so simple but also very powerful.

When comparing directly with Norm i can see some pros:

  • do not depend on macros, just raw data structures and recursion for schema validations
  • it’s very flexible and allows you to build complex schemas structures very easy, with few LoC
  • provide a tree structure of errors, s you can traverse into the error structure easily and provide your own error messages (for now this need to be manual, but i will provide an API to define custom error messages on the schema)

But there are some cons as the Peri just was released 4 days ago haha.

  • Peri does not provide generation of data for your schemas, but i’ll surely tackle this issue in the near future
  • Norm is very mature and maybe handle some corner cases of complex schemas more “gracefully”. I do massive testing on Peri.

In general these are my thinkings. Peri is ideal for flexible and direct schema structure and enforcement while Norm will shine on data generation.

3 Likes

This is more then I expected as an answer. Tank you very much :grinning:

Wow, glad to meet one of the followers :smiley:

Very awesome launch. Looks impressive.

Regarding the cons I will ad a few minor ones:

  • no bang functions to create the schema to allow the Let it Crash mantra, for example MySchema.user!(attrs)
  • no conforms?/1 to later check if the schema is still valid. This may not be necessary if MySchema.user(attrs) cannot be modified directly, like we can do with structs.

If we compare to other tools on the market, what is the advantage of your library over something like nimble_options?

1 Like

Sure, it makes sense! I’ll add it to the new release.

I don’t know if I correctly understood this issue. Could you give me a more concrete example? As I know there’s no way to modify the schema function to validate directly.

Peri can be very similar to nimble_options, the difference though is that nimble_options only work with keyword lists while Peri works with ANY Elixir data structure or type.

Peri doesn’t have a native good handler for keyword lists though, this is exactly what I’m implementing right now :slight_smile:

I really like the way nimble_options validates the structure recursively, so I borrowed to provide meaningful and descriptive error trees.

2 Likes

Hey @here, I want to share with you the new release of Peri, changes are:

  • support for schema definition with literal keyword lists, like nimble_options
  • add conforms?/1 function to know if a data matches some schema
2 Likes

YAY! More schema validations in Elixir - great to see this! I’m also working on a similar lib called Drops right here :point_right: solnic/drops. Would be great to compare our approaches. I also thought about not using macros and keep it simple but in my experience defining large schemas w/o DSLs gets messy quickly, that’s why I built a DSL.

2 Likes

Hey, I do admire the drops library! I was very excited when you announced it! But for some use cases can be very clunky and complex to define a custom DSL for schema conformation. So i did a comparison between drops and peri:

Comparison of Peri and Drops Approaches

Overview

Peri and Drops are two libraries for defining and conforming schemas in Elixir, each with its own unique features and design philosophies. Here’s a detailed comparison between the two:

Simplicity and Flexibility

Peri:

  • Design Philosophy: Peri is designed to keep things simple while remaining powerful. It allows developers to define schemas and validate raw data structures with ease.
  • Schema Definition: In Peri, schemas are defined using simple Elixir or any other data structure. This makes it very approachable and easy to integrate with existing Elixir code.
  • Validation Types: Peri supports various types, including custom validations, conditional types, providing flexibility in schema definitions.
  • Error Handling: Peri accumulates validation errors, allowing for comprehensive error reporting.

Drops:

  • Design Philosophy: Drops offers a more structured and predefined customization approach. It leverages the concept of contracts for data coercion and validation.
  • Schema Definition: Schemas in Drops are defined using the Drops.Contract module, with clear differentiation between required and optional keys.
  • Validation Types: Drops provides a rich set of predefined types and predicates for additional checks. It also supports nested schemas, type-safe casting, and custom types.
  • Error Handling: Drops uses structured error messages, making it easier to understand validation issues and their causes.

Use Cases

Peri:

  • Dynamic and Flexible Validations: Ideal for applications where schemas need to be flexible and dynamically defined. Custom and conditional types are particularly useful for complex validation logic.
  • Raw Data Structure Validation: Peri excels in scenarios where raw data structures validation is required without relying on structs or Ecto changesets. So you can define schemas for simple strings or even tuples, lists, list of tuples and so on. Composability is the main philosophy of the library.

Drops:

  • Predefined Customizations: Suitable for projects that benefit from predefined validation rules and type safety. The rich set of types and predicates ensures thorough validation.
  • Complex Data Structures: Drops is well-suited for validating complex and nested data structures, thanks to its support for nested schemas and type-safe casting.

Examples

Peri Example:

defmodule MySchemas do
  import Peri

  defschema :user, %{
    name: :string,
    age: :integer,
    email: {:required, :string},
    address: %{
      street: :string,
      city: :string
    }
  end

  user_data = %{name: "John", age: 30, email: "john@example.com", address: %{street: "123 Main St", city: "Somewhere"}}
  Peri.validate(MySchemas.user, user_data)

Drops Example:

defmodule UserContract do
  use Drops.Contract

  schema do
    %{
      required(:name) => string(),
      required(:email) => string(),
      optional(:age) => integer(gt?: 18)
    }
  end
end

UserContract.conform(%{name: "Jane", email: "jane@doe.org"})

Conclusion

Peri keeps schema definitions simple and flexible, making it ideal for projects requiring dynamic and extensible validation logic. Drops, on the other hand, provides a more structured approach with rich predefined types and validation rules, making it suitable for projects where type safety and predefined validation logic are paramount.

Both libraries offer powerful validation capabilities, and the choice between them depends on the specific needs of the project and the preferences of the development team.

Also, given a more complex example of schema in peri, I would like to translate it to drops, what do you think?

defmodule MySchemas do
  import Peri

  defschema :user, %{
    name: :string,
    age: {:required, :integer},
    email: {:required, :string},
    address: %{
      street: :string,
      city: :string,
      country: {:either, {:string, :atom}}
    },
    role: {:required, {:enum, [:admin, :user, :guest]}},
    settings: {:oneof, [:map, {:list, :any}]},
    score: {:cond, &(&1 > 100), :integer, :float},
    preferences: %{
      theme: {:enum, [:light, :dark]},
      notifications: :boolean
    }
  }

  user_data = %{
    name: "John",
    age: 30,
    email: "john@example.com",
    address: %{street: "123 Main St", city: "Somewhere", country: :usa},
    role: :admin,
    settings: %{volume: 10},
    score: 105,
    preferences: %{theme: :dark, notifications: true}
  }

  case MySchemas.user(user_data) do
    {:ok, valid_data} -> IO.puts("Data is valid!")
    {:error, errors} -> IO.inspect(errors, label: "Validation errors")
  end

This schema demonstrates:

Conditional Type: score is validated as an integer if it’s greater than 100, otherwise as a float.

Either Type: country can be either a string or an atom.

One-of Type: settings can be either a map or a list.

Enum Type: role and preferences.theme must be one of the specified values.

8 Likes

I think this is missing from the docs. It is an odd validation.

Yeah, I need to revisit the docs! I’ll update it. Thanks for noting that!

Removed. I realized I don’t actually have a strong opinion.

announcing new features and performance improvements in Peri, our schema validation library! Here’s a quick overview:

New Features:

1. Transform Type

  • Apply a transformation function to your data post-validation.
  • Example:
    defschema :user, %{
      name: :string,
      age: {:integer, {:transform, &(&1 * 2)}}
    }
    

2. Default Values

  • Set default values for fields when they are missing.
  • Example:
    defschema :user, %{
      name: {:string, {:default, "Anonymous"}}
    }
    

3. Dependent Types

  • Validate fields based on the value of other fields.
  • Example:
    defschema :user, %{
      age: :integer,
      license: {:dependent, :age, fn age -> age >= 18 end, :string}
    }
    

Performance Improvements:

  • Enhanced Peri.Parser for faster schema traversal and error handling.

Check out the latest version on GitHub and let us know your thoughts!

Happy coding!

2 Likes

Hey, nice library. Works well so far. I’m finding myself writing both 1) Peri schema 2) defstruct. Feels like I’m duplicating work. Any suggestions?

defmodule SightxrApi.Job.WorkflowSchemas do
  import Peri

  defschema(:parse_documents, %{
    "files" => {:required, {:list, get_schema(:file)}},
    "out_dirs" => {:list, :string},
    "parser" => {:enum, ["tika", "azure_document_intelligence"]}
  })

  # ... omitted for brevity
end

And the defstruct:

defmodule SightxrApi.Job.ParseDocumentsWorkflow do
  defmodule Options do
    defstruct [:files, :parser, :out_dirs]

    def new(args) do
      %__MODULE__{
        files: args["files"],
        parser: args["parser"],
        out_dirs: args["out_dirs"]
      }
    end
  end

  # ... omitted for brevity
end
1 Like

not sure if this helps, but in my structs, to reduce duplication, I am using the non-macro API:

  @schema %{
    value: {:required, {:float, {:gt, 0}}},
    unit: {:required, {:enum, [:meters]}}
  }

  @keys Map.keys(@schema)
  defstruct @keys

  def new(params) do
    with {:ok, data}  <- Peri.validate(@schema, params) do
      {:ok, struct(__MODULE__, data)}
    end
  end

```
1 Like