igoldchluk

igoldchluk

RockSolid - Data generation from JSON Schema definitions

Hi everyone

I have been working for a while on RockSolid, a library to generate valid data from JSON schemas.

It is built on top of (the awesome) StreamData, so you can use it directly in property tests, or as a data generator. It supports JSON Schema drafts 04 through 2020-12, including recursive schemas, remote references, dependent schemas, if/then/else, etc.

It is tested against the open SchemaStore catalog, which contains some complex schemas with 10k+ lines, and can generate valid data for over 85% of the documents. So, while far from perfect, it is useful for most of the realistic scenarios. It has been useful for me, so hopefully you find it useful too.

Usage

Inside property tests

  property "generates valid user profiles" do
    schema = %{
      "type" => "object",
      "additionalProperties" => false,
      "properties" => %{
        "birthDate" => %{"type" => "string", "format" => "date"},
        "name" => %{"type" => "string", "pattern" => "^[A-Z][a-z]+$"},
        "email" => %{"type" => "string", "format" => "email"}
      },
      "required" => ["birthDate", "name", "email"]
    }

    check all user_data <- RockSolid.from_schema(schema) do
      assert Regex.match?(~r/^[A-Z][a-z]+$/, user_data["name"])
      assert %Date{} = Date.from_iso8601!(user_data["birthDate"])
      assert String.split(user_data["email"], "@") |> length() == 2
    end
  end

As a regular generator

iex(1)> specs = %{
  "type" => "object",
  "properties" => %{
    "serverIPs" => %{
      "type" => "array",
      "items" => %{"type" => "string", "format" => "ipv4"},
      "uniqueItems" => true,
      "minItems" => 1
    },
    "serverName" => %{"pattern" => "^[a-z][a-z_0-9]{2,255}$", "type" => "string"},
  },
  "required" => ["serverIPs", "serverName"],
  "additionalProperties" => false
}

iex(2)> specs |> RockSolid.from_schema() |> Enum.take(3)
[
  %{"serverIPs" => ["148.50.92.205"], "serverName" => "a5l"},
  %{"serverIPs" => ["230.26.166.121"], "serverName" => "y_w"},
  %{
    "serverIPs" => ["144.154.111.248", "155.134.134.38"],
    "serverName" => "v2_5"
  }
]

Feel free to give it a try and provde any feedback :smiley:

Links

Most Liked

dimitarvp

dimitarvp

Dang that’s awesome. Thanks for making it!

No immediate use for it but reusing schemata as data generators is extremely valuable in many scenarios.

Where Next?

Popular in Announcing Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
tmbb
I’ve published the first version of my Makeup library. It’s a syntax highlighter for Elixir in the spirit of Pygments, Currently it highl...
New
dominicletz
Hi, I thought I had posted my library before but seems I hadn’t. The project is still in early stages but it’s growing and so I think it...
New
restlessronin
The repo is at GitHub - cyberchitta/openai_ex: Community maintained Elixir library for OpenAI API · GitHub. Docs are at OpenaiEx User Gu...
152 10342 134
New
blatyo
The best overview for how things are tied together is this presentation. Modules and functions are pretty well documented at this point, ...
New
michalmuskala
Another small library today. PersistentEts Hex: persistent_ets | Hex GitHub: GitHub - michalmuskala/persistent_ets · GitHub Ets table ...
New
sabiwara
Dune is a sandbox for Elixir and aims to safely evaluate user-provided code. You can try it out using this basic Elixir playground made ...
New
versilov
Could not wait for the missing Elixir ML libraries to appear, so, I wrote one myself, taking https://github.com/sdwolfz/exlearn as a foun...
New
achempion
Hi, I would like to tell about my initiative to further maintain and develop Waffle project which is the fork of Arc library. The progre...
New
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48342 226
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement