Jsonpatch - Pure Elixir implementation of RFC 6902

A JSON patch is a way to define a sequence of manipulating operations on a JavaScript object. The IETF published the RFC 6902 - found here https://tools.ietf.org/html/rfc6902. The Kubernetes API is an user of those JSON patches. In my opinion this is the easiest way to change a Kubernetes resource. Unfortunantely no library was available until yet or I did not fount it. This was my motivation to create the JSON patch library.

In this sense - keep calm and code Elixir
Your Corka/Sebastian

18 Likes

There was:

But it doesn’t seem to have been updated in a while according to hex. Also when I used that library at my last company I noticed it wasn’t that fast. So there is defiantly room for improvement.

I’m not sure it’s good to rely on a json encoder/decoder.

Also be careful about lists they are not that clear in json patch standard and can cause of lot of problems.

1 Like

Thanks for your feedback

Well, my search skills fail. I will have a look at that lib for comparison. But I already saw that it offers a better error feedback. This is currently not the best for Jsonpatch. Anyhow I also have to take a deeper look at performance.

I’m not sure it’s good to rely on a json encoder/decoder.

This was historically grown. I just wanted to offer the option for mapping but I build first the encoder/decoder. I will leave it to the consumer of the lib what encoder/coder they want to use. I removed the dependency.

v0.9.0

The new release added helpful feedback on errors while patching.

In this example the test operation failed.

iex> patch = [
...> %Jsonpatch.Operation.Add{path: "/age", value: 33},
...> %Jsonpatch.Operation.Test{path: "/name", value: "Alice"}
...> ]
iex> target = %{"name" => "Bob", "married" => false, "hobbies" => ["Sport", "Elixir", "Football"], "home" => "Berlin"}
iex> Jsonpatch.apply_patch(patch, target)
{:error, :test_failed, "Expected value 'Alice' at '/name'"}

Other possible errors are :invalid_path and :invalid_index.

1 Like

v0.9.3

It has been a since the last update. I worked on testing to eliminate dead code and bugs (of course). Also I tried to make the documentation a little bit prettier.

I am happy about any feedback.

Your Corka/Sebastian

7 Likes

v0.10.0

A new minor version is out. Recently I discovered that escaping exists for ~ and /. Also “know your standard library”: I was using Enum.with_index + Enum.find which was really not clever and replaced it with Enum.fetch.

Breaking change: Jsonpatch.apply_patch/2 was not really matching the Elixir style in my opinion. Therefore it was changed and Jsonpatch.apply_patch!/2 was added.

Before:

iex> Jsonpatch.apply_patch(patch, target)
%{"name" => "Bob", "married" => true, "hobbies" => ["Elixir!"], "age" => 33}

Now:

iex> Jsonpatch.apply_patch(patch, target)
{:ok, %{"name" => "Bob", "married" => true, "hobbies" => ["Elixir!"], "age" => 33}}

In contrast Jsonpatch.apply_patch!/2 will return no tuple. Instead it will return the patched value or JsonpatchException.

Changelog:

  • Made jsonpatch more Elixir-API-like by adding Jsonpatch.apply_patch! (which raise an exception) and changed Jsonpatch.apply_patch to return a tuple.
  • Implemented escaping for ‘~’ and ‘/’
  • Allow usage of ‘-’ for Add and Copy operation
  • Fixed adding and copying values to array
  • Improved error feedback of test operation
  • Fixed: Replace operation adds error to target
  • Cleaned code: Replaced strange constructs of Enum.with_index with Enum.fetch

Updated documentation: Jsonpatch — Jsonpatch v0.10.0

1 Like

v0.11.0

The whole process for creating patches through Jsonpatch.diff/2 was not perfect. I was encouraged through a github issue to rewrite it with a reduce-loop. This version fixes this. In addition, Jsonpatch.FlatMap was removed because it is not necessary anymore and it was not the focus of this lib.

Changelog:

  • Removed module Jsonpatch.FlatMap because it is not necessary anymore and not the focus of the lib
  • Reworked creating diff to create less unnecessary data and for more accurate patches
  • Fixed adding values to empty lists (thanks @webdeb )

Updated documentation: Jsonpatch — Jsonpatch v0.11.0

7 Likes

(No version update)

Thanks to Mix.install it is really easy to write a small script to create a patch with Jsonpatch. :heart_eyes:

Mix.install([:jsonpatch, :poison])

source =
  File.read!("foo.json")
  |> Poison.Parser.parse!(%{})

destination =
  File.read!("bar.json")
  |> Poison.Parser.parse!(%{})

patch =
  source
  |> Jsonpatch.diff(destination)
  |> Jsonpatch.Mapper.to_map()

IO.inspect(patch, label: :patch)

It is so small. I love it.

4 Likes

v0.12.0

After a long time a new release was published.

Previously Jsonpatch sorted the patches before applying them when they were provided as list. This was wrong.
The RFC says that they should be applied in the order as they appear in the list.

This change can affect the result of applying multiple JSON patches.

Updated documentation: Jsonpatch — Jsonpatch v0.12.0

3 Likes

v0.12.1

The new version brings a better order for generated patches by Jsonpatch.diff/2 when lists were compared.

Before v0.12.1

iex(1)> source = %{"list" => []}
%{"list" => []}
iex(2)> destination = %{"list" => [1,2]}
%{"list" => [1, 2]}
iex(3)> Jsonpatch.diff source, destination
[
  %Jsonpatch.Operation.Add{path: "/list/1", value: 2},
  %Jsonpatch.Operation.Add{path: "/list/0", value: 1}
]

With v0.12.1

iex(1)> source = %{"list" => []}
%{"list" => []}
iex(2)> destination = %{"list" => [1,2]}
%{"list" => [1, 2]}
iex(3)> Jsonpatch.diff source, destination
[
  %Jsonpatch.Operation.Add{path: "/list/0", value: 1},
  %Jsonpatch.Operation.Add{path: "/list/1", value: 2}
]

Now the created patches can be directly applied and will created the expected destination.

Thanks @smartepsh for the PR :slight_smile:

3 Likes

v0.13.0

A new version is out. It is finally possible to patch maps with atoms as keys.

The new option for apply_patch was inspired by Jason.

New option:

  • :keys - controls how parts of paths are decoded. Possible values:
    • :strings (default) - decodes parts of paths as binary strings,
    • :atoms - parts of paths are converted to atoms using String.to_atom/1,
    • :atoms! - parts of paths are converted to atoms using String.to_existing_atom/1

Example

iex(1)> source = %{role: "Developer"}
%{role: "Developer"}

iex(2)> target = %{role: "Developer", team: "A"}
%{role: "Developer", team: "A"}

iex(3)> patch = Jsonpatch.diff source, target
[%Jsonpatch.Operation.Add{path: "/team", value: "A"}]

iex(4)> Jsonpatch.apply_patch patch, source, keys: :atoms
{:ok, %{role: "Developer", team: "A"}}

Cheers
Sebastian

2 Likes

v1.0.0

I know it exists but never had the time to implement it. I am talking about that patching of lists at top level never worked. This was for me the milestone for v1.0.0. Finally it is fixed. :partying_face:.

Changes:

  • Allow lists at top level of Jsonpatch.apply_patch
  • Fix error message when updating a non existing key in list
  • Performance boost for diffing

Example of new feature:

iex(1)> source = [1, 2, 3, 5, 6]
[1, 2, 3, 5, 6]
iex(2)> destination = [1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]
iex(3)> patch = Jsonpatch.diff source, destination
[
  %Jsonpatch.Operation.Add{path: "/5", value: 6},
  %Jsonpatch.Operation.Replace{path: "/4", value: 5},
  %Jsonpatch.Operation.Replace{path: "/3", value: 4}
]
iex(4)> Jsonpatch.apply_patch! patch, source
[1, 2, 3, 4, 5, 6]
8 Likes

v1.0.1

This version fixes a small bug while creating diffs.

Changes:

  • Escape remaining keys before comparing them to the (already escaped) keys from earlier in the diffing process when determining Remove operations

Kudos to @ACBullen for detecting and fixing it. :pray:

Go this way for the new version :point_right: jsonpatch | Hex

5 Likes

v1.0.2

This version brings another fix for creating diffs.

Changes:

Before

source = %{"id" => nil}
destination = %{"id" => nil}

[%Jsonpatch.Operation.Add{path: "/id", value: nil}] == Json.diff source, destination

With hotfix

source = %{"id" => nil}
destination = %{"id" => nil}

[] == Json.diff source, destination

Big thanks to @stuartc for the merge request!

3 Likes

v2.0.0

The Jsonpatch library got thanks to big efforts from @visciang and the Athonet Team a new major version. The new major version was necessary because of a rework of the public API. From my point of view they make the library more easier to use without converting too much to Jsonpatch’s own struct.

Checkout the updated docs: Jsonpatch — Jsonpatch v2.0.0

Changelog

  • Bugfix - ADD behaviour is now compliant with RFC (insert or update)
  • Bugfix - allow usage of nil values, previous implementation used Map.get with default nil to detect if a key was not present
  • Change - COPY operation to be based on ADD operation (as per RFC)
  • Change - MOVE operation to be based on COPY+REMOVE operation (as per RFC)
  • Change - REPLACE operation to be based on REMOVE+ADD operation (as per RFC)
  • Change - Jsonpatch.apply_patch() signature changes:
  • patches can be defined as Jsonpatch.Operation.Add/Copy/Remove/... structs or with plain map conforming to the jsonpatch schema
  • error reason is now defined with a {:error, %Jsonpatch.Error{}} tuple.
    %Jsonpatch.Error{patch_index: _, path: _, reason: _} reports the patch index, the path and the reason that caused the error.
  • Removed - Jsonpatch.Mapper module, in favour of new Jsonpatch.apply_patch signature
  • Removed - Jsonpatch.Operation protocol
  • Feature - introduced new Jsonpatch.apply_patch() option keys: {:custom, convert_fn} to convert path fragments with a user specific logic
  • Improvements - increased test coverage
1 Like

v2.1.1

This version should make the library more compatible with other libraries and frameworks by favoring maps over structs when creating diffs.

An updated examples of creating a diff looks like:

source = %{"name" => "Bob", "married" => false, "hobbies" => ["Sport", "Elixir", "Football"]}
destination = %{"name" => "Bob", "married" => true, "hobbies" => ["Elixir!"], "age" => 33}
Jsonpatch.diff(source, destination)
[
  %{path: "/married", value: true, op: "replace"},
  %{path: "/hobbies/2", op: "remove"},
  %{path: "/hobbies/1", op: "remove"},
  %{path: "/hobbies/0", value: "Elixir!", op: "replace"},
  %{path: "/age", value: 33, op: "add"}
]

The Jsonpatch structs were not removed and are still valid and usable!

1 Like

v2.2.0

Yet another improvement.

The new version solves some edge cases regarding patching the root document by providing "" as path. In addition thx to @tulinmola a new option was added that allows to ignore invalid paths.

      iex> # Patch will succeed, not applying invalid path operations.
      iex> patch = [
      ...> %{op: "replace", path: "/name", value: "Alice"},
      ...> %{op: "replace", path: "/age", value: 42}
      ...> ]
      iex> target = %{"name" => "Bob"} # No age in target
      iex> Jsonpatch.apply_patch(patch, target, ignore_invalid_paths: true)
      {:ok, %{"name" => "Alice"}}
4 Likes