Deep_merge 0.1.0 - deep merging maps, keyword lists + more

Your Libraries & OS Mentoring Libraries
,
1

Hey everyone, I created and published a new library to deep merge maps and keyword lists. You can find it on github, hex and hexdocs. I also wrote a small release blog post detailing how the library came to be after trying to get deep_merge into elixir.

Library usage is easy as expected:

DeepMerge.deep_merge(%{a: 1, b: [x: 10, y: 9]}, %{b: [y: 20, z: 30], c: 4})
# => %{a: 1, b: [x: 10, y: 20, z: 30], c: 4}

More features include:

  • It handles both maps and keyword lists
  • It does not merge structs or maps with structs…
  • …but you can implement the simple DeepMerge.Resolver protocol for types/structs of your choice to also make them be deep mergable
  • a deep_merge/3 variant that gets a function similar to Map.merge/3 to modify the merging behavior, for instance in case you don’t want keyword lists to be merged or you want all lists to be appended

Feedback, comments and others all welcome :slight_smile:

4 Likes
2

This is a very common need on many projects. @josevalim do you think this feature could be part of the Elixir Map and List API?

3

See https://github.com/elixir-lang/elixir/pull/5339 for a discussion on why it is not part of core.

2 Likes
4

How about merge when one of Keyword have two elements with same key?

> iex -S mix
Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10]

Interactive Elixir (1.3.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> DeepMerge.deep_merge([a: 1, b: 2, a: 3], [a: 0])
[b: 2, a: 0]

I expected results like:

[a: 0, b: 2, a: 3] # change value ONLY for first element with key: "a"

How about to generate diff and apply only insert and update changes?

5

As José said you’ll find the reasons there, I described the story in the release blog post - it started with a proposal on elixir-core.
Mainly concerns were about how often it’d be useful, that the right implementation would probably be protocols and that it’d be a bit overkill to add a protocol to the language for a feature with questioned general utilization.

It might be my interpretation, but there seems to be a chance that something like deep_merge will be added to Elixir at some time in the future if it turns out to be more useful/popular/something:

I believe though there are still too many questions to be answered before moving forward.

and

(…) as long as you pick a sane namespace, like DeepMerge, it should be fine. It is unlikely we would add it to Elixir as DeepMerge.

All that said, I think a library is a perfectly fine place for this to live, thanks to hex it’s easy to add and use and I hope the library lives up to all your quality standards and if not please tell me :slight_smile:

Plus, I have to commend the elixir team for rejecting features. Imo it’s a hard but very necessary thing to do to keep a project lean and maintainable :clap:

3 Likes
6

Hi there! Thanks for the feedback :slight_smile:

To be honest I was a bit surprised by the keyword list merging behavior as well, but basically this follows what Keyword.merge/2 does in Elixir (it uses Keyword.merge/3 underneath) and I want it to be the same and be consistent with whatever Elixir does because otherwise it’d just be weird and confusing.

tobi@comfy ~ $ iex
Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.3.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Keyword.merge [a: 1, b: 2, a: 3], [a: 0]
[b: 2, a: 0]

That said, the mailing list discussion showed me that people have different ways they want deep_merge to work especially for lists/keyword lists so that might be a great use case for you for DeepMerge.deep_merge/3 and the reason why I didn’t publish until I had that worked out :slight_smile:

7

Totally agree. :clap::clap::clap:

Nice work guys, if the choice was mine, this language would not be as wonderful as it is! Awesome community and awesome maintainers!

1 Like
8

To be honest I’m also a bit surprised! :smile:
I agree with your arguments, but I’m not satisfied.
I think I can add merge and reverse_merge (for sorting order) methods to my differ package that I’m currently working on it.
My merge methods will be based on diff so it will follow the second way that I asked about.

I see only this two ways to merge (elixir based and diff based). Is there other ways that developers need?

9

I think so, yes.

  • override behaviour (aka take the right side), at least for deep_merge as I could have a map with keywords as values but I don’t want them to be merged but rather treat them as values
  • domain logic - not something I’ve actively tried but I think if you think about ecto’s queries and their keywords you could come up with a number of valid ways to “merge” them (intersection, union, append, connect with OR/AND), although I don’t think it’s a good thing to ever do
  • given that duplicated keys are absolutely fine in keyword lists one might just wish to append - this is also true for merging “normal” lists (someone on the mailing list actually wanted this) - but it’s hard to distinguish as “normal” lists back keyword lists

I’m sure other people can and will come up with more ways :slight_smile:

2 Likes
10

Resurrection time :stuck_out_tongue_winking_eye:

I just released 1.0.0 as the API has been proven stable, added some small goodies to go with it.

I consider it feature completely, so save good feature ideas, bugs, performance improvements, compiler warnings it shouldn’t see any more updates.

Enjoy deep merging :slight_smile:

3 Likes