I know the various Kernel.update_in and related functions.
They don’t seem to work wit regular lists (accessed by index), though, only keyword lists, as well as maps and structs. What is the idiom for updating a struct which has a list attribute, which contains structs.
In my use-case I want to find one of the several structs by ID, then apply an update / replace, update_in style.
What do you use in that case?
Something like this (some additional background)?
If a key is a function, the function will be invoked passing three arguments, the operation (
:get_and_update), the data to be accessed, and a function to be invoked next.
Creating that function can be a bit of a mind bender.
Looked at Access.at/1?
This is what I use:
Mostly because I wrote it and I like the idea of being able to transform structures w/o knowing the overall large structure.
Thanks for your answer. Yes, I did actually overlook Access.at, tbh. That solves the case where I know a position upfront.
For the case where I know an identifying property (id: 123), I’m sure there is an answer in passing the function. But I’m a bit lost over how to use this next function it receives.
Looking at that example from your other post you link above:
all = fn :get_and_update, data, next ->
Enum.map(data, next) |> :lists.unzip
end
I assume the Enum.map part is there, because the function is to process all elements.
I’ll try to fit that onto my problem…
OK, so what I come up with looks actually a bit intense, but it works.
I have this structure:
%Issue{
notes: [
%Note{
uuid: "123",
text: "It’s like this."
},
%Note{
uuid: "456",
text: "It’s like that."
}
]
}
by_id = fn (x) -> x.uuid == "123" end
update_in(issue |> Map.from_struct, [:notes, find(by_id)], fn (note) ->
Map.merge(note, Map.from_struct(%{text: "new text"}))
end)
def find(pred) when is_function(pred, 1) do
fn :get_and_update, data, next ->
Enum.map(data, fn(x) ->
if pred.(x) do
next.(x)
else
{x, x}
end
end) |> :lists.unzip
end
end
I guess the case where one passes in a function was only thought of as an API for internal use, or extension point, but not for “everyday use.” Because for other cases the arguments and expected return value structures are just very demanding. (Unless I use this whole thing wrong, obviously.)
One other thing I’m not happy about is the issue |> Map.from_struct transformation coming into update_in. But otherwise the compiler tells me I’d have to implement Access behaviour, which furthermore seems to be deprecated.
Some sharp edges here. Comments on how to simplify or improve this code are welcome.
What is the before and after?
I’m not understanding what you are trying to do, Update the text field in note with uuid: 123?
Sorry, yes, that’s right.
I’ve edit the code to make that more obvious.
I enjoy the fact that pattern matching works with the anonymous predicate, too.
by_id = fn
%{uuid: "123"} -> true
_ -> false
end
But that’s a distraction.
Thanks for you suggestion. It isn’t quite obvious for me how this package works, tbh. The motivation on the README is spot on, but the suggested solution I don’t quite understand.
defmodule Note do
@uuid_default ""
@text_default ""
defstruct uuid: @uuid_default, text: @text_default
def note_text_get_update(uuid, new_text),
do: fn
%Note{uuid: ^uuid} = note -> {note, Map.put(note, :text, new_text)}
note -> {note, note}
end
def note_text_update(uuid, new_text),
do: fn
%Note{uuid: ^uuid} = note -> Map.put(note, :text, new_text)
note -> note
end
def text_access(:get_and_update, data, next) do
{old, new} = text_access(:get, data, next)
{old, (Map.put data, :text, new)}
end
def text_access(:get, data, next) do
text = Map.get(data, :text, @text_default)
next.(text)
end
end
defmodule Issue do
@notes_default []
defstruct notes: @notes_default
def notes_access(:get_and_update, data, next) do
notes = Map.get(data, :notes, @notes_default)
{old, new} = next.(notes)
{old, (Map.put data, :notes, new)}
end
def notes_access(:get, data, next) do
notes = Map.get(data, :notes, @notes_default)
next.(notes)
end
def all_access(:get_and_update, list, next) when is_list(list) do
old_new = (Enum.map list, next)
:lists.unzip old_new
end
def all_access(:get, list, next) when is_list(list) do
Enum.map list, next
end
def all_notes_access(:get_and_update, data, next) do
{old, new} =
:lists.unzip(all_notes_access(:get, data, next))
{old, (Map.put data, :notes, new)}
end
def all_notes_access(:get, data, next) do
data
|> Map.get(:notes, @notes_default)
|> Enum.map(next)
end
end
defmodule Demo do
def run do
issue = %Issue{
notes: [
%Note{
uuid: "123",
text: "It’s like this."
},
%Note{
uuid: "456",
text: "It’s like that."
}
]
}
IO.puts "Issue: #{inspect issue}"
notes = &Issue.notes_access/3
all = &Issue.all_access/3
ntgu = Note.note_text_get_update("123","new text")
{replaced, updated} = Kernel.get_and_update_in(issue, [notes,all], ntgu)
IO.puts "Updated: #{inspect updated}"
IO.puts "Replaced: #{inspect replaced}"
ntu = Note.note_text_update("123","text new")
updated2 = Kernel.update_in(issue, [notes,all], ntu)
IO.puts "Updated2: #{inspect updated2}"
all_notes = &Issue.all_notes_access/3
{replaced3, updated3} = Kernel.get_and_update_in(issue, [all_notes], ntgu)
IO.puts "Updated3: #{inspect updated3}"
IO.puts "Replaced3: #{inspect replaced3}"
updated4 = Kernel.update_in(issue, [all_notes], ntu)
IO.puts "Updated4: #{inspect updated4}"
text = &Note.text_access/3
IO.puts "Get1: #{inspect Kernel.get_in(issue, [notes,all,text])}"
IO.puts "Get2: #{inspect Kernel.get_in(issue, [all_notes,text])}"
IO.puts "Updated5: #{inspect Kernel.update_in(issue, [all_notes,text],&("Desc: " <> &1))}"
end
end
Demo.run()
$ elixir demo.exs
Issue: %Issue{notes: [%Note{text: "It’s like this.", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
Updated: %Issue{notes: [%Note{text: "new text", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
Replaced: [%Note{text: "It’s like this.", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]
Updated2: %Issue{notes: [%Note{text: "text new", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
Updated3: %Issue{notes: [%Note{text: "new text", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
Replaced3: [%Note{text: "It’s like this.", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]
Updated4: %Issue{notes: [%Note{text: "text new", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
Get1: ["It’s like this.", "It’s like that."]
Get2: ["It’s like this.", "It’s like that."]
Updated5: %Issue{notes: [%Note{text: "Desc: It’s like this.", uuid: "123"}, %Note{text: "Desc: It’s like that.", uuid: "456"}]}
$
Stripped down version:
defmodule Note do
defstruct uuid: "", text: ""
end
defmodule Issue do
defstruct notes: []
end
defmodule Demo do
def note_text_update(uuid, new_text),
do: fn
%Note{uuid: ^uuid} = note -> Map.put(note, :text, new_text)
note -> note
end
def all_notes_access(data, next) do
data
|> Map.get(:notes, [])
|> Enum.map(next)
end
def all_notes_access(:get, data, next) do
all_notes_access(data, next)
end
def all_notes_access(:get_and_update, data, next) do
{old, new} =
:lists.unzip(all_notes_access(data, next))
{old, (Map.put data, :notes, new)}
end
def run do
issue = %Issue{
notes: [
%Note{
uuid: "123",
text: "It’s like this."
},
%Note{
uuid: "456",
text: "It’s like that."
}
]
}
IO.puts "Issue: #{inspect issue}"
all_notes = &all_notes_access/3
ntu = note_text_update("123","new text")
updated = Kernel.update_in(issue, [all_notes], ntu)
IO.puts "Updated: #{inspect updated}"
end
end
Demo.run()
Okay. Here’s how to use phst_tranform for this.
defmodule UpdateNotes do
import PhstTransform
def update_note(note, uuid, text) do
if note.uuid == uuid do
note.text = text
end
end
def update_in(data,uuid, text) do
note_potion = %{ Note => fn(note) -> update_note(note, uuid, text) end) }
transform(data, note_potion)
end
end
Basically, you write the function that updates the item you want ( Note Struct) , create the “potion” map that maps data type to function. Tranform will take in any data structure and do a depth first traversal and any where it finds a Note Struct, will apply the function you have provided. For many uses this is like using a backhoe to drive a nail, but it makes many complicated things very simple.
Did I get this right, the data to be found means it has to match?
So passing %Note{uuid: "123"} would find the struct with all attributes?
That’s actually pretty awesome as an interface. Quite pragmatic, I like it.
I wrote a library called focus that I think would work here. Focus lets you get, set, and apply functions over values inside arbitrarily nested data structures.
I think something like this would achieve what you’re looking for:
defmodule Issue do
import Lens
deflenses notes: []
end
defmodule Note do
import Lens
deflenses uuid: "", text: ""
end
issue = %Issue{
notes: [
%Note{
uuid: "123",
text: "It’s like this."
},
%Note{
uuid: "456",
text: "It’s like that."
}
]
}
Issue.notes_lens
~> Lens.idx(Enum.find_index(issue.notes, &(&1.uuid == "123")))
~> Note.text_lens
|> Focus.set(issue, "New Text")
# %Issue{notes: [%Note{text: "New Text", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
The deflenses macro is a wrapper around defstruct that also defines lenses (a combined getter and setter) for the struct’s fields. You could also create them by hand with e.g. notes_lens = Lens.make_lens(:notes).
The bit at the end:
Issue.notes_lens
~> Lens.idx(Enum.find_index(issue.notes, &(&1.uuid == "123")))
~> Note.text_lens
|> Focus.set(issue, "New Text")
Focus.set/3 takes a lens, a data structure, and a new value for the focus of the lens. The ~> combines lenses to let you traverse the keys/indexes/etc…. So here Focus.set traverses :notes, the index where uuid === "123", and the :text key for issue and sets the value at that path to New Text.
Thanks for your elaborate examples, much appreciated.
Still I don’t quite make sense of this all. Quoting from the docs:
If a key is a function, the function will be invoked passing three arguments, the operation (:get_and_update), the data to be accessed, and a function to be invoked next. This means get_and_update_in/3 can be extended to provide custom lookups. The downside is that functions cannot be stored as keys in the accessed data structures.
It doesn’t explain what the operation parameter is about, what kind of “next” function we’re looking at, or what structure the expected return value should conform to.
What is the purpose or responsiblity of this function? Is it kind of a selector, something that would be equivalent to a map key, like :age? That seems to be the case, as Access.at, Access.all seem to be from that category. So if that’s true, how does the ‘selector function’ (for lack of it’s actual name) is supposed to treat each kind? Should the next function be invoked for the matches only? That seems to be what you are doing.
After spending still more time it’s dawning on my what the ideas are.
Basically there are two very different use cases, :get and :get_and_update.
We are navigating a tree structure by specifying a path. The path is being represented as a list of keys. Each key can also be a function which adheres to a specific contract (signature?).
There are two contracts, as there are two quite distinct use-cases, the simpler nested access, and nested update. The use :get and :get_and_update respectively.
:get just fetches some subset (of possibly one), but potentiall the whole set for values. It’s used for the nested access case. It is expected to return a single result, but that can be a collection, or a single element, and anything in between. It can be the subset that fits a certain condition of a predicate, e.g. to name one way I was thinking of.
Now we look at the update case. Keeping in mind that functional data structures need to handled in a copy-on-write style in order to update them. So when traversing them for update, we need to find the nodes that we follow until we reach the node to replace, and the return back up the call stack and structure in order to replace the newly updated node from the deeper position into the exiting structure. This needs to happen all the way up to the root. And that’s the reason we need two copies of the data on the stack, the updated subset, as well the original structure. On the way down the path we need to put the original data onto the stack, so that we have it for modification when traveling back up the callstack. At the same time we need the updated subset, because we need to combine the previous structure and the updated subset into the new version of the data structure.
:get_and_update is supporting the copy-on-write use case. Only the subset that’s supposed to be modified is fed into the next function.
Still, I’m not quite sure what the next function is. Is it the the function corresponding to the next key element?
FWIW, I had the idea to have path element functions for
And I’ve managed to implement those. Check out this code, implementation and tests:
defmodule NestedUpdate do
def find(key, value) do
fn :get_and_update, data, next ->
Enum.map(data, fn(x) ->
if Map.get(x, key, :miss) == value do
next.(x)
else
{x, x}
end
end) |> :lists.unzip
end
end
def find(pred) when is_function(pred, 1) do
fn :get_and_update, data, next ->
Enum.map(data, fn(x) ->
if pred.(x) do
next.(x)
else
{x, x}
end
end) |> :lists.unzip
end
end
end
defmodule NestedUpdatedFindTest do
use ExUnit.Case
import Access, only: [key: 1, all: 0]
import NestedUpdate, only: [find: 1, find: 2]
defmodule Note do
defstruct [:id, :text, :sections]
end
defp data() do
%{
name: "Tom",
notes: [
%Note{id: 1, sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]},
%Note{id: 2, sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]},
%Note{id: 3, sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]}
]
}
end
setup do
[data: data()]
end
describe "find/1" do
test "update element identified by predicate as middle path element", %{data: data} do
updated = update_in(data, [
key(:notes),
find(&(&1.id == 1)),
key(:sections),
find(&(&1.id == 2))
], fn
note -> %{note | text: "UPDATE"}
end)
assert %{name: "Tom",
notes: [
%Note{id: 1,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "UPDATE"},
%Note{id: 3, text: "Note 3"}
]},
%Note{id: 2,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]},
%Note{id: 3,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]
}
]
} = updated
end
end
describe "find/2" do
test "update multiple elements", %{data: data} do
updated = update_in(data, [key(:notes), all(), key(:sections), find(:id, 3)], fn
note -> %{note | text: "UPDATE"}
end)
assert %{name: "Tom",
notes: [
%Note{id: 1,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "UPDATE"}
]},
%Note{id: 2,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "UPDATE"}
]},
%Note{id: 3,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "UPDATE"}
]
}
]
} = updated
end
test "update element identified by id as middle path element", %{data: data} do
updated = update_in(data, [key(:notes), find(:id, 2), key(:sections), find(:id, 3)], fn
note -> %{note | text: "UPDATE"}
end)
assert %{name: "Tom",
notes: [
%Note{id: 1,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]},
%Note{id: 2,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "UPDATE"}
]},
%Note{id: 3,
sections: [
%Note{id: 1, text: "Note 1"},
%Note{id: 2, text: "Note 2"},
%Note{id: 3, text: "Note 3"}
]
}
]
} = updated
end
end
end
It would be cool to get feedback on any obvious problems, or hints at how to be smarter / simpler / more idiomatic here.
That looks very cool, but I hesitate to settle on a solution that makes me use something other than plain vanilla structs, tbh. I seem to remember that lenses are a well-established FP concept. Just curious, is it common to need specialty data structures for that? (Sorry for the ignorance.)
That’s perfectly understandable. deflenses in that example is just a convenience for defining a vanilla struct and a lens (it was inspired by the schema macro from Ecto). The only specialty data structure needed is the lens itself: a struct with getter and setter functions.
You can just as easily define the structs and the lenses separately:
defmodule Issue do
defstruct notes: []
def notes_lens() do
Lens.make_lens(:notes)
end
end
defmodule Note do
defstruct uuid: "", text: ""
def uuid_lens() do
Lens.make_lens(:uuid)
end
def text_lens() do
Lens.make_lens(:text)
end
end
Everything else in the snipped above would remain the same:
issue = %Issue{
notes: [
%Note{
uuid: "123",
text: "It’s like this."
},
%Note{
uuid: "456",
text: "It’s like that."
}
]
}
Issue.notes_lens
~> Lens.idx(Enum.find_index(issue.notes, &(&1.uuid == "123")))
~> Note.text_lens
|> Focus.set(issue, "New Text")
# %Issue{notes: [%Note{text: "New Text", uuid: "123"}, %Note{text: "It’s like that.", uuid: "456"}]}
OK, got it. That sounds good.
I know another API which might be interesting for this use case, too.
defmodule Commands.AddNote do
defstruct [
:issue_uuid,
:note_uuid,
:text,
]
use ExConstructor
end
All this does is create a new/1 function, populating the struct from keywords, maps, etc.
Maybe that makes it more obvious that the thing is still a plain struct and not overly instrusive.
Your custom function will get :get_and_update if it is used with get_and_update_in or update_in - for get_in it will be :get.
what kind of “next” function we’re looking at
next is simply the function you call with the old value that you just retrieved out of the data. For example:
def text_access(:get, data, next) do
text = Map.get(data, :text, @text_default)
next.(text)
end
text_access expects to be handed a %Note{} for data, it then proceeds to
:text key (hence the function name text_access) andget_and_update_in or update_in.next because that is value that the rest of the navigation retrieved and/or updated.what structure the expected return value should conform to
In general for
:get next will return a single value - the value that the all stages of the navigation found:get_and_update next will return a tuple of {replaced_values, updated_structure} that corresponds to that deeper level of navigation - so your level has to build that tuple up to this level of navigation.:get_and_update calling next under Kernel.pop_in/2 may get a :pop instead of a tuple. This indicates that :get_and_update’s current value needs to be added to the get values while it’s excluded from the update values.Example:
def text_access(:get_and_update, data, next) do
{old, new} = text_access(:get, data, next)
{old, (Map.put data, :text, new)}
end
def text_access(:get, data, next) do
text = Map.get(data, :text, @text_default)
next.(text)
end
text_access(:get_and_update, _, _) gets an {old, new} tuple from next. old would contain the text before the update (this is the “get” portion) while new contains the text that needs to replace it (the “update” portion). So we simply return a new tuple where we leave old “as is”, while we create a new %Note{} with the new text - and we use that %Note{} as the “new” portion in our return tuple.
The {old, new} tuple should make it clear why for get_and_update_in you need to supply a function that returns that tuple:
Kernel.get_and_update_in(note, [text_access],&({&1,"Desc: " <> &1}))}
The tricker bit is
Kernel.update_in(note, [text_access], &("Desc: " <> &1))}
update_in still uses the :get_and_update logic but the return value from your supplied update function (which only gives us the new value) is decorated to something like {:nil, new} so that it can propagate properly through the :get_and_update logic.
I think your understanding fixates too much on “finding” something. For updates you are transforming the entire structure replacing that bit that needs replacing. For retrieval you are simply “navigating” the existing structure to the piece of data that you are interested in.
Edit: Added :get_and_update dealing with a :pop returned from next for Kernel.pop_in/2.
Thanks for the extensive explanation. Yes, that’s mostly what I figured as well in the meantime. Out of curiosity, did you find my code examples for the :get_and_update case reasonable?
FYI - 1.6 introduced Access.filter/1 so this should now work works:
updated = update_in(data, [
:notes,
Access.filter(&(&1.id == 1)),
Access.key(:sections),
Access.filter(&(&1.id == 2))
],
fn note ->
%{note | text: "UPDATE"}
end)
© Copyright Elixir Forum | Terms | Privacy & Cookies
