Iterating through weird collection

Questions / Help
1

Friends,

As I’ve mentioned before, I’m still a newbie to Elixir world, although a very excited newbie :wink:

I’m building a script which I’m supposed to iterate through my S3 buckets at Amazon. I’ve already managed to get the list and transform it into a collection, by using Quinn lib, which is pretty nice.

Now I’m having quite an ordeal to get to pattern_match this generated collection. I just can’t seem to get passed that array.

I need to extract the buckets names into an array and later on iterate over them.

Do you guys know how I get passed it? All the attempts I did - by using the documentation along side with me - failed. I didn’t really find something that would kind of match that seemingly weird structure.

Here’s my collection:

[%{attr: [], name: :Owner, value: [%{attr: [], name: :ID, value: 
["bhjrhrfdfh5435h 345h35j3h5j35h3jh53j2h53h53255h2"]}, 
%{attr: [], name: :DisplayName, value: ["xxxxx"]}]}, 
%{attr: [], name: :Buckets, value: 
[%{attr: [], name: :Bucket, value: [
%{attr: [], name: :Name, value: ["consbr-files"]}, 
%{attr: [], name: :CreationDate, value: ["2013-11-20T20:04:25.000Z"]}]}, %{attr: [], name: :Bucket, value: [%{attr: [], name: :Name, value: ["consbr-files-develop"]}, 
%{attr: [], name: :CreationDate, value: ["2015-05-27T00:32:24.000Z"]}]}, %{attr: [], name: :Bucket, value: [%{attr: [], name: :Name, value: ["consbr-files-staging"]}, 
%{attr: [], name: :CreationDate, value: ["2015-05-26T20:48:04.000Z"]}]}, %{attr: [], name: :Bucket, value: 
[%{attr: [], name: :Name, value: ["consbr-mysql-backup"]}, %{attr: [], name: :CreationDate, value: ["2016-05-09T22:49:37.000Z"]}]}]}]

I’d like to retrieve something like so:

buckets = ["consbr-files", "consbr-files-develop", "consbr-files-staging", "consbr-mysql-backup"]

Any ideas?

2

There is a way to clean this up but

 |> Enum.at(1) 
 |> Map.get(:value) 
 |> Enum.map(&(&1[:value] |>hd |> Map.get(:value) |>hd))
2 Likes
3

Given that chunk of data bound to the binding data, then could do this too:

import Access
get_in(data, [all, :value, all, :value, at(0)]) |> tl |> hd |> get_in([all, :value])

Although a normal for expression might be the most readable. ^.^

iex> for %{value: a} <- data,
...>     %{value: b} <- a,
...>     %{name: :Name, value: [c]} <- b,
...>     do: c
["consbr-files", "consbr-files-develop", "consbr-files-staging",
 "consbr-mysql-backup"]
6 Likes
4

Thank you @andre1sk and @OvermindDL1 for your help. Both options helped me a lot and taught me a great deal about dealing with these collections!

I was firstly trying to get passed that array, by surrounding both the pattern and the match with a [ ], but it wouldn’t do. That Enum.at(1) was pretty obvious but I just didn’t see it.

The rest of both options, took me a while to understand it, pretty much like RegEx, lol!

Thanks a lot

2 Likes
5

Might as well be more selective to begin with:

iex> for %{name: :Buckets, value: bl} <- data,  # get bucket list from all :Buckets maps found in data list                             
...>     %{value: vl} <- bl,                    # get value list from each bucket in each bucket list
...>     %{name: :Name, value: [name]} <- vl,   # extract "name" from single item list on any :Name map found in each value list
...>     do: name
["consbr-files", "consbr-files-develop", "consbr-files-staging", "consbr-mysql-backup"]

The other approaches make assumptions about which position the :Buckets and :Name map occupy within the containing list which may not be a good idea considering that they identify themselves with an explicit name key-value.

2 Likes
6

That’s a very good point about these data coming from S3 @peerreynders, thank you for pointing out. Also, your solution/approach taught me quite a deal about the way we should handle structures of data. Double win! :wink:

Thank you again!

7

Entirely yep, I just created the most simple one to fulfill the given dataset, but if more entries could exist then you definitely want better refinement. :slight_smile:

1 Like
8

It [the structure] can definitely change. So, yes, definitely using that last approach. Thank you again!

9

[quote=“cfernandomaciel, post:6, topic:2686”]
Also, your solution/approach taught me quite a deal about the way we should handle structures of data.[/quote]

Errr … the comprehension was entirely @OvermindDL1’s idea. My initial pass was along the lines of @andre1sk’s approach - but because I was using Enum.filter/2 instead of Kernel.hd/1 it looked quite kludgy and pedestrian. Thinking about it a bit longer I realized that I was even making the unnecessary assumption that the list of values in :Buckets only contained :Bucket items - which can be easily rectified:

iex> for %{name: :Buckets, value: bl} <- data,                             
...>     %{name: :Bucket, value: vl} <- bl,
...>     %{name: :Name, value: [name]} <- vl, 
...>     do: name
["consbr-files", "consbr-files-develop", "consbr-files-staging", "consbr-mysql-backup"]

Personally I haven’t used comprehensions enough to have “goto” familiarity with them yet but in this case the comprehension seems to be the clearest solution.

I realize that and the Kernel.get_in/2 approach looks interesting too. The possibilities of passing a function as a key looks interesting but the documentation is a bit terse on that particular topic.

2 Likes
10

Sounds like you have an idea for a PR. ^.^

2 Likes