Pipe to Complex Map

Questions / Help
1

I need to do the following:

  1. Read data from file
  2. Parse custom syntax
  3. Write result to a map

I have the following code:

File.stream!(load_path,[:read, :utf8]) #Read File
    |> Stream.map(&String.trim(&1)) #Trim Each Element
    |> Stream.map(&String.downcase(&1)) #Convert to Lowercase
    |> Stream.map(&String.split(&1, [":", ","])) #Split at Delimiters
    |> Stream.map(&Map.put(%{}, create_index_name(hd(&1)), tl(&1))) #Write to Map
    |> Enum.map(&IO.inspect(&1)) #Enumerate through stream

Which outputs a list with map entries in it like so:

[
  %{":jhb_team_names" => ["hawks", "maulers", "eagles", wolves"]},
  %{":dbn_team_names" => ["sqeaks", "bunfighters"]}
]

I need it to output something like:

  %{":jhb_team_names" => ["hawks", "maulers", "eagles", wolves"],
  	:dbn_team_names" => ["sqeaks", "bunfighters"]}

i.e. a single map, which can be simply accessed with a key => value lookup.

This should be simple to do, but my newness to Elixir is stumping me.

Any suggestions on how this can be accomplished?

2

Can you post some example source data (and possibly the source of create_index_name)? Also, please format your code blocks with triple backticks, right now it’s almost unreadable.

2 Likes
3

Thx, here goes:

File.stream!(load_path,[:read, :utf8]) #Read File
|> Stream.map(&String.trim(&1)) #Trim Each Element
|> Stream.map(&String.downcase(&1)) #Convert to Lowercase
|> Stream.map(&String.split(&1, [":", “,”])) #Split at Delimiters
|> Stream.map(&Map.put(%{}, create_index_name(hd(&1)), tl(&1))) #Write to Map
|> Enum.map(&IO.inspect(&1)) #Enumerate through stream

And create Index Name:

def create_index_name(name_string) do

    ":"<>String.replace(name_string, " ", "_")

end

The Source file looks like this:

jhb team names:hawks,maulers,eagles,wolves
dbn team names:sqeaks,bunfighters

Hope this clarifies :grinning:.

4

You’ll have to modify part of the code to read the data from a file but otherwise it works. Key insight: use Enum.reduce to accumulate values into a single return value, instead of making a one-keyed map for each parsed line.

defmodule Parse1 do
  def data(), do: ~S"""
  jhb team names:hawks,maulers,eagles,wolves
  dbn team names:sqeaks,bunfighters
  """

  def stream_string(text) do
    {:ok, io} = StringIO.open(text)
    IO.stream(io, :line)
  end

  def create_index_name(name) do
    ":" <> String.replace(name, " ", "_")
  end

  def test() do
    # Replace first two lines with your file reading code.
    data()
    |> stream_string()
    |> Stream.map(&String.trim(&1))
    |> Stream.map(&String.downcase(&1))
    |> Stream.map(&String.split(&1, [":", ","]))
    |> Enum.reduce(%{}, fn [head | tail], acc ->
      Map.put(acc, create_index_name(head), tail)
    end)
  end
end

Copy-paste that in iex and then run Parse1.test().

2 Likes
5

Thx, it works perfectly now!

Also thx for the education to improve my knowledge :grinning:.

1 Like
6

Hi @anthonyl! Welcome!

I saw @dimitarvp answered your question already by asking a little bit more about your problem (good technique). Even though you solved the problem a different way, I thought I’d focus on what you asked first: You had an input in a certain shape (list of maps), you wanted an output in a different shape (one map). Here is a possible solution to the input/output problem:

iex(1)> target = [ %{jhb_team_names: ["hawks", "maulers", "eagles", "wolves"]}, %{dbn_team_names: ["sqeaks", "bunfighters"]}]
[
  %{jhb_team_names: ["hawks", "maulers", "eagles", "wolves"]},
  %{dbn_team_names: ["sqeaks", "bunfighters"]}
]
iex(2)> Enum.reduce(target, %{}, fn map, acc -> Enum.into(map, acc) end)                                                     
%{
  dbn_team_names: ["sqeaks", "bunfighters"],
  jhb_team_names: ["hawks", "maulers", "eagles", "wolves"]
}

Hope this helps you in your Elixir journey! Keep going! :slight_smile:

7

In this case, I wouldn’t even bother with the Stream.map calls. The extra indirection is slowing you down and adding complexity for no real benefit. Thanks to @dimitarvp 's nice sample code, I took it and modified a bit to be more direct.

defmodule Parse2 do
  def data(), do: ~S"""
  jhb team names:hawks,maulers,eagles,wolves
  dbn team names:sqeaks,bunfighters
  """

  def stream_string(text) do
    {:ok, io} = StringIO.open(text)
    IO.stream(io, :line)
  end

  def create_index_name(name) do
    ":" <> String.replace(name, " ", "_")
  end

  def line_to_kv(line) do
    [key | names] =
      line
      |> String.trim()
      |> String.downcase()
      |> String.split([":", ","])

    {create_index_name(key), names}
  end

  def test2() do
    # Replace first two lines with your file reading code.
    data()
    |> stream_string()
    |> Map.new(&line_to_kv/1)
  end
end
1 Like