Pipe to Complex Map

anthonyl

I need to do the following:

Read data from file
Parse custom syntax
Write result to a map

I have the following code:

File.stream!(load_path,[:read, :utf8]) #Read File
    |> Stream.map(&String.trim(&1)) #Trim Each Element
    |> Stream.map(&String.downcase(&1)) #Convert to Lowercase
    |> Stream.map(&String.split(&1, [":", ","])) #Split at Delimiters
    |> Stream.map(&Map.put(%{}, create_index_name(hd(&1)), tl(&1))) #Write to Map
    |> Enum.map(&IO.inspect(&1)) #Enumerate through stream

Which outputs a list with map entries in it like so:

[
  %{":jhb_team_names" => ["hawks", "maulers", "eagles", wolves"]},
  %{":dbn_team_names" => ["sqeaks", "bunfighters"]}
]

I need it to output something like:

  %{":jhb_team_names" => ["hawks", "maulers", "eagles", wolves"],
  	:dbn_team_names" => ["sqeaks", "bunfighters"]}

i.e. a single map, which can be simply accessed with a key => value lookup.

This should be simple to do, but my newness to Elixir is stumping me.

Any suggestions on how this can be accomplished?

6 comments

#pipeline

0 2528 6

2019-12-01 22:25:36 UTC

Marked As Solved

Show Best Posts
Show All (oldest first)
Show All (newest first)

dimitarvp

You’ll have to modify part of the code to read the data from a file but otherwise it works. Key insight: use Enum.reduce to accumulate values into a single return value, instead of making a one-keyed map for each parsed line.

defmodule Parse1 do
  def data(), do: ~S"""
  jhb team names:hawks,maulers,eagles,wolves
  dbn team names:sqeaks,bunfighters
  """

  def stream_string(text) do
    {:ok, io} = StringIO.open(text)
    IO.stream(io, :line)
  end

  def create_index_name(name) do
    ":" <> String.replace(name, " ", "_")
  end

  def test() do
    # Replace first two lines with your file reading code.
    data()
    |> stream_string()
    |> Stream.map(&String.trim(&1))
    |> Stream.map(&String.downcase(&1))
    |> Stream.map(&String.split(&1, [":", ","]))
    |> Enum.reduce(%{}, fn [head | tail], acc ->
      Map.put(acc, create_index_name(head), tail)
    end)
  end
end

Copy-paste that in iex and then run Parse1.test().

Post #3

Also Liked

dimitarvp

Can you post some example source data (and possibly the source of create_index_name)? Also, please format your code blocks with triple backticks, right now it’s almost unreadable.

Post #1

anthonyl

Thx, it works perfectly now!

Also thx for the education to improve my knowledge .

Post #4

gregvaughn

In this case, I wouldn’t even bother with the Stream.map calls. The extra indirection is slowing you down and adding complexity for no real benefit. Thanks to @dimitarvp 's nice sample code, I took it and modified a bit to be more direct.

defmodule Parse2 do
  def data(), do: ~S"""
  jhb team names:hawks,maulers,eagles,wolves
  dbn team names:sqeaks,bunfighters
  """

  def stream_string(text) do
    {:ok, io} = StringIO.open(text)
    IO.stream(io, :line)
  end

  def create_index_name(name) do
    ":" <> String.replace(name, " ", "_")
  end

  def line_to_kv(line) do
    [key | names] =
      line
      |> String.trim()
      |> String.downcase()
      |> String.split([":", ","])

    {create_index_name(key), names}
  end

  def test2() do
    # Replace first two lines with your file reading code.
    data()
    |> stream_string()
    |> Map.new(&line_to_kv/1)
  end
end

Post #6