Replacing placeholders in a string from CSV values

Hi,

It’s not quite clear to me what you’re trying to achieve. It might be useful to show a small example of the two inputs (string input and CSV file) and expected output.

Thank you for your response and apologies for not being clear.

Assuming this is my csv:

name,email,balance
Francis Waters,jolir@jalih.mz,$1810.08
Ina Thomas,duzzigip@hizjos.cl,$5639.13
George Cortez,siw@jijol.ma,$222.81
Oscar Nguyen,ov@rici.nu,$7167.56
Wayne Campbell,nad@tuj.jp,$964.14

My text looks something like this:

#name, you owe us #balance this month. Please see your statement here #email

The result for each row should be something like this:

Francis Waters, you owe us $1810.08 this month. Please see your statement here jolir@jalih.mz

Still not sure what you want, is it this?

test "sentences" do
  data = """
  name,email,balance
  Francis Waters,jolir@jalih.mz,$1810.08
  Ina Thomas,duzzigip@hizjos.cl,$5639.13
  George Cortez,siw@jijol.ma,$222.81
  Oscar Nguyen,ov@rici.nu,$7167.56
  Wayne Campbell,nad@tuj.jp,$964.14
  """
  
  rows = String.split(data, "\n")
  Enum.map(rows, fn line -> line |> String.split(",") |> sentence() end)
end


defp sentence([name, email, balance]) do
  "#{name}, you owe us #{balance} this month. Please see your statement here #{email}"
end
defp sentence(_), do: ""

["name, you owe us balance this month. Please see your statement here email",
 "Francis Waters, you owe us $1810.08 this month. Please see your statement here jolir@jalih.mz",
 "Ina Thomas, you owe us $5639.13 this month. Please see your statement here duzzigip@hizjos.cl",
 "George Cortez, you owe us $222.81 this month. Please see your statement here siw@jijol.ma",
 "Oscar Nguyen, you owe us $7167.56 this month. Please see your statement here ov@rici.nu",
 "Wayne Campbell, you owe us $964.14 this month. Please see your statement here nad@tuj.jp",
 ""]

Using Enum.take(1) on a stream to get the headers should be pretty efficient. Using Nimble CSV:

  alias NimbleCSV.RFC4180, as: CSV

 ...
 # Creates a list of header names
 [header] 
  = input_csv_stream 
    |> CSV.parse_stream(skip_headers: false) 
    |> Enum.take(1)

 # Then you can do what you need to with the main CSV. If you are a bit lazy (like me) you can zip
 # the header with the value into a map to allow easy lookup for processing your string template
 data 
  = input_csv_stream
    |> CSV.parse_stream(skip_headers: true)
    |> Stream.map(fn row -> Enum.zip(header, row) |> Enum.into(%{}) end)
    |> Stream.map(fn row_map -> template_wrangling_stuff(template, row_map) end)
    |> Enum.take(10) # or however you want to deal with the output

It takes about 1 sec to load and convert a 1.000.000 lines CSV.
So I’d just stick with the naive implementation for this one (and - having about 1 Billion revenue in that file - just buy a supercomputer to speed things up if need be).

Thank you Sebb for this.

My columns are however dynamic, won’t always be name, email and balance.

OK, more information needed. It’s still not clear what you want to do.

I need to achieve exactly what you did but bearing in mind that the csv titles and content would be dynamic, for example the next template can be exam scores so message would look like this:

hi #name, you have scored #physics in physics, #chemistry in chemistry and #maths in maths

The csv would then look something like this:

name,maths,chemistry,physics
James Ortega,44,14,49
Eula Garrett,26,74,16
Emily Marsh,47,22,73
Tommy Jenkins,37,54,55
Ronnie Nash,54,04,31

I hacked that into the first solution, so you can get an idea.

def load(csv) do
  [header | rows] = String.split(csv, "\n")
  fun = &sentence(&1, header)
  Enum.map(rows, fn line -> line |> String.split(",") |> fun.() end)
end

defp sentence([name, email, balance], "name,email,balance") do
  "#{name}, you owe us #{balance} this month. Please see your statement here #{email}"
end

defp sentence([other, header], "other,header") do
  "#{other}, ... #{header} ..."
end

defp sentence(_, _), do: ""

EDIT this wont work: If this does what you want, I’d create a lookup header => template instead of the multiple functions.

template =
      "hi #name, you have scored #physics in physics, #chemistry in chemistry and #maths in maths"

input = """
    name,maths,chemistry,physics
    James Ortega,44,14,49
    Eula Garrett,26,74,16
    Emily Marsh,47,22,73
    Tommy Jenkins,37,54,55
    Ronnie Nash,54,04,31
"""

[headers] =
      input
      |> NimbleCSV.RFC4180.parse_string(skip_headers: false)
      |> Enum.take(1)

replacement_hs =
      headers
      |> Enum.map(fn header -> "#" <> header end)

input
|> NimbleCSV.RFC4180.parse_string(skip_headers: true)
|> Enum.map(fn row ->
      Enum.zip_reduce(row, replacement_hs, template, fn col_val, header, acc ->
        String.replace(acc, header, col_val)
    end)
end)

The only reasons to use NimbleCSV here is because sometimes there can be escaped separation characters and if you have a streamable source it’s just easier to get it working. Using @mindok example but with a string I would just map the headers into their replacement tokens ("#some_header") and then brute force the replacements unless it turned out to be slower than acceptable at which maybe compiling a custom regex could be faster.