I have not done a ton of string parsing. I could be parsing potentially A LOT of these. Would appreciate any ideas or direction here!
Here is the string:
"START RequestId: 4d0ff57e-4022-4bfd-8689-a69e39f80f69 Version: $LATEST\n2020-02-19T17:32:52.353Z\t4d0ff57e-4022-4bfd-8689-a69e39f80f69\tINFO\tGetting metadata\n2020-02-19T17:32:52.364Z\t4d0ff57e-4022-4bfd-8689-a69e39f80f69\tINFO\tGetting projects\n2020-02-19T17:32:52.401Z\t4d0ff57e-4022-4bfd-8689-a69e39f80f69\tINFO\tGetting Logflare sources\nEND RequestId: 4d0ff57e-4022-4bfd-8689-a69e39f80f69\nREPORT RequestId: 4d0ff57e-4022-4bfd-8689-a69e39f80f69\tDuration: 174.83 ms\tBilled Duration: 200 ms\tMemory Size: 1024 MB\tMax Memory Used: 84 MB\t\n"
I would like to turn it into this:
%{
request_id: "4d0ff57e-4022-4bfd-8689-a69e39f80f69",
lines: [
%{timestamp: "2020-02-19T17:32:52.353Z", level: "INFO", message: "Getting metadata"},
%{timestamp: "2020-02-19T17:32:52.364Z", level: "INFO", message: "Getting projects"},
%{
timestamp: "2020-02-19T17:32:52.401Z",
level: "INFO",
message: "Getting Logflare sources"
}
],
report: %{
"billed_duration_ms" => 200,
"duration_ms" => 175,
"max_memory_used_mb" => 84,
"memory_size_mb" => 1024
}
}
Here is what I have that is working (with some caveats):
def parse_lambda_message(message) do
message = String.split(message, "\n")
report =
message
|> Enum.find(fn x -> String.contains?(x, "REPORT") end)
|> parse_lambda_report()
request_id =
message
|> Enum.find(fn x -> String.contains?(x, "START") end)
|> parse_request_id()
lines =
message
|> Enum.filter(fn x -> !String.match?(x, ~r"START|REPORT|END") end)
|> parse_lambda_lines()
%{request_id: request_id, report: report, lines: lines}
end
def parse_lambda_lines(lines) do
Enum.map(lines, fn x -> parse_lambda_line(x) end)
end
def parse_lambda_line(line) do
line = String.split(line, "\t")
timestamp = Enum.at(line, 0)
level = Enum.at(line, 2)
message = Enum.at(line, 3)
%{timestamp: timestamp, level: level, message: message}
end
def parse_request_id(start) do
String.split(start, " ")
|> Enum.at(2)
end
def parse_lambda_report(report) do
report
|> String.split("\t")
|> Enum.drop_while(fn x -> String.contains?(x, "RequestId:") == true end)
|> Enum.map(fn x ->
case String.split(x, ":", trim: true) do
[k, v] ->
[v, kind] =
String.trim(v)
|> String.split(" ")
key =
"#{k}_#{kind}"
|> String.downcase()
|> String.replace(" ", "_")
value =
case Float.parse(v) do
{float, _rem} -> Kernel.round(float)
:error -> v
end
{key, value}
_ ->
{"key", "value"}
end
end)
|> Enum.into(%{})
|> Map.drop(["key"])
|> Map.put("parse_status", "success")
end
Caveats:
- I have a
""
from splitting the main string (String.split(message, "\n"
) that I canāt seem to match on (got get rid of it) for some reason. - A
line
could be anything fromconsole.log
(javascript, or console.whatever in some other language) so I could have\n
s which can exist in there. What I really need to do is split on a timestamp and then parse the lines from that.