It is mostly an implementation concern that you should generally not worry about but I am glad to provide more information:
You can think that every combinator in NimbleParsec compiles to a function. Therefore, if you had something like this:
ascii_char([?a..?z]) |> ascii_char([?0..?9])
We would compile it down to this (simplified):
defp node_1(<<x, rest::binary>>) when x in ?a..?z, do: node_2(rest)
defp node_1(_), do: :error
defp node_2(<<x, rest::binary>>) when x in ?0..?9, do: node_2(rest)
defp node_2(_), do: :error
defp node_3(rest), do: {:ok, rest}
However this approach would be expensive both in terms of compilation and at runtime. So we compile it to this:
defp node_1(<<x, y, rest::binary>>) when x in ?a..?z and y in ?0..?9, do: node_3(rest)
defp node_1(_), do: :error
defp node_3(rest), do: {:ok, rest}
You can see “node_1” and “node_2” were fused into the same pattern. The downside of this approach is that if you fail on “node_2”, because they were fused, the pattern reports it failed at “node_1”.
But this should not affect the parsing behaviour, it only affects the reported lines in case of errors.
The first offset is the offset on the current line. The second is the offset since the beginning of the binary. The offset will be the column if you are parsing ascii. However, if you are parsing something with UTF-8 support, then in case of errors you will have to build the column using the line and offset. The algorithm would go like this:
- Get the original input
- Find the nth line given by
line - Now get the offset bytes for that line and call
String.length(binary_part(line_str, 0, offset))
We can probably add a helper function to do this in NimbleParsec.
So as a final parting advice, I think that starting with a hand-rolled combinator is probably a sensible choice which will give you a lot of control and flexibility, while the code should still be readable. As soon as the grammar becomes more complex, consider doing two-phase parsing, with the first pass powered by e.g. nimble.