Ah true true, that is something that needs to be made more simple. What would your recommendation be for grabbing the first grapheme from a binary and return that and the rest of the untouched binary?
Primarily I’ve always baked it into the AST, it’s a pattern in almost every compiler (though they often do it via stream ranges, which is not a pattern Elixir has built in but you could emulate it easily enough with sub-binary though that prevents accessing outside of it, which is occasionally useful though not often).
This is often a thing in other parsers, not really been an issue for me in my Elixir parsing, but definitely in other languages you have only a one-shot iterator, which the standard one-shot input iterator is all that’s required for C++'s Spirit as long as you wrap it with a buffered iterator that only buffers enough to allow for rollbacks for alternatives and so forth where points like expect and a few others flush it back since you know you can’t go back any further, that’s a big thing about expect too that I didn’t touch on here as it’s not relevant for Elixir. A ‘Stream’ supporting parser would be interesting and it was planned to be supported in Parserx, but I just hadn’t had a reason to use it yet (though I can definitely see a lot of reasons).
Yeah this would be highly useful, I intended to but never got around to implementing this in ExSpirit and it would be SO useful when building grammars.
Thoughts ?