Enum/Stream pattern matching parser library?

[engineeringdept]

I’ve got a list of text tokens + metadata as structs from a custom tokeniser. I want to match patterns in that list, some of which can be expressed with Elixir pattern matching, others of which will need a custom function to test for.

Some of the rules will need to be akin to simple regular expressions: not this, optional that, no more than three of these, etc.

Is there a library that will let me express a set of rules and match against those?

I found ExSpirit from @OvermindDL1, but I couldn’t see any examples of it being used for list patterns, just text. It looks like NimbleParsec just operates on text too. Should be I looking at something else? Thanks!

[adw632]

Nimbleparsec is parsing context free languages and it is very fast from what I can tell.

This is a very comprehensive thread on Nimbleparsec and Exspirit: NimbleParsec - a simple and fast parser combinator for Elixir

[engineeringdept]

Thanks, but unless I’m missing something, those operate on strings, not lists of structs.

[adw632]

Doesn’t sound like you want a parser at all but some kind of constraint satisfier/solver. Perhaps you get some inspiration from these:

• GitHub - Lakret/csp: Constraint satisfaction solvers in Elixir
• GitHub - Dkendal/aruspex: A configurable constraint solver
• GitHub - bokner/solverl: Erlang/Elixir interface to MiniZinc.

[engineeringdept]

I think I probably just need to write my own combinator parser for this use case. I’m finding this video very helpful: https://www.youtube.com/watch?v=xNzoerDljjo

[adw632]

I do believe that @OvermindDL1 did say ExSpirit can work on non string based streams so you may be able to make it work with structured data.

I can’t really comment on feasibility other than what he wrote here:

@OvermindDL1

That’s actually a pattern that the C++ Spirit supports (it can accept any kind of iterable, not just strings), and my ExSpirit follows the same conventions. The only things that assume strings are the things in the Text module. It would be very trivial to create a, oh, Matcher module or token matching (you can create it in user code with ease, a single function would probably work very well, I think C++'s token parsing uses just a single terminal to handle it all). The main ExSpirit library is agnostic to what is actually being parsed (ignoring a few things like eoi and lexemethat I really should protocolize or at least handle a few erlang-style forms of). But parsing non-strings was entirely in mind when it was designed.

Strangely enough that was in response to a comment by Sasha from the video you linked, as he used the slower combine parser which wasn’t a significant factor for his use case.