🍵 Matcha - first-class match specifications for Elixir

Your Libraries & OS Mentoring Libraries
, , ,
1

Officially announcing Matcha!

Matcha

Available on GitHub and hex.pm, Matcha is a library for composing and using match specifications in Elixir. It’s intended to help you perform really fast, selective :ets queries, and refine how you can study the function calls in your running programs.

Synopsis

A really powerful but difficult-to-use feature of the BEAM VM are match specifications. I’ve wrapped them with Matcha to make them much more accessible to Elixir programmers, and am working hard to have great documentation on how to use them where they shine:

The Problem

The thing Matcha works to solve: match specifications are already kind of a a chore to use from erlang code. This is compounded when trying to use them from Elixir, since they really just encode an informal erlang AST. Alchemists have to be familiar with erlang syntax and do a lot of mental context switching to tap into their benefits.

Additionally, the APIs and documentation around how to build and use them in :ets and tracing is very dense. Some great guides exist out there, but I want to marry an easier-to-use-API with rich, first-class Elixir documentation and livebooks to make these topics more approachable.

The Solution

I get into the why, how, and when of match specifications as they exist today in my ElixirConf 2022 talk. I also tease a library to help with them—and have been teasing it throughout its several years of intermittent development—and here we finally are.

Matcha employs Elixir’s macro system and compiler to convert familiar Elixir pattern matches into this oblique format, with rich compile-time validation, and exposes some nicer APIs for working with the resulting match specs.

The Future

In the holiday work lull, I’m able to find a little more time to pick up development again. My hope is to treat this thread as a bit of a devlog for now, and a changelog after a v0.2.0 release with a more stable API. Hopefully I can entice some of you to check things out, and even help me improve it—especially the documentation.

Follow this thread, or the forum’s #matcha topic, to stay abreast of further developments!

37 Likes
2

https://hexdocs.pm/matcha/usage.html#content
https://hexdocs.pm/matcha/usage/filtering-and-mapping.html#content

this is broken.

3

Fixed, thanks!

4

Great project!

5

By the way, is there any way to combine two specs together? Or put one spec into the other?

1 Like
6

There’s a bit of discussion around that on GH: Support compile-time pattern / spec composition · Issue #23 · christhekeele/matcha · GitHub, but it wanders off in other directions pretty quickly.

In short, I think it’s a good idea, and want to add it! The main thing I’m waiting on is understanding a few more people’s usecases before figuring out how it’d work. If you wanted to contribute to that issue’s discussion, it’d be very helpful!

1 Like
7

I just played around with this library, love it so far :slight_smile: Not so intimidating to do efficient queries to ETS anymore :slight_smile:

1 Like
8

I’ve just added a helper to latest to merge matchspecs. It comes with a few caveats, but should be useful for composing matches. It’ll be available next release!

2 Likes
9

I meant something like

merge(spec(do: [head | _] -> head), spec(do: {left, right} -> left + right))

To result in

spec(do: [{left, right} | _] -> left + right)
10

https://hexdocs.pm/matcha/usage/tables.html#content
https://hexdocs.pm/matcha/usage/filtering-and-mapping.html#content
https://hexdocs.pm/matcha/usage/tracing.html#content

broken again :stuck_out_tongue:

11

This seems pretty ambiguous, no? It’s not clear why the merge would know to replace the head binding specifically. What if there are two bindings?

It seems like, to get what you want, you’d need to be able to specify “higher order” specs of some kind that explicitly declare the “vars” or whatever that you’re replacing.

var_spec([head], do: [head | _] -> head)

This seems somewhat complex. I wonder if there is a better approach that might allow “nesting” but delegate composition to regular Elixir functions.

s1 = spec(do: {left, right} -> left + right})

spec bind: [head: s1] do
  [head | _] -> head
end
12

I have a library called Pathex which does this kind of nesting. But I just was curious if it is possible with match specs.

For multiple arguments I’d prefer to have the same behaviour. For example,

spec1 = spec(do: [head | tail] -> head + tail)
spec2 = spec(do: {left, right} -> left * right)

merge(spec1, spec2)
# Results in
spec(do: [{head_left, head_right} | {tail_left, tail_right}] -> (head_left * head_right) + (tail_left * tail_right))
13

Yep, this is my intuition about such a feature; I’m not aiming at supporting it mostly because that’s beyond the scope of normal matchspec usage.

The behaviour here seems rather undefined, I can think of dozens of ways to approach embedding complicated matchspecs in each other.

1 Like
14

Hmm. Where are you finding these dead links, in particular? I moved the guides and ExDoc doesn’t really support redirects; but I though any internal links were corrected with my last fix.

15

image
image1200Ă—660 44.3 KB

guides section has 3 links. all broken.

16

Ahh whoops, thanks again so much for reporting. Released v0.1.7 to fix just now.

17

Very cool to see this @christhekeele!

1 Like
18

Thanks for the help with the tracing docs, @Onor.io! Promise I’m getting to more of your notes…

19

DEVLOG.md 2023-03-24

It’s been a while! I thought I’d provide few updates, and talk about the roadmap a little.

Updates on available the latest branch

I’ve not yet cut out v0.1.8, as I want to finish up two bugfixes first. However:

Features

  • Mirror OTP 25’s support for binary_part/2, binary_part/3, byte_size/1 in match specs
  • Got the OTP team to support ceil/2, floor/3, is_boolean/1, is_function/2, tuple_size/1 in match specs, landing in the upcoming OTP 26 release (:bangbang:)
    • already added support to Matcha for when OTP 26 is released
    • this should mean that Matcha supports all guard-safe functions in match specs written in both Elixir and Erlang!
      • except erlang’s is_record/2, which Elixir has a work-around for in OTP 26, and is a longstanding issue in erlang

Docs

  • Started livebook guides and cheatsheets for adopting Matcha
    • They are intended to be a “converting my project’s match specs to use Matcha” tutorial
    • They’re still in progress as I finalize the “high level” APIs, but early feedback is welcome
  • Added a CONTRIBUTORS.md
  • Many more functions documented

Fixes

  • Prevent Matcha from emitting warnings when not using :mnesia
  • Fix remaining issues compiling Kernel.and/2 and Kernel.or/2 when used in match spec bodies
  • Fix remaining issues compiling Kernel.is_exception/{1,2} and Kernel.is_struct/{1,2} when used in match spec bodies
  • This is… pretty much all of the known issues with the compiler resolved, except for the aforementioned remaining two I’m blocking a release on.
    • Matcha has a 1:2 code/test LoC ratio, and this is mostly centered around discovering edge cases in the compiler today, so I’m feeling pretty darn good about it!

Tests

  • Many more codepaths tested (mostly around edge cases in the compiler to discover resolved bugs)

Roadmap

v0.1.8

  • There are those two known edge-cases with spec compilation I intend to address before releasing the above progress.

v0.2.0

  • This release is when I’m declaring Matcha “ready to use”!
  • The main obstacle is fleshing out documentation. I’ve spent more time on guides than module/fn docs, and it shows.
    • one of Matcha’s most ambitious goals is explaining how/when to use match specs in an approachable fashion, so I’m happy to dwell on this.
  • I’ve also spent more time on the Elixir → ms compiler than higher-level APIs to use specs built by the compiler, so tests/documentation need to be fleshed out as these settle.

v0.3.0

  • I intend to rework Matcha’s tracing APIs to support even more use-cases in this release
  • I will end up ditching the :recon dependency for a custom implementation, for a few reasons
    • I’d like to keep Matcha dependency-less
    • :recon only supports tracing function calls safely, I’d like to support tracing send/receive events as well
    • I intend to apply :recon’s safety heuristics to these things other than function calls, so will ape a lot of the great work done there

v1.0.0

  • This is still the release where any breaking change will imply a major version bump
    • I only anticipate hesitating to publish this post v0.3.0 if the high-level trace and table APIs prove to need a little more work post-release
  • I want full documentation/test/typespec coverage before I make this release

That’s the hot tea :tea: on Matcha, thanks for reading!

9 Likes
20

Just released:

:tea: Matcha v0.1.10

  • Fixes remaining known issues with map literals in matchspec bodies
    • Forbids map update syntax correctly
  • Fixes remaining known issues with invalid remote calls in matchspec bodies

This is just a small release as I warm up to working on the project again for a bit.

Up next

I have only one failing test case remaining I want to finish to consider the compiler part of the project complete! It requires rethinking how I’ve been interacting with the underlying Elixir compiler though—defguard macros defined in Elixir stdlib like Record.is_record/1 and Integer.is_odd/1 aren’t expanding correctly.

With that and some documentation work, I’m hoping to get that released in v0.2.0 soon!

4 Likes