Enhance my precommit checks

I would add some more Credo checks. There are some custom ones here.

You should also look into the Boundary library for your projects.

I don’t see any Dialyzer here, either. I think that, if you’re spinning up a new PETAL stack application in this day and age, you should be rigorous about specs on everything and running Dialyzer. You should also think long and hard about your philosophy on specs. For my part, I define a custom type for every custom struct in my Elixir codebases and any function that takes/returns any custom struct must use the corresponding custom type in its spec. There are also important questions about how you handle specs for behaviours/callbacks, e.g. whether callbacks should have specs or just implementations or both and if so, how those should relate.

Look into PropEr or StreamData for property-based testing if you haven’t already. And make sure your agent contract enforces TDD discipline, i.e. failing tests first and property tests over normal ones.

I do use Boundary as well! It’s great.

I’ll look at the additional credo checks. That’s a good call out.

When it came to dialyzer I haven’t seen as much benefit since 1.20 with the addition of the compiler checks. It seemed like there was a lot of overlap there. You make strong points though, I’ll re-evaluate.

I’ll check on PropEr and StreamData.

Thank you!

I’m looking into both of these now and I’m curious if there’s a way to use them together? Particularly to cover this case…

PropCheck can store counter-examples: StreamData doesn’t support storing counter-examples in a file (you have to reuse the seed that caused the failure in order to reproduce it).

It seems like automatically storing the failure from the stream as a counter-example would be a good idea (similar to how fuzz testers work when they identify data that causes a failure state).

EDIT: Update. I looked into this a little further. Ash actually already includes StreamData and apparently my AI was already using to generate some property tests that I didn’t know about yet. We’ll be doing more now!

I would love to hear yours/others’ tips for using Boundary. I use it and it is very helpful as a sort of tripwire to force the LLM to tell you it is going to do something not-smart, but I have found that LLMs don’t really get the concept that well and as often as not will try to talk me into working around it as opposed to designing more cleanly. If anyone has some tricks that help them get more out of it that would be interesting to hear.

My experience using Boundary has been pretty seamless, but I also introduced Boundary fairly early in the current codebase I’m working on. Adding Boundary later on to something with some spaghetti already in it is a totally different challenge.

The way I see it, your app’s architecture should make Boundary basically unnecessary. The AI should, when it looks at the codebase, naturally build without breaking the hierarchy. Boundary itself shouldn’t have to really make this clear. Boundary is more like a tripwire, like you said.

So if the LLM keeps trying to work around Boundary, there’s likely something about your app’s architecture suggesting to the agent that it should do that. You ought to take a look at how your app is built and ask if there are places where kinks in the dependency chain are likely to arise.

Also, how consistently are you using it? Do you have Boundary applied evenly across your codebase? The idea is to define one big dependency graph that makes it obvious what depends on what. Does every module in your app have an obvious place in that graph, even if it doesn’t use Boundary?

If your agent keeps adding things to deps and exports then a rule in your agent contract can help. Something like “Do not change deps or exports lists for Boundary without human review”. If there’s anything worth human review in your. codebase, it’s this.