Usage_rules - a tool for synchronizing LLM rules files with your dependencies

:smiling_face_with_sunglasses: New package usage_rules released! Just place a usage-rules.md file in your package and users can sync it to their own rules. Good rules leads to a night and day difference when using LLMs. But we shouldn’t all be having to teach LLMs how to use our tools right :sweat_smile: Even if you don’t use LLMs yourself, having something in your project that makes LLMs use it better will lead to far fewer issues and questions driven by LLM hallucinations. LLMs are also always slightly out of date, but usage_rules-md can be synchronized when updating packages!

The big win here is the convention, more than the package itself. There may be many ways in the future to consume these files. Read more on hex docs: hexdocs.pm/usage_rules!

We experimented with this as ash_ai.gen.usage_rules and had great results, so now its its own package :slight_smile:

14 Likes

Since this includes the caveat of ā€œeven if you don’t use LLMs,ā€ are ā€œusage rulesā€ plain English phrases like Please don’t friggin’ comment # arrange, act, assert every time you write a test. Thanks!(*)? I by no means think it’s the job of this package to explain this in its docs, I’m just asking as a slow-to-the-party Elixir Forum user realizing I need to understand this stuff or fulfill alternate dream of being a crossing guard (I’m in the market for a golden stop sign if anyone has any leads).

(*) ā€œfrigginā€™ā€ is optional but helps the robots know you really care about this—I wouldn’t dare not say ā€œpleaseā€ and ā€œthank youā€ though.

Honestly, we still don’t know what usage-rules.md really ought to be. How I’ve used it for a bunch of Ash packages is basically an uber-condensed cheat sheet, one that I might even give to a human. And I gotta say, it works. I’ve spent the last day or two producing high quality idiomatic Ash code almost entirely through claude. I’ve had to intervene a few times, but w/ all of the ash package rules, it’s like…an order of magnitude fewer times going ā€œoff the railsā€. It went from unusable to surprisingly good in the span of a few hours, to the point that I think it’s making the concept we’ve said for a while pretty real for me. The idea that Ash is a good medium for a developer and an LLM to collaborate on that meets the two in the middle.

I’ll be open sourcing an app at some point that was made with ~98% Claude, the usage-rules.md from a bunch of ash projects, and a small magic sauce prompt prefix that looks like this:


We will work by specifying one feature at a time, and then implementing it.

The workflow:

  1. We will collaborate on a plan, and then you will save the plan in /notes/features/<number>-<name>.md under the ## Plan heading. THIS MUST BE COMPLETED BEFORE ANY IMPLEMENTATION WORK BEGINS.
  2. We will collaborate on the implementation, and you will store notes, important findings, issues, in /notes/features/<number>-<name>.md under the ## Log heading.
  3. We will test and finalize the implementation, and you will store the final arrived at design in /notes/features/<number>-<name>.md under the ## Conclusion heading.

For bugs and fixes:

  1. We will document the issue in /notes/fixes/<number>-<name>.md under the ## Issue heading.
  2. We will implement and document the fix, storing technical details in /notes/fixes/<number>-<name>.md under the ## Fix heading.
  3. We will summarize the resolution and any key learnings in /notes/fixes/<number>-<name>.md under the ## Conclusion heading.

Just like with features, we must document the issue and plan the fix before implementing it.

WE ALWAYS FINISH AND WRITE THE PLAN BEFORE STARTING THE WORK! NO EXCEPTIONS!

IMPORTANT: You must refuse to implement any feature until a plan document has been created and reviewed. Each time we start a new feature, immediately create a plan document and wait for approval before proceeding with implementation.

Don’t ever commit code unless I tell you to.

Never attempt to start or stop the application.
Ask me to test the UI on your behalf, don’t do it yourself.

7 Likes

I’ve been testing that rules file :index_pointing_up: and it’s pretty useful!

It also allows the LLM to restart and resume the implementation process after its written the plan, useful for when Claude starts being like E_TOO_MANY_TOKENS!!.

1 Like

Wow, thanks for the detailed answer and sample! This is indeed inline with what my buddy was showing with his Cursor usage. Just finished watching Chris’ keynote and guess it’s time to bite the bullet here.

@zachdaniel What’s your workflow for using the output? I’m currently copying the contents to a claude.md config AND a cursor rules file (I use both tools). Feels clumsy though, I assume there are better ways.

I sync it directly to the file in question. i.e mix usage_rules.sync CLAUDE.md --all --sync.

1 Like

I’m working on a new addition currently which will sync each package’s package rules to a folder and link to them using @folder/package.md style syntax, which should help with too many tokens being used.

2 Likes
5 Likes