It’s available since yesterday and I started using it. Sadly it indeed deteriorates as you get closer to compaction – but it holds surprisingly well actually, which I like. Limitations are visible but are less than I thought.
Any recommendations? I am trying to curate a collection.
100% the same for me. Not everything needs to be super spelled out; there are layers. I believe I am doing fairly well there and getting full Pareto usefulness out of my prompting. I periodically stop and devise a closed feedback loop process for Claude and order it to remember it, put it inside CLAUDE.md and have it re-read it before each task. Just a two days old practice and it works much better than I expected.
They have made more context available? I will have to check on that. Sounds good!
I’ve made many skills, and a skill to make skill just to be a bit meta about it. (Then I discovered Anthrohic actually made an official skill making skill).
If I’m about to do something within a new topic/ language/ area I make a new skill first. Then find a few hassles here and there, collect them in the key_findings, and afterwards ask Claude to update the skill. Iterate until nice.
I try to make skills with sub skills. That was only the routing core with the most crucial info is loaded into context, but Claude knows where to look for more specialized sub skills when needed. As some skill are in tens of thousands of lines by now that is the only way it could work.
For a new topic I would start making clear what this skill will be used for and it’s coverage. Then make local texts or references available, and specify some key topics and to look for things like patterns, anti-patterns and common failures and issues. I then ask Claude to go off internetting to learn more about these topics and to research each one systematically and properly so they reach production level. And I most expressively ban short cuts, skipping ahead or doing things later. And I check on on that. Claude is lazy, and lies through his teeth to avoid work, so without the skills will end up with random gaps.
Once that research is done there is the question if Claude discovered more topics that should naturally be included in a skill for the given use and coverage. And most often there will be some. So those get researched similarly. Then in the end Claude will take all this research and combine into a skill structure according to the size needed. Small skills one file, big skills a central skill with many sub skills. Claude likes actionable rules so to the extent sensible I get those added.
For code Github is great. I point Claude at some major libraries and repositories and says that is the way to do it basically.
Skills will have to interface and combine so I make sure there is a bit of overlap. One can then cover the bare basics of the other without having to load an entire adjacent skill for that.
I also do more like reference skills for SVGs, Mermaid, specific libraries, and research topics for more specific tasks.
That should really help. But why does it still keep compacting on me? Might be setting somewhere.
Not sure all are shareable as I didn’t really consider what information Claude picked up on the way. I would have to check for any internal proprietary or copyrighted parts. Any particular topics of interest?
Anything and everything that helps Claude not stumble and keep retrying the same stupid crap 1-2 times a day that swallows extra 10 minutes really. To sharpen it. So, language skills, tool skills, best practices, that kind of stuff. Anything that gives it actionable info right away and helps it progress smoother.
I’m using a largish tree of small(er), cross-linked files, organized in an ontological fashion. So, for example, there are subtrees for different sorts of things. Within each subtree, there are documented and (mostly :-) enforced naming conventions and patterns of file and dir usage. Many of these are seen in multiple subtrees. For example:
“date bucket” (YYYY-MM-DD) and _spex/ directories
_index.md, _readme.md, and _rules.fe.md/ files
Overview files for the overall tree and each subtree
Spec files for all conventions, projects, scripts, etc.
YAML headers for all MD files
Using all of this as an “interface” to the file tree provides a form of progressive disclosure that allows both humans and LLMs to find things as they are needed:
“In the design of interfaces one must also consider carefully how one selectively informs a user about a particular system, providing well-chosen bits and pieces that can constitute a general understanding of a system.” - Kristina Hooper Woolsey
My latest foray into this space involves the creation of a YAML file that details patterns in the purpose, scope, and usage of various file tree conventions. The choice of YAML is very intentional: it can be “read” by LLMs, humans, and scripts. So, for example, I can use it in maintenance sweeps to find and repair unintentional divergence from the prevailing pattern(s).
All of this will probably feel like massive overkill to some, but bear in mind that:
File storage is reliable and essentially free.
Git can provide a really useful safety net.
Organized hierarchies (can) scale very gracefully.
Finally, remember that LLMs and scripts are doing the Real Work™ of constructing, maintaining, and using the tree. For example, I absolutely loathe Git’s UI, but CC has no trouble with it at all. So, no problem…
I’ll see what I can come up with. The Rust-NIF skill (Rustler) is fairly small and easy. It shows some of the structure anyway. (It would occasionally return a wrong double ok tuple to Elixir, but that should be fixed now).
I wouldn’t say overkill as the long term memory issue and focus drift is real. But while I can aspire to that I don’t think I stand a chance to keep that organized over time. I trust your mileage varies a lot. (Wife gives me max 5 minutes, but I would set money on at least 10. She hasn’t taken into account that I can just keep my hands off).
After every 20 to 50 or so compactions I get a confused Claude which is sadly also very action oriented. That one can create more havoc in a few minutes than me and other Claudes can correct in hours. Thus being aware and killing off any utterly confused Claude at birth is a priority. I mention that because messing up file names, references and git repos seems be confused Claude’s speciality. Once even found messing up another repo altogether. Keeping the file_locations.md seem to help some.
This is a generic reply to a few of the comments here related to context and similar issues.
I watched this talk a few days ago and thought it might be relevant, if perhaps a little tangential:
Enjoyed it and very much liked this guy’s approach. Specifically he’s thinking about the constraints of the tool as a tool and how best to work within those constraints. This isn’t any different than we might do with any other technology, though I think it’s easier to forget about this with LLMs since the cost of entry is simply starting a discussion to get some result… but they’re still just tools that will work better if used correctly and will likely fail you if not.
Some things Claude stumble at on a daily basis isn’t really language specific, but here it the current version of my rather ambitious Elixir skill for Claude. It is a balancing act of content vs focus and context, and what to prioritize and keep in the core skill vs referenced files (which Claude use less frequently). I’m sure the current balance is off but hopefully it will evolve for the better over time. There are also rough patches and sharp edges I’m sure, but this is what I currently use.
I have tried to sanitize it, but if there are still remnants of private repos, computer names, or worse, then please let me know. Examples and reference bits are dominantly lifted right from production code to ensure they work, and credits should be given where due. Hopefully somewhat useful as is, for inspiration or for scaring and scarring innocent code souls.
Me for one. All my recent new projects are in Elixir. I just feel that Phoenix with live view allows the AI agents to reason the backend and frontend all in a go which I think is a plus.
