This is not a comment on your implementation, but I had never heard of these before and had a look at the original repo and I have to say it gives off some weird vibes. It seems like their main argument is that UUIDv4 is bad if you use a weak source of entropy (yeah, obviously) and that CUID is better because it combines “multiple sources of entropy”.

There are a lot of weird claims about security and other things. The section on k-sortability is particularly bizarre (“cloud databases” store all their data in memory so it doesn’t matter? seriously?).

Anyway, :crypto.strong_rand_bytes() should have sufficient entropy to generate ids, or at least I hope it does! Is there any other functionality that CUID2 offers over the (well-standardized) UUIDv4/7?

Of course, if someone is already using these it’s still good to have a library for them, so definitely don’t take my comments as directed at you

I really don’t mean to be rude here, but I would appreciate it if you could reply in your own words rather than with an LLM.

This entire reply is AI slop. Almost every part of it is completely wrong, and there’s no way for me to know whether it’s because you’re mistaken or because GlazeGPT is hallucinating nonsense.

I had a look at your code because I was curious and it seems to me to be AI generated as well. The implementation is not good. :crypto.strong_rand_bytes/1 is cryptographically secure and provides sufficient entropy.

You do not need to start an agent with a counter (serializing every single id generation via message passing in the process and incurring massive overhead) and then sha256 that with the actual random bytes. That does not add entropy. It’s completely absurd.

You are choosing the first letter with Enum.random(). That function is not crypto-safe, all you’re doing is decreasing the entropy of your ids.

Under no circumstances should anyone ever use this ridiculous id format for a new project, but if you have a legacy project you need to support then you can just generate random bytes with :crypto.strong_rand_bytes/1 and then base36-encode them directly. You do not need to implement any of this additional “entropy” nonsense.

I recommend you update your library to do exactly that.

TL;DR: CUID is designed for a specific kind of solution to problems you may have never had yet.

I know I’m dropping in a bit after the fact, but I didn’t see anyone directly address your questions about the weird vibes given off by the CUID spec. It turns out that the short attention spans and pervasive mediocrity of our field makes it reasonably likely that you wouldn’t have been in a situation where you would need this.

The stilted ranting of the CUID spec doesn’t help, for sure. They do a terrible job explaining what they mean. Then again, once you’ve learned how to solve certain problems, it sometimes becomes very difficult to even discuss it with people who haven’t already learned what you know.

That’s essentially what’s going on with the CUIDv2 spec and its arguments. It turns out that all of their reasoning is sound, but won’t make any sense unless you are writing your application a certain way. And they can no longer conceive of building things any other way.

There are particular assumptions that you find in Phoenix and most MVC-esque frameworks. You typically find them in most HTTP APIs as well. Why that is the case involves… well… a lot of history.

Allow me, if you will, to take you on a strange journey. (Apologies. This will be long.)

A History Lesson

Note that I didn’t say REST APIs. That’s important. What you’re probably used to doing is writing a REST-like (or RESTish?) APIs. Basically, JSON-over-HTTP with a few conventions about methods and maybe status codes. There’s nothing wrong with this. It’s works just fine for 98% of projects.

It turns out that these “conventions” are not at all what REST was originally concerned with. It often surprises people to discover that REST as originally defined is not really the blueprint for making APIs we struggle with today. It was really more of the distillation of the wisdom that came out of designing HTTP itself.

HTTP was born trying to solve a problem that has sometimes been referred to as “anarchic scalability”. Basically, how to build apps and APIs in a world where everything is moving fast and nobody can claim to have absolute authority to dictate how it works.

Out of this, he created this idea of “Hypertext As The Engine Of Application State”—usually referred to by the world’s worst acronym: HATEOAS. REST was a set of principles that grew and blossomed as HTTP was first designed and implemented.

This version of REST has been largely lost in modern API discourse. It’s actually pretty fascinating and well worth learning. You can read the original paper here.

That’s only half of the story, though. The gory details of how it got lost to history are a whole tale of their own. If you really care, you can read more about it here.

(You see this a lot, actually. It’s pretty common. For example, Object Oriented Programming as conceived by Alan Kay is very different than what people have turned it into. Ironically, his version of OOP is closer to Elixir than C++. Though maybe that’s not so surprising because Erlang had strong influence from Smalltalk. But, I digress…)

If you dig into the original REST paper, you’re going to find out that it hardly even mentions most of the things you think of as REST.

Some “Modern” Assumptions

Coming back to the problem at hand, consider the following assumptions you probably have about how to design an HTTP API:

• IDs are generated on (and arbitrated by) the server (probably the database itself).
• You’re probably going to have to strap some sort of idempotency keys onto your API to make it resilient in the face of transport failures.
• Retry and recovery logic may end up being fiendishly complex.
• Caching is only for browsers. APIs use Cache-Control: no-cache liberally.
• Creation requests should only use the POST method.
• Only update requests will use the PUT method.

As it turns out, these assumptions (regardless of how common they are) create problems that don’t need to exist. They are largely ignorant of the benefits or a more classically RESTful approach.

Commonly we end up using HTTP verbs in ways that not only make things ambiguous, they often end up using these methods in ways that clearly conflict with how they’re defined to be used in the HTTP spec.

In doing so, we make applications less reliable, less predictable, more complex, and harder to scale. It doesn’t have to be this way.

CUID v1 and v2 are designed to provide a minor but very important missing piece of an approach that entirely avoids creating any of these problems.

Feel My Pain

Anyone with much experience almost certainly has trauma here. There’s this story that repeats itself every new startup you work for. Every time you try to avoid it. But, like the Classical Greek story of Cassandra, you tragically can only predict the crisis, but never avert it.

It starts when you try to ask for a little forethought before designing an API. There’s a kind of “get it done” personality that insists that “Do it the ‘normal’ way. Everything is fine. There are no decisions to make. Something something Best Practices. Everybody knows this is settled.” Their tragic lack of imagination and woeful inability for introspection thoroughly suppress any chance for innovation and will doom us all.

They’ll immediately roll their eyes when you suggest “This isn’t really that RESTful, maybe we should take a second to actually define some terms. How about we actually design something for a minute?” They shriek “Big Up-Front Design BAD! That’s not agile!”

They insist that REST == CRUD and dismiss any attempt at deeper understanding as “bike-shedding”. Their assumptions are rapidly baked into the codebase. For a time, it seems like it will be fine. They crank out code at a prodigious rate. The KPIs look good.

Then… things break. Who knew the network isn’t perfect? What do you mean it succeeded but we didn’t get the response? Somehow the find the idea to naively implement retries lurking somewhere within their smooth brains.

Oops, now we’re sometimes doing creation or updates twice. Then they’ll generate massive amounts of “defensive programming” code in clients. They’ll generate this web of custom code for each request just so clients can figure out the ID of whatever they may or may not have created.

Invariably their confused and befuddled retry logic will start hammering other services into the ground retrying in tight loops. Here they begin excitedly rambling about circuit breakers and exponential backoff and rate limiting because they heard about it on YouTube or at the coding bootcamp they went to.

They’ll e-mail the team blog posts about a million half-measures. Many of them don’t even fit the problem.

Meanwhile, the SREs are becoming just as restless as your app is RESTless. Just ignore the fact that your dashboards are full of errors that aren’t errors and the useful signals lost in “normal” bursts of errors-that-aren’t-really-errors. Who needs actually meaningful error metrics?

Is this monstrosity generating tons of inconsistent or even incorrect data? We can manually clean up any junk that gets left behind in the melee. What’s a little toil between friends? The SREs now begin to exhibit the “500 yard stare” in Zoom meetings. At least remote work means you don’t have to listen uncomfortably to their anguished sobs (so long as they remember to stay muted).

Every day’s standup starts with “Oh, yeah, we can fix that. All you’ve gotta do is… ” At this point, you can’t really make headway. They’ve created a beast so complicated that nobody can understand it—least of all them. Not that stops them from thinking that they understand it.

Consequently, nobody can make the case for how or why it’s busted. The system develops “behavior”. They blithely coast along with the momentum of someone who has mastered being confidently wrong. They’ve made into more than just a profession, it’s their way of life.

Invariably they start throwing around the word “idempotent”. They double-down and insist on adding yet-another-key to the request. Maybe it’s field in the JSON body. Maybe it’s a header. Could be a query parameter! Regardless, it becomes just more metadata that requires infrastructure to develop, maintain, and scale.

You see, it’s just to prevent consistent updates from corrupting state. Clearly that’s a different, new, and entirely separate problem. It can’t be because we’re taken an oversimplified approach to building an API such that it can’t even make consistent updates. Clearly, one more middleware is all it needs!

Now you’ve got this giant mess where you need to track these new keys consistently; despite the fact that you already have a single ID that you can’t track consistently. The client still has to read tea leaves to divine this elusive object ID for the resources it may or may not have created successfully.

The retries and recovery queries are absolutely hammering your services, now. A million requests, made by an idiot, full of sound and fury, signifying nothing.

Then they have an insight: “The clients can get back the object ID that was in the response they missed if we save the original response and replay it. All you’ve gotta do is save the response under the idempotency key somewhere.”

It turns out that replaying cached responses provides no guarantee that the response even corresponds to the same request. And how long do we save those responses? Nobody knows.

Maybe it’s an hour. That works fine until the fiber between regions goes down for an afternoon. Now all of the hung up requests start retrying. They end up changing things out-of-order multiple times because we didn’t save the cached responses.

Or better yet, their code starts generating duplicate idempotency keys for unrelated requests (because they invented some half-baked scheme for generating the keys). Maybe we’re trusting the client too much here.

Or maybe they’re accidentally sending subsequent requests with the same key, but different parameters, because they vibe coded a bit too deeply and greedily. Now what comes back depends on timing, ordering, and the phase of the moon.

Or maybe caching the whole response takes too much storage. We’ll just reconstruct the response. Oops! Some data was only available at creation time and it’s missing now. Surely the client won’t care if there are subtle differences between original responses and the ones that we’re now synthesizing.

Eventually Mr. Get-It-Done gets all defensive when concurrent updates start to drastically corrupt state. The same guy that complained about “know-it-alls whining about the meaning of REST” start using terms like “phantom reads”, “the last update problem”, and “serializability”. Apparently it’s “pedantic” when you tried to talk about it up-front; but it’s “state-of-the-art” when they do it.

Since they’ve unerringly missed the point for months, they then desperately start trying to layer on more half-measures and hardly considered “fixes”. Let’s bundle multiple requests in a single POST. Or try distributed transactions (because they read a blog post about that a year ago). Maybe they’ll discover “sagas”.

All the while insisting this unmaintainable mess was inevitable. They’re like some kind of software-engineering Thanos… they just snap their fingers and half of your velocity turns to dust.

Eventually they push to do an entire rewrite. Or maybe they want to move to NoSQL (since your relational database is spouting flames from all of the superfluous work they’re piling onto it). They’ll probably end up being promoted for building such a high-tech “solution”. It’s not clear if that’s good because they’re not working directly on the codebase anymore or bad because they’re now a “tech lead” or “architect”.

Can we maybe learn anything from HTTP itself?

Let’s start with some highlights from RFC9110:

• GET
  • §9.2.1: The GET method is “safe”.
  • §9.2.2: They are guaranteed to be idempotent.
  • §9.3.1: Responses are cacheable and update caches on the way through.
• POST
  • §9.2.1:
    • The response POST method is not safe (implied).
  • §9.2.2:
    • POST is not guaranteed idempotent.
    • Unless it is.
    • But you’ve got to know.
    • Good luck.
  • §9.3.3:
    • POST responses are not cacheable.
    • Except when it is, but only if some extra metadata is returned.
    • But not for POST requests; because they’re allowed to be “unsafe”.
    • But, sure, for GET requests… if you want to.
    • Oh, and it doesn’t necessarily create resources, though it might.
    • In fact, it can even create things without explicitly telling you if it wants.
    • For that matter, nothing about a POST guarantees that it’s going to create what you asked it to.
• PUT
  • §9.2.2: PUT requests are guaranteed to be idempotent.
  • §9.3.4:
    • Responses are not cached and invalidate caches on the way through.
    • If they create resources, they MUST tell you.
    • If it returns successfully, you can be sure that the state has been updated to exactly what you requested.
• DELETE
  • §9.2.2: DELETE requests are guaranteed to be idempotent.
  • §9.3.5: Responses are not cachable and invalidate caches on the way through.

Notice a pattern there? POST is the wild west. It has no strong semantics. It’s whatever it wants to be. Nobody in their right mind should use it. If you do, don’t expect anything sane out of it

We can at least use GET, PUT, and DELETE sanely. They’re tightly defined. That’s great. But how do I map four operations to only three methods? And what about retries and double creation and concurrent updates and all of the other problems we ran into?

Be The Change You Want To See In The Database

If only there was a way we could make sure that a retried request (or an update) is operating on the state we expect to be there.

Going back to RFC9110, let’s learn about conditional requests:

• §13.1:
  • There are “preconditions” you can specify to ensure that your state doesn’t get thrashed because other requests (retries, other clients, etc.) changed something out from under you.
• §13.1.2: If-None-Match
  • Want to ensure you’re creating something?
  • Use If-None-Match: * and your PUT won’t turn that create into an update.
  • Suddenly.. you don’t need POST!
• §13.1.4: If-Unmodified-Since
  • What if you’re doing a read-update-write cycle?
  • If there are dueling requests with other clients, your updates might step on each other!
  • But, if you know the date returned when you did the read, you can say to only do the request if it hasn’t changed!
  • Now retries are safe without corruption due to “the last update problem”
• §13.1.1: If-Match
  • Time is kind of imprecise here. Maybe you’re nervous about using modification dates to prevent accidents.
  • If only we could use a hash of the data or something?
  • That’s what the etag is for.
  • Now you can use If-Match: <etag> and you’ll get consistent updates based on the actual content.
  • This even works if retried PUT requests update the timestamp twice without changing the data.
• §15.5.13: 412 Precondition Failed
  • There’s a dedicated 4xx series error code just for this.
  • No longer do I have to figure out if I should return a 400 Bad Request, a 401 Unauthorized, a 403 Forbidden, a 404 Not Found, a 410 Gone, or 418 I’m A Teapot (if you support the HTCPCP extensions from RFC2324).

Well, that’s neat and all; but does it really help that much? It kind of eliminates the need for idempotence keys and eliminates lost updates and the last read problem. But it doesn’t help with the problem we started with: Getting the ID that the server used if we lose the initial response.

Well… do we really need the server to do that for us? Maybe it doesn’t have to? Can we have the client-side just submit one of their own? Then they already have it before the server even gets started.

Trust The Awesomeness

At first, this sounds questionable. What if people intentionally create collisions? How can you keep IDs consistent without arbitrating on the server? How can I trust the client?

Then you start to realize suddenly that client-side IDs would demand significantly less from your database. By itself, it takes a huge load off for no other reason that the server no longer needs to maintain a consistent ID counter. Now that you think about it, could you do sharding without even consulting the server?

Next you realize that you can also use a client-side ID itself as the idempotency key. You can use conditional requests to instruct the server what to do if there’s a conflict. They give you all you’ll need to ensure that you’re manipulating the state you expect to be there.

Not only is this safer, you no longer have the risk of giving a duplicate idempotency key

You also get a clean recovery workflow for clients. Now retries are safe. And if you get “Precondition Failed”, you can do a GET to see if your create succeeded or if there was an intervening update you didn’t expect.

If you only care about success or failure, you don’t even need a GET. You can just do a HEAD and look at the etag. And if you do decide to do a GET you update the cache in-between consistently automatically.

Hey… you get caching in there, too. You can deploy something like Squid and your API will actually work with it, not against it. Bonus points if all of your microservices perform better because there’s a giant, consistent, shared cache sitting in front of them.

Whoa. Now that I look at it, Squid even has facilities for me to make this work across our WAN. It turns out that ICP is about more than clowns (i.e. Inter-Cache Protocol >> Insane Clown Posse).

This improves everything!

UUIDs to the Rescue!

Theoretically, if you have them use UUIDs, then collisions are statistically unlikely.

Of course, there are so many to choose from!:

• UUID v1 ensures no collisions by using timestamps and MAC addresses.
• Nobody talks about UUID v2. It’s like IPv5. We just skip over it.
• UUID v3 lets us throw in a “namespace” and “identifier” to hash for more uniqueness.
• UUID v4 lets us just go with pure randomness.
• UUID v5 lets us use a stronger hash function than UUID v3… progress!
• UUID v6 lets us basically do UUID v1 again, but now they sort by time!
• UUID v7 sort, too; but they’re random like UUID v4.
• UUID v8… well… it can be anything we want!

UUIDs to the Rescue?

Well, okay… it turns out all of those break in one way or another.

• MAC addresses get duplicated all the time in VM clusters and containers.
• Lots of embedded systems reset to a predictable date on bootup until NTP or something kicks in.
• In tests it’s even worse, because now some UUID typesl generate the same UUID in each test because they seed the RNG predictably.
• Or maybe it works but your test are nondeterministic because of time values.
• But then you mock it out and now you broke the rest of the UUID generation, too.
• Even with pure randomness, you’re something stuck with a busted client.

Oof, now that I’m trying to use them, it’s a real pain.

• They can’t be used in some contexts that don’t like leading numbers or hyphens.
• They’re leaking my MAC address.
• They’re leaking when I created them with that timestamp, too.
• Even the sorting is killing my database because it causes current updates to hammer a tiny spot in my BTree index.
• Sorting is also making it easier for someone who might try to cause a collision, too.

So they perform worse, they’re bad for privacy, they might be bad for security, embedded systems cause problems, testing gets weird, and now every client can naively create problems with bad PRNGs.

Let’s Look At Scalable Cloud Databases for Inspiration

Hmmm, MongoDB has an ObjectID() function! What do the docs say it does?

• A timestamp
• Some randomness
• A client-side session counter

This sounds familiar…

So That’s What That CUID Manifesto Was Rambling About!

And now you’ve got CUID:

• Leading letters for easier use in more places.
• Strong randomness.
• Uses client info but hashes it for privacy.
• Salting also gives us some security, too.
• Lack of order helps performance.
• Has that session counter in there, too.

CUID gives you a client-side key that you can generally trust. You’re freed from dealing with idempotency keys. You get consistent updates in the face of concurrent changes. You can prevent double-creation or deletion. The server is no longer a bottleneck for ID creation. You can shard statelessly. You get potentially global caching almost for free.

Maybe the real treasure was the HTTP features we met along the way!

Conclusion

So there you have it. It’s not about the server generating better UUIDs for IDs.

It’s about a system where it doesn’t have to. It’s an entirely different paradigm. It’s about shared-nothing scaling. It’s about correct functioning despite malicious or very broken clients. It’s about keeping your data consistent. It’s about robustness, resilience, and recovery. It’s about security and privacy.

But, mostly, it’s about building the kind of systems that are possible if you’re willing to question the assumptions that everyone else is making.

P.S. Thank you for coming to my TED talk.
P.P.S. No, I didn’t use any AI to generate this. I write like this. Always have—just… em-dashes… everywhere.
P.P.P.S. It’s 4am and I’m very tired. I hope this makes any sense at all.