fireproofsocks

fireproofsocks

I wanted to get a reality check from the wise folks of the forum on the differences and pros/cons in uuid vs. nanoid. UUIDv4 has been my go-to for primary keys for a while, but the nanoid package is garnering some attention (there is an Elixir port). NanoID promises to save on real-estate: a human may be able to type one more easily because a NanoID can be shorter than a UUID (at least on the screen).

Just to make sure I’m getting my facts straight, my understanding is that a UUIDv4 ID is stored as binary data 128 bits long. It is often represented as an alpha-numeric string, e.g. fcfe5f21-8a08-4c9a-9f97-29d2fd6a27b9, but this is just a human-readable view of the underlying binary data. So if I’m doing my math correctly, the usual representation is as 32 characters (discounting the hyphens), each represented by a 4-bit hexadecimal number (0, 1, 2, … a, b, c, d, e, f); 32 x 4 = 128 bits.

NanoIDs, on the other hand, seem to always be strings. So if we store a UUID representation (minus the hyphens, e..g. fcfe5f218a084c9a9f9729d2fd6a27b9) as a literal string, it requires 256 bits because it is represented on disk as thirty-two 8 bit numbers, instead of thirty-two 4 bit hexadecimal numbers – i.e. storing the value as a string requires at least twice the space. I’ve seen this mistake made many times when a database schema uses a TEXT or CHAR column to store UUIDs instead of the native binary format… this mistake can really slow down indexing and queries.

So the question is: couldn’t we just offer a different VIEW on top of the existing UUIDv4? In other words, couldn’t we just represent those 128 bits differently to save screen real-estate? For example, if we choose an alphabet of a-z and digits 0 - 5, we would have 32 characters at our disposal and we could represent a 128 bit UUID using only 4 screen characters, e.g. pf3c. Or if we wanted to expand our alphabet, e.g. to a-z, A-Z, 0-9, plus 2 more characters – that would bring us to 64 characters in our arsenal, and we could represent the 128 bit UUID using only 2 screen characters, e.g. Q3. (This is just another way of saying “base-64 encoding”). Wouldn’t that make for nicer REST URLs? E.g. http://localhost/posts/Q3 instead of http://localhost/posts/fcfe5f21-8a08-4c9a-9f97-29d2fd6a27b9

Am I reasoning about this correctly? It feels like I’m missing something. Am I correct that UUIDv4 requires only 128 bits? So would it be useful to have a package that offered a custom and compact “view” of the UUID data? That way the database and everything else could stick to the tried-and-true UUID generation and support under the hood, but if humans were involved, a shorthand could be used to provide an easy-to-type short-hand of the UUID (e.g. using some base-64 or base-32 scheme). This is more or less the idea behind URL shorteners, it’s just a lot simpler when you only have to represent 128 bits of data.

Am I looking at this the right way? Thanks for any thoughts.

Showing Posts 1 to 10

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

This bit is right, and this is why things like ecto use binary_id with Postgres so that it can be stored “natively” in the DB.

Textual representations are a tricky subject. Some big considerations are “how easy to confuse are adjacent characters”. You generally want to avoid things like 0O00Oo or IlIlIlIl (that’s capital I and lower case l). So usually, particularly when you’re just dealing with transient representation, clarity is more important than succinctness.

As far as whether you can get down to 2 characters, I think that math is off. 2 characters of 64 possibilities is 4032 total permutations. v4 UUIDs have 2^122 possible combinations https://stackoverflow.com/questions/1155008/how-unique-is-uuid.

mayel

mayel

I would also suggest ULID. There’s an ecto type for it here.

kip

kip

ex_cldr Core Team

There are some new UUID formats, UUID 6, 7 and 8 that are quite interesting and solve some of the challenges of UUIDs when it comes to database serialisation (index locality being quite a big one, distributed support too). But they are all still 128 bits, which I think for a primary key is a pretty good choice.

trisolaran

trisolaran

Math is definitely off here. A sequence of 4 characters chosen from an alphabet with size 32 is 32**4 = 1048576 ~1e+6 combinations. A 128-bit UUID has 2**128 ~ 3.4e+38 combinations. Ridiculously far apart.

antoine-duchenet

antoine-duchenet

AFAIK, since 64 = 2**6, 2**128 = 64**(128/6).
So you would still need 128/6 ~ 22 characters to represent a 128 bits ID with a 64 characters alphabet.

jc00ke

jc00ke

I switched to and then away from ULID if only because you cannot paste a ULID into a SQL query. There’s no built-in conversion from the textual representation of a ULID to a UUID and it was really annoying to have to manually convert. I like the ideas behind ULID, but until it’s supported in Postgres I’d stick with UUID.

I also wanted to point out another potential solution: UXID. It supports “t-shirt” sizes (small, medium, large, etc) which would get you your smaller IDs but I think they’re stored as strings so the space-concerns are still present.

stefanchrobot

stefanchrobot

There’s also puid:

Puid

Define modules for the efficient generation of cryptographically strong probably unique identifiers (puids, aka random strings) of specified entropy from various character sets

Examples

The simplest usage of Puid requires no options. The library adds a generate/0 function for generating puids:

iex> defmodule(Id, do: use(Puid))
iex> Id.generate()
"p3CYi24M8tJNmroTLogO3b"

By default, Puid modules generate puids with at least 128 bits of entropy, making the puids suitable replacements for uuids.

Nicd

Nicd

Note that a UUIDv4 only has 122 bits of randomness out of the 128 bits; the version digit (first digit of the third part in the typical string representation) is always 4 and the first digit of the fourth part is [89ab].

bartblast

bartblast

Creator of Hologram

Another option is snowflake id.

trisolaran

trisolaran

I’m not a fan of UUIDs, I would use them only if strictly necessary, namely when you need distributed generation of unique IDs. Otherwise, I really don’t see the appeal. However, once the names start to morph into HUID (hopefully unique), MUID (maybe unique) or LPTUID (let’s pray they’re unique) I’ll keep away from them for good :slight_smile:

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
AstonJ
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
maennchen
:warning: Security advisory: Decimal DoS vulnerability A vulnerability has been published for decimal where very large exponents can cau...
New

Other Trending Topics Top

marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews