Hello Folks,

Rexd implements the rsync algorithm as a library. The side that has the old data computes a signature, the side that has the new data computes a delta against that signature, and the first side applies it.

The sender never needs the old version. Signatures and deltas use the librsync 2.x format, so rdiff reads what Rexd writes and the other way round.

It does not do directories, file lists or transport. Signatures and deltas are binaries; moving them is up to you.

sig = Rexd.signature(old, block_len: 2048)
wire = sig |> Rexd.Signature.encode() |> IO.iodata_to_binary()

other side

{:ok, sig} = Rexd.Signature.decode(wire)
delta = Rexd.delta(sig, new)

back on the first side

{:ok, ^new} = Rexd.patch(old, delta, max_size: 1_000_000_000)

What is in 1.0:

• All four librsync signature types (RabinKarp or rollsum, BLAKE2b or MD4).
• Streaming versions of signature, delta and patch over enumerables, in bounded memory.
• In-place patching, after Rasch and Burns (USENIX 2003), for storage that can’t hold two copies. The deltas are still ordinary librsync deltas, since output offsets follow from the command lengths.
• Signatures and deltas are treated as untrusted input. Decoding returns error tuples for any byte sequence, patch checks copy ranges and takes a :max_size, and the test suite fuzzes both.
• No runtime dependencies, no NIFs. BLAKE2b-256 and MD4 are implemented in Elixir: :crypto only exposes BLAKE2b-512, which is a different function, and its MD4 depends on OpenSSL’s legacy provider.

Tests compare signatures and deltas byte for byte against rdiff (librsync 2.3.2 and 2.3.4) and b2sum. Committed vectors cover the same ground when those tools aren’t installed.

Rough numbers on one scheduler (Apple M5, OTP 28): signatures at about 55 MiB/s; deltas at about 55 MiB/s where data matches and 30 MiB/s where it doesn’t. Memory measurements for 100 MiB inputs are in BENCH.md.

Limits worth knowing:

• The librsync format has no checksum of the result. A delta applied to the wrong basis can produce wrong output without an error.
• The rolling checksum is unkeyed. Data crafted against a known signature can slow delta computation to about 30 KiB/s; the output stays correct.
• MD4 signatures are there for compatibility only.

If you go near librsync yourself, two details from the source are easy to get wrong: its strong hash is BLAKE2b-256, not truncated BLAKE2b-512, and the RabinKarp rolling update needs a correction term for the seed.
Both are in NOTES.md.

Hex: rexd | Hex
Docs: Rexd v1.0.0 — Documentation
Source: GitHub - thatsme/rexd: The rsync algorithm in pure Elixir: signatures, deltas and patches over binaries and streams, wire-compatible with librsync (rdiff). In-place patching, zero runtime dependencies. · GitHub

If you do delta updates, on Nerves or anywhere else, and this doesn’t fit your case, I’d like to know why.