ursuladurable streams
v0.9.0 · apache-2.0

Exactly-Once Writes

Network retries can produce duplicate appends. Ursula provides server-side deduplication so your application doesn't need its own bookkeeping. Three headers form the identity:

  • Producer-Id: a stable client identifier (UUID, hostname, etc.).
  • Producer-Epoch: bumped on producer restart. Must be ≥ the last epoch the server saw from this producer.
  • Producer-Seq: a per-epoch sequence number. Starts at 0 for a new epoch and must increase by exactly 1 per append.

Both epoch and seq are capped at 2^53 − 1 so they survive a round-trip through JSON.

How dedup works

The server keeps one record per Producer-Id per stream: the current epoch, its highest applied sequence, and the receipt of the producer's newest write. On an append:

  • Next in sequence (seq = last_seq + 1): accepted with 200. The response carries the write's receipt: Stream-Offset where its bytes start, Stream-Next-Offset where they end, and Stream-Closed: true when it closed the stream.
  • Repeat of the newest write: deduplicated and answered 204 with the original receipt.
  • Repeat of an older write of the epoch: deduplicated (never written twice) and answered 204 with Producer-Outcome: receipt-evicted and no range headers.
  • Out of order (seq skips ahead): rejected with 409. The response carries Producer-Expected-Seq, the sequence the server wanted, and Producer-Received-Seq. A seq above 0 of a later epoch that is not claimed yet answers 400 with Producer-Expected-Seq: 0.
  • Older epoch: rejected with 403 Forbidden, whether or not the write applied before. The response's Producer-Epoch is the current epoch. For a write of the epoch that the current epoch's claim closed, Producer-Outcome: epoch-closed;by=claim comes with Producer-Closed-Epoch and Producer-Closed-Seq: every seq up to the closed seq applied, and none above it ever will. For any older epoch, Producer-Outcome: stale-epoch says the server cannot tell which of its writes applied. A producer that gets 403 has been replaced by a newer instance and should stop writing.

On 200 and 204, Producer-Seq is the highest applied sequence of the epoch, not the request's own seq. When the outcome of an append is unknown, for example after a timeout or a 5xx, send the same append again, unchanged. The answer to the repeat says what happened.

A write may also carry Producer-Digest, the BLAKE3-256 hash of its body as 64 lowercase hex digits. The server checks it against the body (400 on a mismatch) and makes it part of the write's identity: a different body under the key of the newest write, or of the current epoch's claim, answers 403 with Producer-Outcome: key-conflict and applies nothing. Without a digest, a repeat with a different body is answered as a duplicate. The full set of answers is in write keys and typed answers.

Restart and epoch hygiene

When a producer crashes and restarts, bump Producer-Epoch. Otherwise the server still expects the next contiguous seq within the old epoch, and the producer (which has lost its in-memory seq counter) will collide. A new epoch starts the seq counter back at 0. The write at seq 0 of a new epoch is its claim. When it closes an earlier epoch, its answer, and the answer to any repeat of it, carries Producer-Closed-Epoch and Producer-Closed-Seq, which say what of the old epoch applied.

A create (PUT) with producer headers answers 400. Create the stream, then claim an epoch.

Per-stream state

Dedup state is scoped per stream. The same Producer-Id writing to two different streams maintains two independent epoch/seq counters, so you do not have to elect a single global writer per producer.

The server keeps the receipt of each producer's newest write only, and forgets older epochs once a higher epoch is claimed. A producer that has not written to a stream for 7 days is forgotten. A stream holds at most 4,096 producers: a new producer beyond that replaces the least recently seen producer idle for at least an hour, and when none has been idle that long its append is rejected with 429 and the error code PRODUCER_LIMIT.

When the server forgets a producer, it raises the stream's epoch floor to that producer's epoch. A write that carries Stream-Incarnation never reaches an epoch at or below the floor that the stream may have forgotten: it answers 403 with Producer-Outcome: producer-expired and the floor as Producer-Epoch. Claim an epoch above the floor, starting at sequence 0. Without Stream-Incarnation, a forgotten producer is new, as the base protocol says.

When you don't need this

For append-only workloads where each write is independent (event logging, audit trails), exactly-once headers are optional. Just POST without them. Add them when retries are real and double-applies would be visible.