Durable Streams Extensions
Document: Durable Streams Protocol Extensions Status: Draft Date: 2026-07-18
Abstract
This document defines extensions to the Durable Streams Protocol. It adds bucket namespacing, snapshot and bootstrap semantics, snapshot upload in parts, byte ranges on snapshot reads, multipart bootstrap delivery, immutable reads that a shared cache can serve, write keys with typed answers, server metadata, error bodies, a JSON message text rule, and read path parameters.
The snapshot and bootstrap design in sections 2 and 3 is based on a proposal by Loro. Ursula implements part of it: there is no latest-snapshot redirect, no snapshot deletion and no conditional publish.
All base protocol semantics remain in effect. Where this document is silent, the base protocol governs.
Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174].
Table of Contents
- JSON Message Text
- 1. Buckets
- 2. Snapshots
- 3. Bootstrap
- 4. Immutable Reads
- 5. Write Keys and Typed Answers
- 7. Ordinary SSE Compatibility
- 8. URL Encoding
- 9. Read Path
- 10. Server Metadata
- 11. Error Bodies
JSON Message Text
This section defines the "normalized message" that the base protocol's JSON mode refers to. It is a storage rule, not an optional extension, and has no extension token. A server MUST apply it to every stream whose content type is application/json (with or without parameters).
It applies to every JSON write: the body of a creating PUT and the body of POST. It applies to every read representation: the default NDJSON view, SSE, and bootstrap parts (Section 3).
- The server MUST validate each request body as JSON text per [RFC8259].
\uXXXXescapes that encode unpaired surrogates are valid. - After top-level array flattening, the server MUST store each message as the original text with insignificant whitespace removed, followed by one LF (U+000A). Insignificant whitespace is U+0020, U+0009, U+000A and U+000D outside string literals. The server MUST NOT reorder object members, drop or merge duplicate member names, or rewrite number or string literal text.
- The nesting depth of a scalar is 0. The depth of an array or object is 1 plus the maximum depth of its elements or member values; an empty array or object has depth 1. A message whose depth exceeds 127 MUST be rejected with
400 Bad Request. Depth is measured per message, after flattening, so a top-level array body may be one level deeper than its deepest message. - A body with invalid JSON or invalid UTF-8 MUST be rejected with
400 Bad Request, and no message from that request is committed. - Every read representation MUST carry the stored message text unchanged. Where a representation embeds the message in a larger JSON document, the embedded text MUST be the stored text.
- Offsets and ETags refer to the stored text.
Messages stored before a server adopted this rule are not rewritten. Clients MUST NOT assume that numbers survive storage as any particular binary floating-point value: a literal such as 1.50e3 or 1e400 is stored and returned as written.
Compatibility note (informative). Earlier Ursula releases re-serialized JSON messages: they sorted object members, kept only one of several duplicate members, rewrote numbers through a binary double, and rejected lone-surrogate escapes and numbers outside the double range. This rule removes these behaviors for new writes. The depth limit is unchanged for a bare message; a message inside an array body gains one level.
A JSON message boundary is an offset whose preceding stored byte is LF, plus the stream's first offset. Stored messages contain no other LF, so the boundaries are exactly the ends of the stored messages.
Client Event Time
The server does not assign a timestamp to messages. Applications that need event time SHOULD include one client-supplied timestamp in each JSON value, for example a captured_at field in a telemetry event.
Client event times MAY be out of order and MUST NOT change committed message order, stored bytes, or offsets. Their HTTP metadata representation and any secondary index over them are outside this document.
1. Buckets
Every stream belongs to a bucket. The stream URL takes the form {base_url}/{bucket_id}/{stream_id}.
1.1. Identifier Constraints
bucket_id: MUST match^[a-z0-9_-]{4,64}$. Globally unique within the service.stream_id: Any UTF-8 string. MUST NOT exceed 122 bytes. MUST NOT contain/,\0, or... On the bucketed/surface, the combined{bucket_id}/{stream_id}key MUST also not exceed 122 bytes.- The literal local stream ID
streamsis reserved, and astream_idMUST NOT start with$, which is reserved for bucket-level subresources.
1.2. Bucket Lifecycle
In Ursula, buckets are implicit namespaces. A bucket exists once a stream has been created in it; PUT /{bucket_id}/{stream_id} needs no earlier bucket create. PUT /{bucket_id} is kept for clients that create the bucket first: it validates bucket_id, changes no state, and answers 201 Created for a valid ID or 400 Bad Request for an invalid one. The client surface has no bucket metadata read, stream listing or bucket delete. Removing a bucket with everything in it is an operator action, the admin-plane bucket purge, which permanently fences the bucket ID: later stream creates in it answer 404. It is not a protocol operation.
This section describes Ursula's server profile. Earlier drafts of this document required explicit bucket creation and defined bucket metadata, stream listing and bucket deletion in Sections 1.3 to 1.5. Those section numbers stay unused so that later section numbers do not change.
1.6. Stream Operations
All stream operations defined in the base protocol apply under {base_url}/{bucket_id}/{stream_id}. A path segment after {stream_id} names a stream subresource (Sections 2, 3 and 4), never another stream. Ursula answers 404 Not Found for any other path with three or more segments under a bucket (paths under /__ursula/ are operational endpoints, not buckets).
The names $transaction, append-batch, attrs, bootstrap, incarnation, retention and snapshot are reserved for any future co-location form and stream subresource; apart from the $ rule in Section 1.1 they remain valid stream IDs. Earlier drafts defined a three-segment path-affinity form and a group-local $transaction endpoint in Sections 1.7 and 1.8; those features are removed and the section numbers stay unused.
2. Snapshots
A snapshot is a materialized representation of a stream's content from offset -1 (inclusive) to snapshot_offset (exclusive). Snapshots enable clients to skip full replay and resume from a compacted state.
2.1. Offset Conventions
- Offsets are opaque tokens as defined in the base protocol. Clients MUST use server-returned
Stream-Next-Offsetvalues.
The messages of a JSON stream are its LF-terminated messages (JSON Message Text); servers MUST use the same message boundaries for snapshot and retention offset validation and for bootstrap parts. A non-JSON stream has no message boundaries that this extension exposes: any committed offset is a valid snapshot or retention offset, and a bootstrap part MAY contain several consecutive messages. This extension does not require any particular binary framing format.
2.2. Publish Snapshot
PUT {stream_url}/snapshot/{snapshot_offset}
Creates a snapshot at the specified offset. The snapshot represents the materialized result of folding all updates in the range [-1, snapshot_offset).
Request Headers:
Content-Type: The content type of the snapshot blob. Servers MUST store this value and return it on subsequent reads. Defaults toapplication/octet-stream.
Response Codes:
204 No Content: Snapshot published successfully.400 Bad Request:snapshot_offsetis invalid or, for a JSON stream, not a message boundary.404 Not Found: Stream does not exist.409 Conflict: another snapshot is the latest (below),snapshot_offsetexceeds the current tail, or the server cannot produce a consistent view at that offset.410 Gone:snapshot_offsetis older than the current earliest retained offset.413 Payload Too Large: the request body or the snapshot exceeds the server's size limit. The response names the limit (Section 10).503 Service Unavailable: For a JSON stream, the server could not verify thatsnapshot_offsetis a message boundary. It MUST NOT publish an unverified offset; the client MAY retry.
Note (informative). Ursula accepts request bodies of up to 32 MiB. With a cold backend it stores snapshot bodies of at least its external-payload threshold (1 MiB by default) as cold-tier objects, and accepts snapshots of up to 1 GiB in parts (below); only the reference is replicated, and the digest is the same as for an inline body. Without one, every body stays inline, up to 32 MiB.
Idempotency:
The server MUST return Stream-Snapshot-Digest for a successful publish. Repeating a publish at the same offset with the same content type and body MUST succeed and return the same digest while that snapshot is the latest. Publishing different content at an already-published offset, or publishing at an offset below the latest snapshot's, MUST return 409 Conflict with the latest snapshot's Stream-Snapshot-Offset and Stream-Snapshot-Digest, so that a client tells a superseded publish (a later offset) from a conflicting one (the same offset). A 409 that names no snapshot refuses the publish for another reason, such as an offset beyond the tail.
Upload in Parts:
A server MAY accept a snapshot larger than one request in parts. It then sends Ursula-Max-Snapshot-Bytes on HEAD (Section 10).
POST {stream_url}/snapshot/{snapshot_offset}/parts
stages one part, whose body is at most Ursula-Max-Request-Bytes. The response is 201 Created with Stream-Snapshot-Part, an opaque part id of 1 to 64 visible ASCII characters. Staging a part changes nothing a reader can see. Repeating it stages another part. A part MAY carry the Stream-Incarnation precondition (Section 2.4), and a part for a stream that does not exist answers 404 Not Found. A part belongs to the incarnation of the stream it was staged for, not to an offset: {snapshot_offset} in its path is checked as an offset and otherwise ignored, so a part staged under one offset MAY be committed at another.
A server MAY bound the parts a stream holds staged and unused, by their number or by their bytes. A server that refuses a part for such a bound MUST answer 429 Too Many Requests with the header of the bound it applied, set to that bound: Ursula-Max-Staged-Parts for a number of parts, Ursula-Max-Staged-Bytes for bytes. The answer carries the stream's Stream-Incarnation. It SHOULD decide the refusal from the request's head, after routing and before it reads the body. A short wait does not lift such a bound, so the answer carries no Retry-After: a commit that uses parts, a discard, or the discarding of unused parts after a bounded time does. A client SHOULD commit or discard the parts of one snapshot before it stages the next.
DELETE {stream_url}/snapshot/parts
discards every staged part of the stream's current incarnation, and MAY discard the unused parts of its earlier incarnations, then answers 204 No Content with Stream-Incarnation. It MAY carry the Stream-Incarnation precondition (Section 2.4), and a stream that does not exist answers 404 Not Found. A part whose upload ends during the discard may or may not survive it. A server that runs a commit of the stream MAY refuse the discard with 503 Service Unavailable and Retry-After, and MUST NOT answer 204 when it failed to delete the parts.
PUT {stream_url}/snapshot/{snapshot_offset}
Stream-Snapshot-Parts: <count>
Stream-Snapshot-Digest: <digest>
publishes the snapshot made of the listed parts, in order. Stream-Snapshot-Parts is a canonical decimal from 1 to 4096: a commit names at most 4096 parts (MAX_SNAPSHOT_PARTS), and no part id more than once. The body lists the part ids, each followed by one LF and nothing else; Content-Type is the snapshot's. The server MUST compute the digest of the assembled snapshot and refuse with 400 Bad Request when it differs from Stream-Snapshot-Digest, when a part is unknown or staged for another stream, when the body does not list exactly Stream-Snapshot-Parts distinct ids, or when it is longer than any such list, and with 413 naming Ursula-Max-Snapshot-Bytes when the assembled snapshot exceeds it. Otherwise the request is a publish of the assembled snapshot, with the same answers and the same idempotency. A server SHOULD check the commit's key, its offset and digest, against the latest snapshot before it resolves the parts: a repeat of a published snapshot succeeds, and a commit whose offset holds another snapshot, or that a later snapshot supersedes, answers 409 Conflict naming the latest, even after its parts are gone. Such a 409 comes before the retention check, so a superseded commit below the earliest retained offset answers 409, where a single-request publish answers 410. A server SHOULD keep the parts until the commit's answer is final for its key (204, 400, 404, 409, 410 or 412), so that a commit refused with a retryable answer such as 503 can be repeated. A server MAY discard a part that no publish used after a bounded time.
Note (informative). Ursula stages parts in its cold backend for the incarnation of the stream, and copies them into one object at the commit, reading a few pieces ahead while it writes. It deletes the parts once the commit's answer is final, and after a digest mismatch, whose copy used them up; it keeps them after a
400that copied nothing. Its orphan sweep reclaims any part, of a live stream or of one since deleted or expired, once it is older than the sweep's grace, a day by default. The node owns a commit once it copies: a client that goes away cancels nothing, a repeat while the commit runs answers503withRetry-After, and a repeat after it answers from the key. A node copies at most two commits at once and answers another with503andRetry-After. A stream recreated during the copy is not published to: a commit withoutStream-Incarnationthen answers400for an unknown part, and one with it412. Without a cold backend Ursula answers parts, commits and discards with501 Not Implementedand sends noUrsula-Max-Snapshot-Bytes.
Note (informative). Ursula bounds each stream to 8192 staged, unused parts and 2 GiB of them, twice what one commit can use, so an abandoned upload and a full retry fit. The parts of the stream's earlier incarnations count until they are deleted, so a delete and a recreate do not reset the bound. Ursula answers a part past either bound with
429,Ursula-Max-Staged-Parts: 8192orUrsula-Max-Staged-Bytes: 2147483648, the stream's currentStream-Incarnationand the error codeSTAGED_PARTS_LIMIT. A part counts itsContent-Lengthwhile it uploads, or 32 MiB without one, and its size once staged. The leader of the stream's group counts the parts in memory. It lists the stream's parts in the cold backend at the stream's first part after it starts to lead or after the orphan sweep reclaimed some of them, after every commit and discard, and at a refused part when its last listing is over a minute old. A part's or a discard's listing also deletes the parts of earlier incarnations staged more than a minute before itsHEAD, a margin that assumes the nodes' clocks agree within a minute. A delete of the stream deletes the parts of the incarnations the leader knew before it at any age, and the stream's other parts staged more than a minute before the delete, after it answers. A commit's listing only counts. The orphan sweep reclaims the rest after its grace. A node runs one commit or discard per stream incarnation at a time and answers another with503,Retry-Afterand the error codeSNAPSHOT_COMMIT_IN_PROGRESS. When a commit's delete of its parts fails, the parts stay until a discard, a delete of the stream or the sweep removes them. A discard whose delete fails answers502withCOLD_STORE_FAILED. After a leader change the new leader does not see the parts that were still uploading to the old one, so a stream can hold more than the bound by those parts until a commit, a discard, a delete or the sweep removes them. Ursula reports the bytes of staged parts per bucket for each group in its usage export, with the leader's term, but sets no bound per bucket and charges nothing: bounding or billing them is the operator's policy.
Retention Effect:
Publishing a snapshot MUST NOT change the earliest retained offset. Retention is a separate, explicit operation:
PUT {stream_url}/retention/{retained_offset}
Like every request that addresses a stream, a retention request MAY carry the Stream-Incarnation precondition (Section 2.4); its success response carries Stream-Incarnation. The server MUST reject a retention boundary that is not aligned to committed data (for a JSON stream, not a message boundary: 400 Bad Request) or is newer than the latest published snapshot. A boundary at or below the current earliest retained offset succeeds and changes nothing, and the response reports the current Stream-Retained-Offset: retention only moves forward, so a repeated request is safe. As for a snapshot, a JSON retention boundary the server cannot verify MUST NOT be applied (503 Service Unavailable). After success, offsets less than retained_offset MUST return 410 Gone. A server MAY retain less than requested, at a committed boundary at or below retained_offset; the success response's Stream-Retained-Offset then reports the effective boundary, and offsets below it MUST return 410 Gone. Ursula retains exactly the requested offset.
Note (informative). Ursula checks a JSON boundary at apply time when the byte before the offset is still in memory. Otherwise it reads that byte from the leader (as a
consistency=leaderread), and applies the operation only if the stream was not deleted and recreated in between; a failed read answers503.
Publishing a new snapshot replaces the previously visible snapshot immediately. The superseded snapshot MAY be garbage-collected asynchronously after the new snapshot becomes visible. Clients MUST NOT depend on older snapshots remaining readable after overwrite.
Concurrency:
Snapshot creation MUST NOT block concurrent appends. The snapshot's consistency boundary is snapshot_offset. Updates at or beyond that offset MUST NOT be folded into the snapshot.
2.4. Snapshot Metadata on HEAD and Stream Incarnation
HEAD {stream_url}
Servers MUST include Stream-Snapshot-Offset and Stream-Snapshot-Digest when a latest visible snapshot exists for the stream. Apart from bootstrap (Section 3), HEAD is the only way to learn which snapshot is the latest.
Response Headers:
Stream-Snapshot-Offset: The latest visible snapshot offset.Stream-Snapshot-Digest: A digest of the latest visible snapshot's content type and body, equal to the digest its publish returned.Stream-Retained-Offset: The earliest retained offset (Section 2.2, Retention Effect).Stream-Incarnation: An opaque token that identifies this incarnation of the stream.Stream-Previous-Incarnation: The stream's token before its history was last rewritten, when the server knows what survived the rewrite (below).Stream-Inherited-Offset: WithStream-Previous-Incarnation, the offset below which the stream's bytes are the same under both tokens.
Semantics:
- Absence of
Stream-Snapshot-Offsetmeans the stream has no visible snapshot. - To read the latest snapshot, a client takes
Stream-Snapshot-OffsetfromHEADand fetches it with Read Snapshot (Section 2.5). - A server implementing this extension MUST send
Stream-Retained-Offseton every successfulHEAD. Reads at offsets below it MUST return410 Gone. Before any retention advance it is the offset of the stream's first byte. Like every offset, it is opaque: clients pass it back asoffsetor compare it lexicographically with other offsets of the same stream. - A server implementing this extension MUST send
Stream-Incarnationon every successfulHEAD. It MUST change when the stream is deleted and recreated at the same URL, and whenever the bytes at an offset the server served under it may differ from what it served: when the stream's history is rewritten (below). It MUST NOT change otherwise. Clients MUST compare it only for equality. Offsets do not identify an incarnation, so a client that stores offsets and snapshot offsets across a possible recreate or rewrite SHOULD store the incarnation with them and send it as a request precondition (below).
History rewrites:
A server rewrites a stream's history when it can no longer serve the bytes it served at some offset: an operator restore from a backup, which rewinds the stream to the backup's tail, or an operator's acceptance of the loss of acknowledged writes after a crash. A rewrite gives the stream a new Stream-Incarnation, so a client holding the old token gets 412 with the new one on its next request that carries it, and learns that its offsets, snapshot offsets and producer state may no longer hold. A rewrite closes every producer epoch claimed before it (Section 5.3).
After a rewrite, a successful HEAD MAY carry Stream-Previous-Incarnation and Stream-Inherited-Offset. The bytes below Stream-Inherited-Offset under the current token are exactly the bytes the server served, or could have served, at those offsets under Stream-Previous-Incarnation. Bytes at or above it may differ. Stream-Inherited-Offset is at most the stream's tail, and Stream-Previous-Incarnation never equals Stream-Incarnation. Closure, the snapshot and the retained offset are reported as usual and say nothing about the previous token. A stream that has not been through a rewrite since it was created sends neither header. One predecessor is reported, not a chain. After two rewrites with no write to the stream between them, the headers name the stream's token before both, with its tail then: no byte changed in between. After two rewrites with a write between them, they name the token between the two, and a client that holds an older token finds no match.
A client that holds token T and finds Stream-Previous-Incarnation: T keeps what it derived from bytes below Stream-Inherited-Offset, under the new token. Everything at or above that offset is void. A client that finds any other token, or neither header, treats everything it derived from the stream as void.
Stream-Incarnation on responses:
A server implementing this extension MUST send Stream-Incarnation on every successful response of a request that addresses a stream: create (201 and 200), append and close, every read (catch-up, the long-poll 204, and the SSE response), HEAD, bootstrap, snapshot publish and read, retention, and the resources of Section 4. It MUST also send it, naming the token of the stream that answered, on every 410 and 416 of a read (catch-up, long-poll, the resources of Section 4, and a snapshot range that cannot be satisfied) and on a 404 of a snapshot read when the stream exists but the snapshot is not its latest. These answers come before the precondition (below), so the token tells a client whether the stream it holds state for answered them. It is absent when the stream does not exist. A client learns the token from its first request, without a separate HEAD.
Stream-Incarnation as a request precondition:
Requests address streams by URL only, so a client holding state from a deleted incarnation (offsets, producer state, a local cache) would otherwise act on a stream recreated at the same URL, including one recreated by TTL expiry and a later create, without any delete. Every request that addresses a stream MAY carry Stream-Incarnation: create (PUT), append (POST, including close), every read (catch-up, long-poll, SSE), HEAD, DELETE, bootstrap, snapshot publish and read, and retention. The resources of Section 4 carry it in their path instead.
- Absent: the request behaves as the base protocol and this extension define it, unchanged.
- Present: the server compares the value for equality with the stream's current incarnation, atomically with the operation. A write is compared when it is applied: a write that was accepted before a delete and recreate but applied after it is refused. A read is compared against the same stream state that serves it. A live read (long-poll or SSE) ends when the stream it was opened against is deleted or recreated: a long-poll answers
404or412, and an SSE stream ends with anerrorevent. - Mismatch:
412 Precondition Failed. A refused write applies nothing; a refused read returns no data, though, like any request that reaches a stream, it may still renew that stream's sliding TTL. The response carries the currentStream-Incarnationwhen the stream exists.412, not409, so it never mixes withStream-Seqand producer conflicts. - Evaluation follows RFC 9110 §13.2: a request that would answer
404without the header still answers404, except that aPUTwith the header for a stream that does not exist answers412and creates nothing. Likewise a read that would answer410or416without the header still does; only a request that would otherwise succeed (or aPUTthat would create) is answered412. - Malformed: a value that is not a token the server could have issued, or more than one value, answers
400 Bad Request.
Note (informative). Ursula derives every token from the stream's creation time and its Raft group's history id, a random 64-bit value the group draws at its creation and again at every history rewrite: a restore (one per backup, whatever the number of attempts) and an accepted loss. A rewrite changes the token of every stream of the group at once. Every token lies in [2^52, 2^53), so it is exact as a JavaScript
Number. A rewrite reportsStream-Previous-Incarnationfor every stream of the group: a restored stream names its token on the source cluster, with the restored tail asStream-Inherited-Offset, unless the source had not written it since a rewrite of its own: then it names the token from before that rewrite, as for any two rewrites with no write between them. After an accepted loss each stream names its token before the loss, with the tail it kept. The report holds until the stream's next rewrite. After a rewrite, every producer record that survived it is closed withepoch-closed;by=history(Section 5.3). A producer whose record did not survive, because the backup predates its claim or the loss took it, is one the stream forgot. A claim of it withoutStream-Incarnationtherefore applies as the base protocol requires for a new producer, at any epoch, and so may reopen an epoch the old history used. A claim withStream-Incarnationnames the old token and gets412. The Ursula SQLite VFS sends the token on every write, so none of its writes from before a rewrite applies after it.
2.5. Read Snapshot
GET {stream_url}/snapshot/{snapshot_offset}
Returns the snapshot blob at the specified offset.
Response Headers:
Content-Type: The content type stored at publish time.Stream-Snapshot-Offset: The snapshot offset.Stream-Snapshot-Digest: The snapshot digest returned at publish time.Stream-Next-Offset: The next offset after the snapshot.Stream-Up-To-Date: true: Present when no update follows the snapshot (the snapshot offset is the stream's tail); absent otherwise. Clients then read on fromStream-Next-Offset.ETag: A strong tag of the whole snapshot. A request whoseIf-None-Matchnames it answers304 Not Modifiedwith the same headers, and ignores any byte range.
Note (informative). Ursula's snapshot
ETagis the snapshot's digest in quotes.
Byte Ranges:
A server MAY honor a single byte range [RFC9110] on a snapshot read: Range: bytes=first-last, bytes=first- or bytes=-length. A server that honors it:
- MUST send
Accept-Ranges: byteson every snapshot answer, and answers206 Partial Contentwith only the selected bytes andContent-Range: bytes first-last/total; - MUST answer a range that starts at or past the end of the snapshot with
416 Range Not SatisfiableandContent-Range: bytes */total; - answers a request with several ranges, a range it cannot parse, or
If-Rangewith the whole snapshot as200; - decides
404and410first, then416for a range that cannot be satisfied, then412, then304. A request whoseIf-None-Matchnames theETagignores the range, so it answers412or304, never416; - sends the same
Stream-Snapshot-Offset,Stream-Snapshot-Digest,Stream-Next-Offset,Stream-IncarnationandETagas for the whole body. The digest always covers the whole snapshot, never the range.
Stream reads keep their offset protocol and ignore Range.
Note (informative). Ursula honors a single range and reads only the part of a cold-tier body that the range covers. A client can read a snapshot's header, such as its first 56 bytes, without the rest of the body. Ursula does not compress snapshot answers.
Response Codes:
200 OK: Snapshot exists.206 Partial Content: The requested byte range of the snapshot.304 Not Modified:If-None-Matchnames the snapshot'sETag.404 Not Found: Snapshot does not exist or has been garbage-collected.416 Range Not Satisfiable: The requested byte range starts at or past the end of the snapshot.
3. Bootstrap
Bootstrap provides single-request initialization: a snapshot (if any) plus retained updates after the snapshot point, returned as a single ordered response that preserves per-message content types. A server returns either every retained update after the snapshot point, or a contiguous prefix of them and an honest continuation offset from which the client reads the rest through ordinary reads (Section 3.3).
3.1. Request
GET {stream_url}/bootstrap
Query Parameters:
/bootstrapis a one-shot initialization endpoint. It does not define any query parameters of its own.- Servers SHOULD reject any
livequery parameter on/bootstrapwith400 Bad Request.
3.2. Response
Response Headers:
Content-Type: multipart/mixed; boundary=<token>Stream-Snapshot-Offset: The snapshot offset, or-1if no snapshot exists.Stream-Next-Offset: The offset immediately after the last returned update byte. When the response contains no update parts, it is the snapshot point (defined below).Stream-Up-To-Date: true: Present whenStream-Next-Offsetequals the stream's tail offset at the time the response was produced. Otherwise the server MUST either omit the header, as for ordinary reads in the base protocol, or sendfalse. Clients MUST treat an absent header asfalse.
The snapshot point S is the visible snapshot's offset when a snapshot exists, and otherwise the earliest retained offset.
Completeness:
A server MUST return one of the following:
- Complete: update parts holding every retained byte in
[S, tail), withStream-Next-Offsetequal to the tail andStream-Up-To-Date: true. - Partial: update parts holding the retained bytes in
[S, P)for some offsetPwithS ≤ P < tail, withStream-Next-Offset: Pand noStream-Up-To-Date: true.PMAY equalS, in which case the response contains the snapshot part and no update parts.
In both cases the update parts MUST hold the retained bytes starting exactly at S, contiguous and in stream order: a server MUST NOT skip or duplicate any byte between S and Stream-Next-Offset. For a JSON stream each part MUST be exactly one message. For a non-JSON stream a part MAY contain several consecutive messages, and a part boundary carries no meaning. Stream-Next-Offset MUST be an offset that ordinary reads accept, and MUST NOT be below S.
A server MAY choose the partial form for any reason, for example because some messages after S are held only in a storage tier that does not keep per-message boundaries in memory. Serving /bootstrap never requires a server to read such a tier. A client MUST NOT treat a partial response as an error.
Note (informative). Ursula never reads cold storage for update parts while serving
/bootstrap(a snapshot body stored in the cold tier is streamed from it as the first part), and it omitsStream-Up-To-Datefrom a partial response. For a non-JSON stream it returns[S, tail)as one part whenSis at or above the stream's cold frontier, every byte of that range is still hot and the range is at most 8 MiB; otherwise it returns the partial form withP = S(the snapshot part only). For a JSON stream, under the same hot-range conditions, it splits[S, tail)on LF into one part per message, and otherwise returns the partial form withP = S; it caps the update parts of one response at 8 MiB, returning the partial form withPat the last message boundary that keeps the parts within the cap, and always includes at least one message, even when that message alone exceeds the cap. Each response uses a fresh 128-bit random multipart boundary.
Response Body:
The body is an RFC 2046 multipart/mixed entity. The multipart boundary, not any outer binary framing, delimits the parts. Servers SHOULD choose an unpredictable boundary for every response, so that no stream content can contain it.
- First part: Snapshot message. If a snapshot exists, the part body MUST be the raw snapshot bytes and the part
Content-TypeMUST equal the snapshot blob's stored content type. - Subsequent parts: Zero or more parts of retained updates starting at the snapshot point, as required by the completeness rules above: one message per part for a JSON stream; one or more consecutive messages per part for any other stream. The part
Content-TypeMUST be the content type of the update. For streams with a fixed stream-level content type, all update parts will normally share that value.
If no snapshot exists, the server MUST still emit an empty first part, Stream-Snapshot-Offset MUST be -1, and that empty part's Content-Type MUST be application/octet-stream. Update parts, if any, then begin at the earliest retained offset.
Servers MUST NOT wrap the entire bootstrap response in an outer binary framing container, and they MUST NOT reinterpret or reframe binary payloads when constructing bootstrap parts.
Example:
HTTP/1.1 200 OK
Content-Type: multipart/mixed; boundary=rr-bootstrap-9f1c2e7a
Stream-Snapshot-Offset: 0000000000000012
Stream-Next-Offset: 0000000000000026
Stream-Up-To-Date: true
Cache-Control: no-store
--rr-bootstrap-9f1c2e7a
Content-Type: application/octet-stream
<snapshot-bytes>
--rr-bootstrap-9f1c2e7a
Content-Type: application/json
{"op":"set","path":["title"],"value":"hello"}
--rr-bootstrap-9f1c2e7a
Content-Type: application/json
{"op":"insert","path":["body",0],"value":"world"}
--rr-bootstrap-9f1c2e7a--
A partial response that returns only the snapshot. The client continues with GET {stream_url}?offset=0000000000000012:
HTTP/1.1 200 OK
Content-Type: multipart/mixed; boundary=rr-bootstrap-4b7d01c3
Stream-Snapshot-Offset: 0000000000000012
Stream-Next-Offset: 0000000000000012
Cache-Control: no-store
--rr-bootstrap-4b7d01c3
Content-Type: application/octet-stream
<snapshot-bytes>
--rr-bootstrap-4b7d01c3--
Response Codes:
200 OK: Bootstrap data returned.404 Not Found: Stream does not exist.
3.3. Follow-Up Live Reads
Bootstrap is one-shot only:
/bootstrapreturns an ordered initialization payload once./bootstrapdoes not supportlive=sseorlive=long-poll.- After applying the bootstrap response, clients MUST continue tailing through the ordinary stream read APIs using the returned
Stream-Next-Offset. - When the response does not carry
Stream-Up-To-Date: true, the ordinary catch-up reads fromStream-Next-Offsetreturn the remaining updates; clients MUST apply them before treating their state as current. - Clients SHOULD prefer
GET {stream_url}?offset=<offset>&live=sseand fall back toGET {stream_url}?offset=<offset>&live=long-pollwhen SSE is unavailable.
3.4. Compatibility
- Regular
GET {stream_url}?offset=...MUST NOT return snapshot bytes. Snapshots are only delivered through/bootstrap. - When a server returns
410 Gonefor a read (because retention has advanced), clients SHOULD call/bootstrapto rebuild state, then continue tailing from the returnedStream-Next-Offsetthrough the ordinary read APIs. - If a server has begun returning
410 Gonefor a stream (i.e., earliest retained offset >-1), the server MUST ensure/bootstrapis available and can provide a snapshot covering that earliest retained offset. - Clients MUST parse
/bootstrapas an ordered sequence of parts. For a JSON stream each update part is one message; for any other stream, clients MUST NOT treat a part boundary as a message boundary.
4. Immutable Reads
Most of a stream can no longer change: every byte below the tail and every published snapshot. This section lets a shared cache, such as a CDN, serve those parts while the server keeps serving the tail. It is independent of the application: any reader that starts behind the tail benefits, such as a subscriber replaying history or a replica that attaches from a snapshot.
It adds four pieces:
- two resources per incarnation, a chunk of the stream's bytes and a snapshot, named by the incarnation token and an offset in the path (Section 4.1);
- a fixed chunk grid, so that independent readers request the same URLs (Section 4.2);
- a response header that marks a response as immutable (Section 4.3);
- a hint naming a separate origin for these resources (Section 4.4).
A client that uses none of them sees no change.
4.1. Immutable Resources
GET {stream_url}/incarnation/{token}/chunk/{offset}
GET {stream_url}/incarnation/{token}/snapshot/{snapshot_offset}
{token} is a Stream-Incarnation value (Section 2.4). {offset} and {snapshot_offset} are offsets, encoded as Section 8 requires. The token in the path is the Stream-Incarnation precondition of Section 2.4, with the same evaluation: when the stream's current token differs, the server answers 412 Precondition Failed with the current Stream-Incarnation, after 404, 410 and 416, which take precedence. Because the token and the offset are in the path, every cache key tells them apart. A Stream-Incarnation request header has no effect on these resources: the path names the token.
- A chunk request returns the stream's bytes from
offsetto the end of the chunk that contains it (Section 4.2), or to the tail when the tail comes first. - A snapshot request returns the snapshot published at
snapshot_offset(Section 2.5), or404 Not Foundwhen it is not the latest visible snapshot. It honors a byte range as Section 2.5 describes. - These resources define
GETandHEADonly. A request with a query component, or with a token or offset that is not in the form the server issues them, MUST be rejected with400 Bad Request.
Response headers on 200 OK, on a snapshot's 206 Partial Content, and on a 304 Not Modified that stands for one of them:
- Chunk:
Content-Type(the stream's read representation, Section 9.4),Stream-Incarnation,Stream-Offset(the first offset the body holds, equal to the requested offset),Stream-Next-Offset(the offset after the last byte),Stream-Closed: truewhen the response ends at the tail of a closed stream,Stream-Immutable: truewhen marked (Section 4.3), andETag. - Snapshot:
Content-Type(stored at publish),Stream-Incarnation,Stream-Snapshot-Offset,Stream-Snapshot-Digest,Stream-Next-Offset(equal to the snapshot offset),Stream-Immutable: trueandETag, with the byte-range headers of Section 2.5.
The body of a chunk response MUST be exactly the stream's bytes in [Stream-Offset, Stream-Next-Offset). Responses of these resources MUST NOT carry Stream-Up-To-Date or Stream-Cursor.
Note (informative). Ursula's chunk
ETagis a strong tag over the token, the range, where the response ends (below the tail, at an open tail or at a closed tail), the content type and the body. The body is hashed although the token and range name the bytes, so that a deployment that broke the incarnation guarantee without changing the token, such as one restored from volume snapshots, gets new bytes at revalidation instead of a304. Its snapshotETagis the snapshot's digest in quotes, the same on200,206and both snapshot resources. Ursula comparesIf-None-Matchweakly, as RFC 9110 requires, so a tag a layer weakened still revalidates. A layer in front forwards tags unchanged otherwise.
4.2. Chunk Grid
The server partitions each stream's offsets into chunks. A reader that starts anywhere and continues from Stream-Next-Offset reaches a chunk boundary after one response, and from then on requests exactly the URLs every other reader of the stream requests.
- A chunk MUST NOT be larger than the server's response cap (Section 9.3).
- A server SHOULD NOT change its grid. Responses for one URL from before and after a change hold different but correct byte ranges (Section 4.3), and caches miss once.
- On a JSON stream a chunk MAY end inside a message (Section 9.4).
Ursula's chunk boundaries are the multiples of 4 MiB (4,194,304) in its offset space.
4.3. Immutable Responses
A server marks a response with
Stream-Immutable: true
when every later successful response for the same URL carries the same body and the same Content-Type, Stream-Incarnation, Stream-Offset, Stream-Next-Offset, Stream-Closed, Stream-Snapshot-Offset and Stream-Snapshot-Digest. The one exception is a change of the server's chunk grid, after which a chunk response for the URL still holds exactly the incarnation's bytes in [Stream-Offset, Stream-Next-Offset).
The server MUST send the marker only on:
- a chunk response that reaches the end of the chunk holding its offset and ends strictly below the tail of the stream state that serves it;
- a chunk response that ends at the tail of a closed stream, including an empty one whose offset is that tail;
- a snapshot response,
200or206.
A chunk response that ends at the tail of an open stream MUST NOT be marked, even when it reaches the end of its chunk, because the stream can still be closed at that offset. A server that serves from a replica that may not hold the tail SHOULD obtain a chunk it cannot mark from one that does, so that a response without the marker means that the stream's tail lies within the chunk. The server SHOULD send the marker whenever these rules allow it.
Note (informative). An Ursula follower serves a chunk only when it can mark it. Otherwise it forwards the request to the stream's Raft leader. The leader answers a chunk it cannot mark, and a
404,410or416of a read, only after it confirmed with a majority of voters that it still leads and applied everything committed (Raft ReadIndex), so a leader that was just elected, or deposed without knowing it, never answers short of a chunk another replica marked, nor for a stream a recreate replaced. A follower forwards these answers to the leader.
A later request for the same URL MAY fail instead: 404 after the stream is deleted or the snapshot is superseded, 410 once retention passes the offset, 412 once the token is retired, or 416 once the token is retired and the stream now at the URL ends before the offset, since 416 takes precedence over 412 (Section 2.4). Each of these, when the stream exists, carries the stream's current Stream-Incarnation, so a client or cache tells a retired URL from a stream that changed under it without another HEAD. A cache that keeps serving a stored marked response serves bytes that were correct for that incarnation and history. The token in the URL keeps them apart from any later stream at the URL and from any later history of the stream, because a server changes the token (Section 2.4) before it serves a stream after any operation that may have rewritten it.
A 304 Not Modified that revalidates a marked response MUST carry the marker and the headers of Section 4.1. Closing a stream MUST NOT change the ETag of a marked chunk response that ends below the tail. A server MUST NOT send two representations of a marked response under one strong ETag, for example by compressing some of them.
The marker states a fact about the stream. It does not make a response storable: Cache-Control still governs that (base protocol Sections 8 and 10.7). A layer that knows who may read a stream MAY replace Cache-Control on a marked response with a directive that a shared cache stores and reuses for the requests it authorizes, for example public, max-age=86400, immutable for a stream that every client may read, or s-maxage=86400 for a private stream behind authorization before the cache lookup (Section 4.5). The lifetime MUST respect the bound of Section 4.5.
Note (informative). Ursula's node sends
Cache-Control: no-storeon every response, marked or not, because it cannot tell a private stream from a public one. Relaxing it is left to the layer in front. The node never compresses these resources, so their responses carry noContent-Encodingand noVary.
4.4. Read Origin Hint
Stream-Read-Origin: https://reads.example.com
A server MAY send Stream-Read-Origin on a successful HEAD. Its value is an origin: a scheme, a host and an optional port, with no path. It suggests that the client request the resources of Section 4.1 for this stream at the same paths on that origin, typically a CDN in front of the same server. The stream's own origin keeps every other request.
A server MUST NOT send it unless requests to that origin reach the same streams at the same paths and are authorized the same way. A read origin MAY act on an authorization decision for a bounded time after it was made, so a revoked grant or a deleted stream stops being served there within that bound rather than at once. The deployment MUST document the bound.
Note (informative). An Ursula node never sends
Stream-Read-Origin. A deployment that runs a read origin adds it in the layer in front.
Client rules:
- A client MAY ignore the hint. A read origin the client is configured with, including a setting that turns the hint off, takes precedence.
- A client learns the token from
HEADat the stream's own origin, never from the read origin. - A client MUST send only requests for the resources of Section 4.1 to the read origin.
- A client MUST ignore a hint that is not
https, and MUST NOT send credentials to a read origin outside the stream URL's site (its registrable domain) unless configured to. - A client MUST accept a chunk response from the read origin only when its status is
200; itsStream-Incarnationequals the token in the URL; itsStream-Offsetequals the offset in the URL; itsStream-Next-Offsetis greater than itsStream-Offset, or equal only in a response without the marker or in one that carriesStream-Closed: true; the length of its body equalsStream-Next-OffsetminusStream-Offset; itsContent-Typeis the stream's read representation;Stream-Closed: trueappears only together with the marker; and the body passes the client's own checks. - A client MUST accept a snapshot response from the read origin only when its status is
200, or206whoseContent-Rangenames exactly the range it asked for, resolved against the size theContent-Rangestates, and whose body is exactly that long; itsStream-Incarnationequals the token in the URL; itsStream-Snapshot-Offsetequals the offset in the URL and the oneHEADreported; itsStream-Snapshot-Digestequals the oneHEADreported; and the body passes the client's own checks, which SHOULD include computing the digest over a whole body when the client knows the server's digest function. - On any other response or a transport error, the client repeats the request at the stream's own origin. A
412,404or a different token from the read origin is not evidence that the stream was deleted or recreated: only the stream's own origin answers that. - A client that used bytes from the read origin and later finds them wrong MUST discard what it built from them and read them again from the stream's own origin.
- After a response without the marker, the client continues at the stream's own origin from
Stream-Next-Offset.
Note (informative). Ursula's snapshot digest is the lowercase hexadecimal BLAKE3 hash of the content type's length (a little-endian 64-bit integer), the content type, and the body.
4.5. Security Considerations
- A shared cache keeps an immutable response after the stream is deleted, until the entry expires. A deployment that stores private stream bytes in a shared cache MUST authorize every request before the cache lookup, and SHOULD bound the lifetime of entries to how long the bytes of a deleted stream may stay reachable. An authorization decision it reuses for a later request is subject to the bound of Section 4.4.
- The cache key MUST include whatever the authorization depends on beyond the URL.
- Reads served from a cache do not reach the server and do not renew a stream's sliding TTL (base protocol Section 5.1).
5. Write Keys and Typed Answers
A write whose response is lost leaves its client unsure whether it applied. This section gives writes a key, so that repeating one is safe, and gives every repeat of a producer write a typed answer, so that a client never has to infer an outcome from a refusal.
5.1. Keys
| Request | Key | A repeated request |
|---|---|---|
Append or close with producer headers (base protocol Section 5.2.1), including a claim: the write at Producer-Seq: 0 that opens an epoch | (Producer-Id, Producer-Epoch, Producer-Seq) within one stream incarnation (Section 2.4), with the write's identity (Section 5.2) | gets the answer of Section 5.3 for the write that holds the key |
| Create | the URL and the configuration (base protocol Section 5.1) | 201 or 200 with the same Stream-Incarnation, or 409 |
Close without producer headers, and DELETE | the URL and Stream-Incarnation (Section 2.4) | a close answers the same Stream-Next-Offset; a delete answers 204, or 404 or 412 once that incarnation is gone |
| Snapshot publish | the URL, the offset, the content type and the body (Section 2.2) | the same Stream-Snapshot-Digest, or 409 |
A create MUST NOT carry producer headers, and a server MUST reject such a request with 400 Bad Request. A client creates the stream, then claims an epoch. Bytes that need a key are appended by a producer. A write without producer headers has no key: repeating it can apply it twice.
5.2. Write Identity
Producer-Digest: <64 lowercase hexadecimal digits>
A producer write MAY carry Producer-Digest, the BLAKE3-256 hash of its body. A server that receives it MUST verify it against the body before it applies the write, and reject a mismatch with 400 Bad Request, applying nothing. It checks this last, for a write that would apply (Section 5.3, step 5): a repeat or a key conflict is decided by the header alone. An answer before the body (Section 5.5) applies nothing and reads no body, so it verifies nothing. The digest is part of the write's identity: a later write with the same key is the same write only when both carry the same digest, or neither carries one.
When a write arrives whose key already applied with another identity, and the server still holds that identity, the server MUST refuse it with 403 Forbidden, Producer-Epoch and Producer-Outcome: key-conflict, and apply nothing. A server MUST hold the identity of each producer's newest applied write, and of the claim of its current epoch for as long as that epoch is current. A repeat of an older write, whose identity the server no longer holds, is answered as a duplicate (Section 5.3), as the base protocol requires.
Because a claim's identity is held for its epoch's lifetime, two claimants of one epoch whose claim bodies differ never both hold it: the later one gets key-conflict. A client that must tell its claim from another claimant's sends a claim body unique to it, with its Producer-Digest. Two claims of one epoch with the same body, or without digests, cannot be told apart, as in the base protocol.
5.3. Producer Write Answers
A server implementing this section evaluates every write in one order, whether it answers before or after reading the body (Section 5.5). The first step that refuses the write decides its answer:
- Request shape, from the headers alone: malformed or unparseable headers, producer headers on a create, a sequence number without an epoch. These answer
400. - The stream: it exists and has not expired. Then idle producer state is discarded (Section 5.4).
- The producer rules of this section, keyed by
(Producer-Id, Producer-Epoch, Producer-Seq)with theProducer-Digestheader as the write's identity. The body is never consulted here, so a repeat gets its receipt and another identity getskey-conflictwhatever the body is. - A closed stream: a write that would apply answers
409withStream-Closed: true. - Content, for a write that would apply: a
Content-Typepresent and matching the stream, a non-empty body unlessStream-Closed: trueis set (400), well-formed message text for a JSON stream, and the body againstProducer-Digest(400on a mismatch).
A write without producer headers has no step 3, so a server MAY answer its content faults of step 5 (a body without a Content-Type, malformed JSON) before step 2: such a write to a missing stream can answer 400 rather than 404.
It answers producer writes with the base protocol's status codes, and adds:
- Every response about an applied write, new (
200) or repeated (204), carries the write's receipt:Stream-Offset(where its bytes start),Stream-Next-Offset(where they end), andStream-Closed: truewhen it closed the stream. A close-only producer write (an empty body withStream-Closed: true) answers204even when it applies now: the base protocol requires204for a close-only request, and its conformance suite checks it for a fresh producer close. A close without producer headers is answered as the base protocol defines. A server MAY keep the range of a producer's newest write only. A repeat of an older write is then answered204withProducer-Outcome: receipt-evictedand no range. Producer-Seqon a200or204is the epoch's highest applied sequence number, as the base protocol defines it, while the receipt describes the request's own write.- A claim's response carries
Producer-Closed-EpochandProducer-Closed-Seqwhen the claim closed an earlier epoch of the same producer: every write of that epoch up toProducer-Closed-Seqapplied, and no other write of it ever will. A repeat of the claim carries the same. - A write of an epoch older than the producer's current one is refused with
403 ForbiddenandProducer-Epoch(the current epoch), whether or not it applied before. When the write's epoch is the one that the current epoch's claim closed, the server addsProducer-Outcome: epoch-closed;by=claim,Producer-Closed-EpochandProducer-Closed-Seq, which say whether the write applied. For any other older epoch it addsProducer-Outcome: stale-epoch: the server cannot say which writes of that epoch applied. - A write of the producer's current epoch, when the stream's history was rewritten since that epoch was claimed (Section 2.4), is refused with
403 Forbidden,Producer-Epoch(that epoch),Producer-Outcome: epoch-closed;by=history,Producer-Closed-EpochandProducer-Closed-Seq, unless it is a repeat of a write that survived the rewrite, which gets its receipt.Producer-Closed-Seqis the epoch's highest seq that survived: every write up to it is in the current history, and no write above it ever applies. The producer claims a later epoch and resends what it needs afterProducer-Closed-Seq. - A write that carries
Stream-Incarnationand could reach an epoch the server may have forgotten is refused with403 Forbidden,Producer-Epoch: <floor>andProducer-Outcome: producer-expired(Section 5.4): the server cannot say which writes of that epoch applied. - Otherwise, a write with a sequence number above 0 of an epoch that is not claimed yet carries
Producer-Expected-Seq: 0, with400when the producer holds an older epoch, as the base protocol requires, or409withProducer-Received-Seqwhen the server holds no state for the producer. A pipelining client then resends in order. - Every
403about a producer write carriesProducer-Epochand exactly oneProducer-Outcome. A403withoutProducer-Epochcomes from a layer that enforces access, not from the producer rules.
Producer-Outcome is a structured field [RFC9651]: a token, with an optional by parameter whose value is a token. by names what closed the epoch. This document defines two values: claim, a claim of a later epoch, and history, a rewrite of the stream's history (Section 2.4). Producer-Closed-Epoch and Producer-Closed-Seq are decimal integers up to 2^53 - 1, like the base protocol's producer headers.
Producer-Outcome | Status | Meaning |
|---|---|---|
| absent | 200 or 204 | applied, with the receipt |
receipt-evicted | 204 | a write under this key applied; the server no longer keeps its range |
epoch-closed;by=claim | 403 | epoch Producer-Closed-Epoch is closed: every seq at or below Producer-Closed-Seq applied, and no seq above it ever will |
epoch-closed;by=history | 403 | the stream's history was rewritten since epoch Producer-Closed-Epoch was claimed: every seq at or below Producer-Closed-Seq is in the current history, and no seq above it ever will be |
stale-epoch | 403 | the write's epoch is older than the producer's current one and is not the epoch the current claim closed; the server cannot say which of its writes applied |
key-conflict | 403 | the key holds a write with another identity; this write did not apply and never will under this key |
producer-expired | 403 | a write with Stream-Incarnation that could reach an epoch the server may have forgotten (Section 5.4) |
5.4. Epoch Floor
A server that discards the state of idle producers SHOULD keep, per stream, the highest epoch of any producer state it discarded: the stream's epoch floor. The floor binds writes that carry Stream-Incarnation, the same opt-in as the incarnation token itself. A write without it keeps the base protocol's rule: a producer whose state expired is treated as new, at any epoch.
A server that keeps a floor MUST refuse, with 403 Forbidden, Producer-Epoch: <floor> and Producer-Outcome: producer-expired:
- a write that carries
Stream-Incarnation, from a producer the server holds no state for, whose epoch is at or below the floor; - a write that carries
Stream-Incarnationon producer state that a write without it opened at or below the floor, unless the write claims an epoch above the floor. Such a claim makes the state as good as one opened above the floor, and it reports no closure: noProducer-Closed-Epochon its answer, and a later write of the epoch it replaced answersstale-epoch. That epoch may be one the server forgot, and its writes may be another writer's, so no answer reports which of them applied. A claim on such state that stays at or below the floor reports no closure either.
A claim without Stream-Incarnation above the floor lifts such state too, and it also reports no closure: no Producer-Closed-Epoch on its answer, and a later write of the epoch it replaced answers stale-epoch. The state then counts as opened above the floor, so later writes that carry Stream-Incarnation are no longer refused for it.
A producer's state opened above the floor holds epochs above every epoch the server discarded for that producer, so its claims of later epochs are never refused for the floor, even after the floor rose for other producers. A writer that carries Stream-Incarnation therefore never reaches an epoch the server forgot, and a stale writer that returns after its state expired cannot claim an old epoch. The floor is per stream: a new producer with Stream-Incarnation on a stream that discarded others' state claims above it.
5.5. Answers Before the Body
A server MAY answer a producer write that carries Stream-Incarnation before reading its body when the answer does not depend on the body: a repeat of a write that applied under that token (204, Section 5.3), or a key conflict (403, Section 5.2). It follows the order of Section 5.3: step 1 runs before any early answer, and the early answers are steps 2 and 3, which never read the body. An early answer is decided by the request's headers alone: the producer headers, Producer-Digest when present, and Stream-Incarnation. The body takes no part in it, not even through its length, its digest or its JSON. The content checks of step 5 apply only to a write that would apply, so a repeat or a key conflict gets the same answer before and after the body, whatever the body holds. The other answers that may come before the body are routing (Section 5.7) and size (Section 10). A client that sent Expect: 100-continue then sends no body.
5.6. Client Rules
- A client MUST NOT send the bytes of one write under two sequence numbers of the same epoch. After a response with
Producer-Expected-Seq, it resends its unacknowledged writes from that number, in order, with their original numbers. - A client MUST NOT resend the bytes of a write under a new epoch until it knows that the write's epoch is closed above its last applied write (from
Producer-Closed-Seqon anepoch-closedanswer or on its own claim's response), or until it has read the stream after its claim applied and found the write absent. It then resends only the writes that did not apply, in order, after the claim. - When the outcome of a write is unknown, the client repeats the same write: the same key, body and
Producer-Digest. A client MUST NOT conclude that a write did not apply, except from the answers of Section 5.3 and from the refusals Section 11.1 names. A refusal says that its own request applied nothing. Whether an earlier attempt under the same key applied, only a receipt or a producer403says.
Note (informative). Ursula keeps a producer's state while the producer wrote within 7 days, and for at least 1 hour when the stream holds more than 4,096 producers. It keeps the range and identity of each producer's newest write and the identity of the claim of its current epoch. It answers a write of the epoch the current one replaced with
epoch-closed;by=claim, and of older epochs withstale-epoch. Its epoch floor rises whenever it discards a producer's state: after 7 idle days, to admit a producer past the 4,096 cap, or when it tidies an idle stream. It marks each producer's state as opened above or below the floor, one byte. The leader answers a repeat or a key conflict before the body when the request'sStream-Incarnationequals the stream's current one afterExpect: 100-continue, or for a declared body of at least 64 KiB on any HTTP version. It decides a write from its head in this order: size (413), which its admission checks before anything else, malformed headers (400), routing (Section 5.7), the early answer, and then load (503withRetry-After), so a repeat gets its answer under load. On HTTP/2 it does not read the rest of the body. On HTTP/1.1 withoutExpect: 100-continueit reads and discards the body, up to 32 MiB, 5 seconds without data and 30 seconds in all, and closes the connection past those bounds, so the early answer saves the staging and the proposal, not the upload.
5.7. Routing Answers
A server that does not lead a stream's writes answers a write 307 Temporary Redirect to the server that does, or 503 Service Unavailable with Retry-After when it knows none, before it reads any of the request body, so that a proxy that sent Expect: 100-continue can send the unread body elsewhere. Once it has read any of the body, a server MUST NOT answer with a redirect. The client resolves the outcome of a request it could not complete by repeating it (Section 5.6).
Note (informative). Ursula routes a create, an append or close, and a snapshot publish before its body. The redirect carries
x-ursula-raft-leader-id,x-ursula-raft-groupandx-ursula-raft-term, so a router can cache the leader per group and discard a hint of an older term. The503carriesx-ursula-raft-group. A node that hosts no replica of the group redirects to a node that does and names only the group, inx-ursula-raft-group, since it knows no leader and no term. A307that carriesx-ursula-raft-groupwithoutx-ursula-raft-leader-idandx-ursula-raft-termis such a redirect, and its target answers as any replica does. Every other redirect, of a read as of a write, carries the leader, the group and the term. A node that knows a leader but not its term answers503withRetry-Afterandx-ursula-raft-groupinstead, so every307it sends parses as one of these two shapes. Each of these503s carries the error bodyNO_LEADER(Section 11). A node redirects a write, or answers itNO_LEADER, only when nothing of the write applied: it refused the write before proposing it (before reading the body, or after reading it when its own leadership check failed before any proposal), Raft refused it before appending it (a leader that no quorum acknowledged within its lease, or one handing its leadership over), or an apply refused it deterministically and the node lost the leadership before it could check what the refusal asked. A node that loses the group's leadership after it proposed a write and before it observed the apply answers503withRetry-AfterandOUTCOME_UNKNOWN, since the write may have applied. SoNO_LEADERalways means that nothing applied, andOUTCOME_UNKNOWNnever does (Section 11.1). A read proposes nothing, so its leadership refusal is a redirect orNO_LEADER, neverOUTCOME_UNKNOWN.
5.8. Stream-Seq Conflicts
A server that refuses an append because its Stream-Seq is not greater than the stream's last accepted one (base protocol) answers 409 Conflict with Stream-Last-Seq: <seq>, the stream's last accepted Stream-Seq, as it was sent, and the error code STREAM_SEQ_CONFLICT (Section 11). A client that uses Stream-Seq as a compare-and-set retries from that value. Only a stream that accepted a Stream-Seq refuses one, so the answer always carries the header, and no other 409 carries it: a 409 for a content type that differs from the stream's carries CONTENT_TYPE_MISMATCH and no Stream-Last-Seq. The refusal is decided where the append applies, so the answer is the same on the leader and through any node that forwards the write.
7. Ordinary SSE Compatibility
Ordinary SSE behavior is defined by the base protocol. Binary SSE: see Section 9.5.
- Extension endpoints MUST NOT redefine the payload shape of ordinary binary SSE
event: dataframes unless they document an explicit endpoint-local contract.
8. URL Encoding
snapshot_offset values originate from Stream-Next-Offset and MAY contain characters requiring URL encoding. Clients and servers MUST apply standard URL encoding/decoding when using offset values in path segments.
9. Read Path
This section collects read parameters and read representations that apply to every stream and that the base protocol leaves to the server.
9.1. Long-Poll Timeout
timeout_ms={t} bounds how long a live=long-poll read waits for new data before the server answers 204 No Content. The default is 1000. An unparseable value means the default, and a parsed value is clamped to 1..60000. It applies only to live=long-poll reads; every other read, including live=sse, ignores it.
9.2. Leader Consistency
consistency={c} selects how fresh a catch-up read must be.
local, the default: any replica MAY serve bytes it has applied. Offsets are immutable, so those bytes are correct, but a replica that lags MAY omit appends that were already acknowledged.leader: the read MUST be linearizable. It MUST reflect every append acknowledged before the request was received, including appends acknowledged by a newer leader.- Any other value MUST be rejected with
400 Bad Request.consistency=leadercombined withliveMUST be rejected with400 Bad Request; live reads follow the leader's watchers already.
A server that cannot prove the read is linearizable MUST NOT serve it. It answers 307 Temporary Redirect to the current leader when it knows one, and otherwise 503 Service Unavailable with Retry-After. Clients SHOULD follow the redirect and retry the 503 after the advertised delay. Routing answers to writes follow Section 5.7.
Note (informative). Ursula forwards a
consistency=leaderread from a follower to the group's leader. The leader confirms with a majority of voters that it still leads (Raft ReadIndex) and waits until it has applied everything committed up to that point. Concurrent reads to the same Raft group share confirmation rounds. A leader cut off from the majority, or one that cannot confirm in time, answers307or503rather than its own state. Ursula applies the same confirmation toHEAD,/bootstrap, snapshot reads and the registration of a long-poll or SSE reader.
9.3. Response Size
A server MAY cap how many stream bytes one response returns. Ursula caps every read response, and every SSE data event, at 8 MiB (8,388,608 bytes).
max_bytes={n} on an offset read asks for at most n bytes. A value above the server's cap is clamped to it. Ursula ignores a value that is not a decimal integer and applies the cap alone; clients SHOULD send a positive integer. On a live read, max_bytes bounds each response or SSE data event; Ursula raises a smaller value to 1 byte, and to 4 bytes on a text SSE read so an event can hold a whole UTF-8 character.
A response that max_bytes or the cap shortened is partial: it MUST NOT carry Stream-Up-To-Date: true, and its Stream-Next-Offset is the offset just past the returned bytes. Clients continue from Stream-Next-Offset until a response carries Stream-Up-To-Date: true.
9.4. JSON Stream Reads
Reads of an application/json stream return the stored messages (see JSON Message Text) in their stored order, each followed by one LF, as Content-Type: application/x-ndjson. HEAD reports the configured content type, application/json. SSE responses for such a stream carry Stream-Data-Content-Type: application/x-ndjson.
max_bytes and the server's cap (Section 9.3) count bytes of this representation, so a response or SSE data event MAY end inside a message. Clients MUST resume from Stream-Next-Offset and buffer an incomplete trailing line until the rest arrives. SSE data: lines are transport framing, not message boundaries.
9.5. Binary SSE
A live=sse read of a stream whose content type is neither text/* nor application/json carries binary data:
- The response MUST include
Stream-Sse-Data-Encoding: base64, andStream-Data-Content-TypeMUST be the stream's content type. - Each
event: datapayload MUST be the standard base64 encoding [RFC4648] of a contiguous byte range of the stream, with no JSON envelope. The server MAY split the text across severaldata:lines; the client joins them, removes the line breaks and decodes. - Each data event is followed by an
event: controlevent whosestreamNextOffsetis the offset just past the decoded bytes.
There is no client opt-in or opt-out: the stream's content type decides the encoding.
10. Server Metadata
A server implementing this extension sends on every answer to a HEAD, 404, redirects and 503 included:
Ursula-Protocol: 2, the version of the contract that Sections 2.4, 5, 10 and 11 define. A later breaking change to them raises it. A server that sends none speaks version 1. A client MAY refuse a server that sends a lower version or none.Ursula-Cluster-Id: <id>: 1 to 64 characters fromA-Z,a-z,0-9,.,_and-, naming the deployment that serves the request. It does not change across restarts, upgrades, leader changes or history events, and an operator keeps it when a restore replaces a deployment. Two deployments that one client can reach MUST NOT share an id. A client that keeps state across requests SHOULD keep the id with it, and treat a different id as a different deployment, not as a new history of the same stream.
These two headers are route-level, so a client tells a host alias routed to another deployment from a deleted stream. Intermediaries pass them through. They come on answers from a server: an answer an intermediary sends itself, such as a gateway's 401, 403, 429 or 503, carries neither, and a client judges the protocol and the deployment only from answers that carry them.
On every successful HEAD of a stream it also sends:
Ursula-Max-Request-Bytes: <bytes>: the largest request body that any request to the stream may carry, as far as the server and the intermediaries that implement this section know.Ursula-Max-Snapshot-Bytes: <bytes>, when the server accepts snapshots in parts (Section 2.2): the largest snapshot it accepts. A commit names at most 4096 parts (MAX_SNAPSHOT_PARTS), each at mostUrsula-Max-Request-Bytes.
Both values are canonical decimal integers: no sign and no leading zero.
Request limit. The request limit of this protocol is 32 MiB (33,554,432 bytes). A server MUST accept a request body of up to that size on every route and MUST refuse a larger one as a size refusal (below). It MUST NOT send an Ursula-Max-Request-Bytes above that value. An intermediary MAY lower the limit for its own clients and MUST NOT raise it. A client SHOULD NOT send a request body larger than the Ursula-Max-Request-Bytes it last read, or than 32 MiB when it has read none.
Size refusals. A server that refuses a request for its size MUST answer 413 Payload Too Large with the header of the limit it applied, set to that limit: Ursula-Max-Request-Bytes for a request body, Ursula-Max-Snapshot-Bytes for a snapshot assembled from parts. It SHOULD decide the refusal from Content-Length before reading the body. A body without Content-Length is refused once it crosses the limit, after the server read that much, and the client may then see a reset instead of the 413. Clients that want the typed answer send Content-Length.
Intermediaries. An intermediary that limits request bodies and implements this section MUST replace Ursula-Max-Request-Bytes on every response that carries it with the smaller of the received value and its own limit for that request, and MUST answer its own size refusals as above. It leaves Ursula-Max-Snapshot-Bytes alone. A 413 without either header comes from a layer that does not implement this section, such as a proxy with its own body limit. It tells the client only that some layer refused the size. The error body of a typed 413 (Section 11) is REQUEST_TOO_LARGE for Ursula-Max-Request-Bytes and SNAPSHOT_TOO_LARGE for Ursula-Max-Snapshot-Bytes. The header names the limit, and the body adds nothing to it.
Note (informative). An Ursula node sends
Ursula-Max-Request-Bytes: 33554432. With a cold backend it accepts a snapshot of up to 1 GiB (1,073,741,824 bytes) in parts and sendsUrsula-Max-Snapshot-Bytes: 1073741824. Without one it sends noUrsula-Max-Snapshot-Bytes. The Ursula gateway lowersUrsula-Max-Request-Bytesto the smaller of its--max-request-body-bytesand the tenant'smax_request_body_bytes, and its own413names that limit. An Ursula cluster's id is set by its operator (server.cluster_id). A single node without one generates one at its first start and keeps it in its WAL.
11. Error Bodies
Every error answer of a server implementing this extension, a 4xx or 5xx on a route this document or the base protocol defines, carries one body shape. It is the shape the Durable Streams protocol uses for its subscription API errors. The protocol defines no error body for its stream operations:
Content-Type: application/json
{"error":{"code":"<code>","message":"<detail>"}}
error is an object with two string members. A server MUST send both. A client MUST accept a body without message, as the protocol's subscription errors may omit it. code is a stable code of uppercase ASCII letters and underscores that names the refusal. A client matches code and nothing else. message is a human-readable detail, such as which header or parameter was malformed and why. It is informative only: its text MAY change in any release, and a client MUST NOT parse it or base a decision on it. A server MUST NOT echo the request body or a credential in message, and MUST NOT name its internals there: no node or peer address, no object key, bucket or endpoint of its storage, no request id of a storage service. The status and the typed headers of Sections 2, 5 and 10 keep their meaning, and the body adds the code beside them: a 413 still names its limit in a header, a 503 still carries Retry-After, a producer refusal still carries Producer-Outcome. An answer to HEAD carries the Content-Type and no body. A redirect is no error answer and carries no error body. An answer that a server's HTTP layer sends before any route sees the request, such as a 400 for a request it cannot parse, may carry no body.
A server MAY add members to either object, and codes, within a minor version. A client MUST ignore members it does not know, and MUST treat a code it does not know by the status and the headers alone. A client MUST NOT require the body: an answer from a layer that does not implement this section, such as a proxy, may carry any body or none.
A live SSE read (Section 7) that ends in error sends an error event whose data is the same body, on one line.
An intermediary that implements this section renders its own error answers with the same shape, and passes the server's through unchanged. Its messages name no internal address either. An intermediary that conceals a resource from a client answers with the status, headers and body the server answers for a bucket that does not exist (BUCKET_NOT_FOUND), so the concealment tells the client nothing. Like every answer of the intermediary's own, it carries neither Ursula-Protocol nor Ursula-Cluster-Id (Section 10), so a client reads it as a refusal, not as a stream its deployment deleted.
| Status | Codes |
|---|---|
400 | INVALID_REQUEST, INVALID_HEADER, INVALID_QUERY, INVALID_OFFSET, QUERY_NOT_ACCEPTED and INVALID_RESOURCE (the immutable resources of Section 4), INVALID_BUCKET_ID, INVALID_STREAM_ID, INVALID_PRODUCER, PRODUCER_DIGEST_MISMATCH, MISSING_CONTENT_TYPE, EMPTY_APPEND, INVALID_RECORD_BOUNDARIES, NOT_JSON_BOUNDARY, INVALID_RETENTION, INVALID_SNAPSHOT, INVALID_COLD_FLUSH, IMPORT_INVALID, INVALID_PARTS_COMMIT, UNKNOWN_PART, SNAPSHOT_DIGEST_MISMATCH, BODY_UNREADABLE, UNSUPPORTED |
401 | INVALID_CREDENTIAL, CREDENTIAL_EXPIRED (an intermediary) |
402 | PAYMENT_REQUIRED (an intermediary) |
403 | PRODUCER_EPOCH_CLOSED, PRODUCER_EPOCH_STALE, PRODUCER_EXPIRED, PRODUCER_KEY_CONFLICT, and FORBIDDEN, ACCOUNT_REVOKED (an intermediary) |
404 | STREAM_NOT_FOUND, BUCKET_NOT_FOUND, BUCKET_ERASED, SNAPSHOT_NOT_FOUND, NOT_FOUND (no such route) |
405 | METHOD_NOT_ALLOWED, UNSUPPORTED (the removed latest-snapshot path) |
408 | BODY_TOO_SLOW (an intermediary) |
409 | STREAM_CLOSED, STREAM_SEQ_CONFLICT, CONTENT_TYPE_MISMATCH, STREAM_ALREADY_EXISTS_CONFLICT, PRODUCER_SEQ_CONFLICT, SNAPSHOT_CONFLICT, IMPORT_CONFLICT |
410 | STREAM_GONE |
412 | INCARNATION_MISMATCH |
413 | REQUEST_TOO_LARGE, SNAPSHOT_TOO_LARGE |
416 | OFFSET_OUT_OF_RANGE, RANGE_NOT_SATISFIABLE (a snapshot read's byte range) |
429 | PRODUCER_LIMIT, STAGED_PARTS_LIMIT, and RATE_LIMITED, LIVE_READS_EXHAUSTED, UPLOADS_EXHAUSTED, UPLOAD_BYTES_EXHAUSTED (an intermediary) |
500 | INTERNAL, CREATED_AT_OUT_OF_RANGE, HISTORY_MISMATCH |
501 | NO_COLD_STORE |
502 | COLD_STORE_FAILED, and UPSTREAM_FAILED (an intermediary) |
503 | NO_LEADER, WAL_DISK_PRESSURE, RAFT_LOG_PRESSURE, INGRESS_BODY_BYTES_LIMIT_REACHED, COLD_BACKPRESSURE, RAFT_UNCOMMITTED_BACKPRESSURE, LIVE_READ_BACKPRESSURE, HISTORY_PENDING, APPLY_STOPPED, GROUP_RESTARTING, OUTCOME_UNKNOWN, JSON_BOUNDARY_UNVERIFIED, SNAPSHOT_COMMITS_BUSY, SNAPSHOT_COMMIT_IN_PROGRESS, and AUTHENTICATION_UNAVAILABLE, AUTHORIZATION_UNAVAILABLE, NO_UPSTREAM, UPSTREAM_UNREACHABLE, UPLOAD_BUDGET_EXHAUSTED (an intermediary) |
504 | UPSTREAM_TIMEOUT (an intermediary) |
Two codes name no particular refusal. INVALID_REQUEST answers a 4xx that no other code names: a malformed request a route rejects before its handler, such as an undecodable path, or any other 4xx a layer in front of the handlers sends. INTERNAL answers a 5xx that no other code names: a server fault, which a retry may not fix. A client treats either by its status alone.
A code says what refused the request. For a 408 or a 5xx it also says whether the write could have applied (Section 11.1).
Note (informative).
ursula-headersdefines the codes (ErrorCode, which keeps a code it does not know asErrorCode::Unknownwith its message) and parses and renders the body (ErrorBody). Ursula sends the detail of the request's fault only for the refusals it allowlists, whose messages name only what the request named, such as a malformed header, aStream-Seqor a producer conflict. Every other answer, a new kind of refusal included, carries a fixed message per code, so a message never names a node, a peer or the cold store. A gateway's concealing404and a node's404for a bucket nothing created carry the same status, body and error headers. The node's404to aHEADadds its identity, and the gateway's does not. Ursula's operator routes under/__ursula/keep their own answers. The upstream reference server sends its subscription errors in this shape, with codes such asFENCEDandALREADY_CLAIMED. Its stream operations answer errors in plain text.
11.1. Write Outcomes
A write is every request that changes what a stream or a bucket holds: an append or a close, with or without producer headers, a create of a stream, a snapshot publish, a parts commit, a part staged, a discard of the staged parts (DELETE {stream_url}/snapshot/parts), a retention advance (PUT {stream_url}/retention/{offset}), a delete of a stream, and a create of a bucket. A client reads every answer to a write as one of five outcomes:
- Applied: a
2xxthat carries the typed answer of its write. The write holds. - Repeat: a producer
204with its receipt, or withProducer-Outcome: receipt-evicted(Section 5.3). An earlier request with the same key applied the write. A close-only producer write answers204even when it applies now, so its receipt reads as a repeat either way, and both hold. - Refused: this request applied nothing of the write and never will. Either the server refused it before proposing it, or it was proposed and its apply refused it deterministically.
- Redirected: a
307or308. This target applied nothing and names another (Section 5.7). - Unknown: the write may have been proposed and its apply result was not observed, so it may or may not have applied, or may still apply. The client repeats it with its key (Section 5.6).
NO_LEADER and OUTCOME_UNKNOWN follow from these definitions (Section 5.7). A refusal speaks for its own request only: whether an earlier attempt under the same key applied, only a receipt or a producer 403 says.
One rule decides an answer that carries no error code this document defines: it is unknown, unless its status alone proves an outcome. A 2xx proves an apply only with the typed answer a server always sends for its write: the receipt of a write with producer headers, Stream-Next-Offset of any other append or close, Stream-Incarnation of a create or of a discard of the staged parts, the published snapshot of a publish or a parts commit, Stream-Snapshot-Part of a part, and Stream-Retained-Offset with Stream-Incarnation of a retention advance. A 2xx without it comes from a layer outside the server, such as a misrouting proxy, and says nothing of the write. A delete of a stream and a create of a bucket carry no typed answer, so their 2xx proves an apply by its status. A 307 or 308 proves a redirect. A 4xx other than 408 proves a refusal, by the requirement on intermediaries below. A 408, a 5xx, and a 1xx or 3xx other than a redirect prove nothing: a load balancer or a proxy may send one after it forwarded some or all of the request. A request that got no answer, such as one whose connection failed during its upload, is unknown.
An intermediary in front of an Ursula server or gateway MUST NOT answer a write with a 4xx other than 408, a 307 or a 308 of its own after it forwarded any of the request, and MUST NOT resend a write it forwarded unless the answer proves the target applied nothing, as a server's redirect does (Section 5.7). An intermediary that limits request bodies MUST NOT accept a body larger than the server's request limit (Ursula-Max-Request-Bytes, Section 10), so that its own 413 refuses every body the server would refuse before reading it.
A 408 or a 5xx is a refusal only when its code proves one. Each of these codes is sent only when nothing of the write applied:
| Status | Codes that prove a refusal |
|---|---|
408 | BODY_TOO_SLOW (an intermediary cut the body before it forwarded any of it) |
501 | NO_COLD_STORE |
503 | NO_LEADER (routing, or a leadership check before the proposal), WAL_DISK_PRESSURE, RAFT_LOG_PRESSURE, INGRESS_BODY_BYTES_LIMIT_REACHED (admission), COLD_BACKPRESSURE, RAFT_UNCOMMITTED_BACKPRESSURE, APPLY_STOPPED (checks before the proposal), GROUP_RESTARTING (no engine of the group runs on the node), HISTORY_PENDING (a check before the proposal, or an apply that refused it), JSON_BOUNDARY_UNVERIFIED (an apply that refused it: never applied unverified, Section 2.2), SNAPSHOT_COMMITS_BUSY (refused before the copy), and AUTHENTICATION_UNAVAILABLE, AUTHORIZATION_UNAVAILABLE, NO_UPSTREAM, UPSTREAM_UNREACHABLE, UPLOAD_BUDGET_EXHAUSTED (an intermediary that forwarded nothing) |
Every other 408 or 5xx is unknown: OUTCOME_UNKNOWN, SNAPSHOT_COMMIT_IN_PROGRESS (an earlier attempt still runs), INTERNAL, CREATED_AT_OUT_OF_RANGE, HISTORY_MISMATCH, COLD_STORE_FAILED, UPSTREAM_FAILED, UPSTREAM_TIMEOUT, LIVE_READ_BACKPRESSURE (a read's code, which no write gets), and a code the client does not know. The proof is the code, never the status: a 503 comes before a proposal and after one, so only its code tells which.
An intermediary's own refusals are refusals by these rules. An intermediary MAY let a program that embeds it refuse a request before it forwards it, such as one of an unpaid account. Such a refusal carries the same body shape and Cache-Control: no-store, one of the codes PAYMENT_REQUIRED (402), ACCOUNT_REVOKED (403), FORBIDDEN (403) or RATE_LIMITED (429, with Retry-After), no producer header and no server identity (Section 10). A 403 with Producer-Epoch and Producer-Outcome is a producer answer (Section 5.3), and a 403 without them is an access layer's. Both are refusals, each with its own code. A client believes a producer code in the body of a 403 only together with the producer headers that the code needs.
Note (informative).
ursula_headers::classifyimplements these rules for every client, given the write's kind (WriteKind). With thetest-supportfeature,ursula_headers::write_outcome::inventorylists every answer a write can get from an Ursula node or gateway, with its outcome. Ursula's tests drive each through a real node and a real gateway, and a deterministic simulation checks under leader churn that a key's first repeat follows an attempt answered applied or unknown. The Ursula gateway'sGateway::refuseis the refusal an embedding program sends, and the gateway refuses to start with a request limit above the server's.