ursuladurable streams
v0.3.22 · apache-2.0

Snapshots

As a stream grows, replaying it from offset=-1 becomes expensive. Snapshots solve this by storing a compacted representation of all data up to a given offset.

PUT    /{bucket}/{stream}/snapshot/{offset}    publish a snapshot
PUT    /{bucket}/{stream}/retention/{offset}   advance retained history
GET    /{bucket}/{stream}/snapshot             read the latest snapshot
GET    /{bucket}/{stream}/snapshot/{offset}    read a specific snapshot

Who publishes

Snapshots are an application-level operation. Ursula stores them but does not produce them. The writer that knows how to compute the merged state is responsible for publishing. A CRDT editor, for example, periodically computes the merged document and publishes it as the latest snapshot. An agent system might snapshot accumulated tool-call state.

Application snapshots are separate from the Raft-internal snapshots Ursula takes for replication and recovery. The two never collide.

Size and content type

Snapshot bodies are capped at 128 MiB. The snapshot has its own content type (separate from the stream's), stored as snapshot_content_type and defaulting to application/octet-stream. A binary CRDT document can be snapshotted under application/octet-stream even when the stream itself carries application/json deltas.

Checkpoint and retention

Publishing a snapshot is safe on its own: it creates a replay checkpoint but leaves all stream history readable. After the application has verified the checkpoint, it may explicitly advance retention to the same or an older checkpoint boundary with PUT /{bucket}/{stream}/retention/{offset}. Only that operation makes earlier reads return 410 Gone and makes old cold-tier data eligible for GC.

JSON streams also accept record coordinates: PUT /{bucket}/{stream}/snapshot?record=N and PUT /{bucket}/{stream}/retention?record=N.

Immutability

Snapshots are immutable at an offset. Retrying the same body and content type is idempotent and returns the same Stream-Snapshot-Digest; different content at the same offset returns 409. Stream-Snapshot-Match can conditionally publish against the current digest. The DELETE /{bucket}/{stream}/snapshot/{offset} endpoint exists for protocol-level completeness, but Ursula refuses every delete: targeting the latest snapshot returns 409, anything else returns 404. To replace a snapshot, publish a new one at a higher offset.