Configuration

Everything configurable in Ursula lives on this page. Ursula is configured by a TOML config file, an optional resource preset, and a small set of CLI overrides, merged in this order:

built-in defaults < preset defaults < config file < CLI overrides

The normal startup command is:

ursula --config /etc/ursula/ursula.toml --node-id 1

With no config file, ursula falls back to the default preset, which starts a single-node in-memory runtime without Raft, and nothing survives a restart. That's what Install uses. Every other preset, and a config file loaded without --preset default, runs Raft on the disk WAL described under Raft.

CLI flags

Flag	Default	Notes
--config FILE	searches ./ursula.toml, /etc/ursula/ursula.toml, then user config paths	TOML config file
--preset PRESET	default when no config file is found, otherwise none	Resource preset: default, dev, tiny, small, standard, or large
--node-id ID	config file or preset	Overrides raft.node_id

CLI --node-id intentionally overrides any raft.node_id value in the config file so the same file can be mounted on every node. Helm derives it from the StatefulSet ordinal.

Resource presets

--preset supplies a base configuration before the config file is merged, and config-file values win. Pick by node size:

Preset	Intended use	Main defaults
default, dev	Local development	Single-node in-memory runtime without Raft
tiny	Memory-bound tests or very small nodes	64 groups, 64 MiB cold cache, 4 MiB cold flushes, 2 cold writes, 8 MiB hot/admission caps, 64 MiB HTTP in-flight body budget
small	Small nodes or cost-sensitive tests	128 groups, 64 MiB cold cache, 4 MiB cold flushes, 2 cold writes, 16 MiB hot/admission caps, 64 MiB HTTP in-flight body budget
standard	Production baseline	256 groups, 256 MiB cold cache, 8 MiB cold flushes, 4 cold writes, 64 MiB hot/admission caps, 256 MiB HTTP in-flight body budget
large	Larger nodes with more cache and write headroom	512 groups, 512 MiB cold cache, 16 MiB cold flushes, 8 cold writes, 128 MiB hot/admission caps, 512 MiB HTTP in-flight body budget

Server and runtime

Config key	Default	Purpose
server.listen	127.0.0.1:4437	Client-plane listener: the stream API, metrics, readiness and usage
server.cluster_listen	none	Separate cluster-plane listener for Raft gRPC between voters. Unset, the cluster plane shares server.listen. Set, every [[raft.peers]] entry needs a client_url
server.admin_listen	127.0.0.1:4438	Admin-plane listener for the mutating operator routes, plus metrics, readiness and usage. Keep it on loopback and reach it through a tunnel
server.cluster_id	none	The deployment's id, sent as Ursula-Cluster-Id on every HEAD: 1 to 64 characters from A-Z, a-z, 0-9, ., _ and -. Required when raft.peers is set, the same on every node and different for every deployment a client can reach. A single node without it generates u followed by 32 hex digits at its first start and keeps it in its WAL. A single node without raft.wal.path gets a new one on every start. The WAL records the id, and a node whose configured id differs refuses to start. Peers check it: every Raft gRPC request carries the sender's id, and a node refuses a peer with another id (see Peers of another cluster)
server.http_inflight_body_size	256MiB (64MiB in the tiny and small presets)	Write body bytes the node holds at once. A write that would exceed it answers 503 with INGRESS_BODY_BYTES_LIMIT_REACHED. A snapshot commit's copy of parts is charged against it too, from 32 MiB to 56 MiB depending on the budget, and starts only while 32 MiB stays free beside it. With a cold backend it must be at least 64MiB, or the node refuses to start
runtime.cpu_affinity	automatic	On Linux, true forces pinning and false disables it; omitted pins only when cpuset size equals core_count
runtime.core_count	available_parallelism	Owner worker threads; Linux automatically pins only when the allowed cpuset matches this count. Journal writers and the main Tokio runtime use additional threads
runtime.live_read_max_waiters_per_core	65536	SSE waiter cap per core. 0 disables the limit
observability.tokio_console	false	Enables Tokio console when the binary is built with the feature

A node refuses a request body over 32 MiB with 413 and names that limit in Ursula-Max-Request-Bytes. The limit is the protocol's and is fixed. The gateway's --max-request-body-bytes (the chart's gateway.maxRequestBodyBytes) is at most 32 MiB, and a gateway or a chart set above it refuses to start, since the node refuses such a body before reading it and the gateway would answer 502 instead of a typed 413. With a cold backend, a larger snapshot goes up in parts instead.

Without server.cluster_listen, server.listen serves both the client and the cluster plane, and each [[raft.peers]].url points at that address. Leader redirects then name the same URL, so clients must be able to reach it too.

With server.cluster_listen set, server.listen serves only the client plane. Each [[raft.peers]].url points at the peer's cluster listener, and each [[raft.peers]].client_url at its client listener. A follower redirects a client to the leader's client_url, so only peers need to reach the cluster listener. A node refuses to start when a peer lacks client_url. ursula gateway follows a redirect to an upstream directly when its --upstream equals the leader's client_url, and searches its upstreams for the leader otherwise. See Security for what each plane serves.

[server]
listen = "0.0.0.0:4437"
cluster_listen = "0.0.0.0:4439"

[[raft.peers]]
node_id = 1
url = "http://10.0.0.1:4439"
client_url = "http://10.0.0.1:4437"

Raft

Config key	Default	Purpose
raft.node_id	none	This node's stable ID, usually overridden per host with --node-id
raft.append_transport_budget_bytes	134217728	Node-wide charged append bytes per direction, shared across cores and peers; excludes tonic/socket buffers
raft.group_count	core_count × 16	Total Raft groups. Higher values improve stream-to-group hash spread
raft.wal.path	none	Directory of the Raft WAL. Required when raft.peers is set. Raft logs live under PATH/raft-log, and removing the directory wipes state cleanly. Without it, a single node runs its WAL in a temporary directory that is removed on clean shutdown
raft.wal.fsync	never	When journal appends reach stable storage. never acknowledges ordinary data appends from the page cache. Membership batches are always fsynced before acknowledgment. always acknowledges a batch after fsync. Votes, each group's log state and the node's run state are always written with fsync
raft.wal.segment_size	64MiB	Size at which each core's journal starts a new segment. Purge deletes whole segments, so this is also how much disk space comes back at a time. At least 4KiB
raft.wal.cache_size	256MiB	Recent Raft entries kept in memory across the node, split evenly between its groups. Older entries, such as a lagging follower's catch-up, are read from disk
raft.wal.min_available_size	512MiB	Reject writes and mark /__ursula/ready unavailable below this much free WAL-filesystem space. 0 disables the guard
raft.wal.resume_available_size	1GiB	Free space required before disk-pressure admission clears. Must exceed the minimum
raft.peers	empty	Static gRPC peer list, with one [[raft.peers]] entry per voter, including this node. Each entry has node_id, url (the address peers dial for the cluster plane) and client_url (the client-plane address leader redirects name, required with server.cluster_listen and url by default)
raft.init_membership_per_group	false	Per-group membership bootstrap. One voter per group initializes it once every configured voter reports the group empty and no leader. A replica that ever held its group never initializes it again, so keep it true across restarts. The Helm chart sets true

The Raft WAL is always the per-core disk journal. A node with raft.peers set refuses to start without raft.wal.path. A single node without a path creates a fresh ursula-wal-* directory under the system temporary directory, logs its location at startup, and removes it on a clean shutdown (SIGTERM or Ctrl-C). A failed start, or a shutdown that cannot close the WAL, leaves the directory in place. Its streams survive no restart. Set raft.wal.path to keep them.

With fsync = "never", a process crash loses nothing because the page cache survives it, and a graceful shutdown fsyncs every journal before it exits. A host crash or power loss can drop the unsynced tail of that replica's journal. The replica restarts from the frames that still verify and reports that it is recovering. Each of its groups stays behind a recovery gate: it does not campaign or vote until it has applied a fresh barrier from the group's leader, so it never helps elect a leader that lacks a write it acknowledged. While at most a minority of a group's voters lost their tail, the other voters hold every acknowledged write and the group keeps serving. If a majority lost it at once, the group has no leader and refuses writes until an operator accepts the loss (see Operations).

With fsync = "always", a host crash loses no acknowledged write. Every batch waits for an fsync, which costs write latency. A host crash can still leave a complete invalid frame after the last fsync in a journal's newest segment. Recovery cannot tell it from media corruption, so it gates that core's groups as above. A group gated on a majority of its voters, or a single-voter group, then waits for an operator although nothing was lost (see Operations).

Cold storage

Ursula keeps recent data in an in-memory hot ring on every replica and flushes older segments and snapshot blobs to a cold backend. Multi-node clusters should use S3 or an S3-compatible object store so every replica can read chunks flushed by any leader. All replicas must point at the same shared bucket.

[storage.cold]
backend = "s3"
root = "ursula-prod-20260518"

[storage.cold.s3]
bucket = "my-ursula-bucket"
region = "us-east-1"
# endpoint = "http://127.0.0.1:9000"
# access_key_id = "AKIA..."
# secret_access_key = "..."
# session_token = "..."
# server_side_encryption = "aes256"   # default; "aws-kms" or "none"
# kms_key_id = "arn:aws:kms:..."      # customer managed key for "aws-kms"

When access_key_id and secret_access_key are omitted, Ursula uses the standard AWS SDK credential chain (instance profile, environment, profile, and so on). Prefer that over static keys. Set endpoint for S3-compatible stores such as MinIO, R2, or TOS.

Every cold-tier object write requests server-side encryption by default (server_side_encryption = "aes256", i.e. SSE-S3) — free on AWS S3, and Raft snapshot uploads inherit the same setting. aws-kms switches to SSE-KMS, using the AWS managed key unless kms_key_id names a customer managed key. MinIO honors SSE headers only when a KMS/KES is configured; for a MinIO deployment without one, set server_side_encryption = "none" explicitly or writes fail with an SSE error.

Config key	Default	Purpose
storage.cold.backend	none	none, memory, or s3
storage.cold.root	none	Prefix prepended to every cold key. Use a date-stamped value for benchmark runs so cleanup can't touch production data
storage.cold.s3.bucket	none	Required when backend = "s3"
storage.cold.s3.server_side_encryption	aes256	SSE mode for every object write: aes256 (SSE-S3), aws-kms, or none
storage.cold.s3.kms_key_id	none	Customer managed KMS key; only valid with aws-kms
storage.cold.s3.probe_timeout	5s	Deadline of the snapshot-store health probe, retries included. A probe whose first attempt fails fast still has time for one retry, while an attempt that hangs runs into this deadline. Must be non-zero
storage.cold.s3.unhealthy_ticks	3	Bad health ticks, with no good tick between them, before the node sheds its leaderships. Must be non-zero
storage.cold.s3.heal_ticks	2	Good health ticks in a row before a node that shed may lead again. Must be non-zero
storage.cold.flush_interval	1s	Background flush worker tick interval
storage.cold.flush_size	8MiB	Target bytes flushed per group per tick
storage.cold.flush_max_concurrency	4	Parallel cold writes in flight
storage.cold.max_hot_size_per_group	64MiB	Backpressure ceiling: when a group's hot bytes exceed this, new writes get HTTP 503 until the flush catches up. Explicit 0 disables the cap
storage.cold.flush_max_hot_age	5m	A stream's hot tail older than this is flushed whole even when its group is below the flush threshold. 0 disables
storage.cold.compaction_enabled	true	Background compaction: merges small chunks that flushes recorded as compaction debt (no bucket listing) and rewrites packed slices once a stream holds 64 of them
storage.snapshot.backend	auto	auto (S3 snapshots when storage.cold.backend = "s3", inline otherwise), inline, or s3. A node without raft.peers keeps its snapshots inline
raft.snapshot_log_budget	1GiB	Node Raft-log budget of the snapshot driver. A group snapshots once its log since the last snapshot reaches the larger of this budget over twice the group count (at most 16MiB) and twice its last snapshot's size. Above three quarters of the budget a pressure pass snapshots the groups that hold the most log first, one at a time, until the node would fall to half the budget. Past the budget it runs more builds at once, in even steps up to raft.snapshot_build_max_concurrency before twice the budget. With the default of 2 that is two builds past the budget. With 4 it is two past the budget, three past 1.33 times it and four past 1.67 times it. Over twice the budget a cluster node refuses writes with a body until its log is below 1.5 times the budget. A node without raft.peers never refuses them for its log
raft.snapshot_build_max_concurrency	2	Snapshot builds the node runs at once. Each build holds its group's captured state (the hot window, at most storage.cold.max_hot_size_per_group, and the streams' metadata) and, while it uploads, its compressed snapshot, about as large again, and it sends its snapshot-store requests one after another. The snapshot driver starts a build only while fewer run than the node's log calls for: one below raft.snapshot_log_budget and more past it. Builds it started past the budget finish after the log falls below it. A snapshot an operator triggers while every build slot is taken is deferred. With the driver off (storage.snapshot.drive_interval = 0), OpenRaft's own policy starts the builds, on a cluster node up to this many at once, where the default of 1 before 0.7.4 ran one. A node without raft.peers with the driver off builds each group on its own, as 0.8.0 did, with no node-wide cap. Set 1 to keep one build at a time, as before 0.7.4. Raise it, to 4 for example, when the node keeps reaching twice the budget with every build in use (raft_snapshot_builds_in_flight_max at this value) and its memory holds that many builds of its largest groups. The server refuses to start with 0 or more than 16
raft.snapshot_pressure_max_groups_per_tick	16	Groups one snapshot driver pass starts while the node's log is below raft.snapshot_log_budget. Past the budget a pass starts this many for each build it keeps in flight. It also caps the snapshot GC passes one driver tick runs
raft.deleted_data_max_age	the larger of 1h and eight times raft.install_snapshot_timeout	How long a replica keeps a deleted, purged or expired stream in its current Raft snapshot and its Raft log after it applies the removal. The snapshot driver snapshots a group a quarter of this after its oldest removal that no snapshot covers, sooner when the removed bytes reach the group's snapshot threshold, and then purges the group's log through that snapshot. See Deleted data for what the bound covers. With an S3 snapshot store, snapshot GC deletes the snapshot object a replica replaced a quarter of this later, at most an hour or two raft.install_snapshot_timeouts later, whichever is longer. A set value must be at least eight times raft.install_snapshot_timeout (16 minutes at the default), or the server refuses to start, so that a peer has time for two installs of the replaced object

Hot bytes here are payload bytes: the hot window keeps no per-message bookkeeping, so admission and flush thresholds charge exactly the unflushed payload. Flush defaults are conservative, so tune them under load. A typical benchmark profile drops the interval to 200ms, raises concurrency to 32, and bumps the per-group ceiling. Advanced deployments can split the flush threshold and batch size with storage.cold.flush_min_hot_size and storage.cold.flush_max_size, but the single storage.cold.flush_size knob is the normal path. Once a group's accumulated hot payload reaches the threshold, Ursula packs eligible slices from that group into one immutable object up to the configured maximum; single-stream slices of 1 MiB or more keep the direct-object layout, and smaller ones are packed even when they are alone.

With S3 snapshots, a health gate checks the store every storage.snapshot.drive_interval (5s by default). Setting drive_interval = "0s" turns off the snapshot driver and this gate with it. Each tick probes the store and reads the node's cold-flush writes. A tick is bad when the probe fails or misses its deadline, or when a write failed within the flush window and no write has succeeded since. The window is as long as the S3 client can take to give up on a write: storage.cold.s3.timeout for each of its max_retries plus one attempts, the backoff between them, and the flush worker's pause between passes. That is 55 seconds with the defaults. A tick in which writes both failed and succeeded neither ends a bad streak nor adds to it. One failed probe is not enough on its own, and once a later write succeeds a failed write weighs only on its own tick. A store that refuses writes but answers the probe is still caught. After storage.cold.s3.unhealthy_ticks bad ticks with no good tick between them the node stops campaigning and hands its leaderships to its peers. With the defaults that is 10 to 20 seconds into an outage when the probes fail, about 10 to 30 seconds when only writes fail (sooner when the store refuses them outright), and about a minute when every write attempt hangs until its timeout. While a tick is bad the node also holds routine snapshots back, though a log-pressure pass still runs. After storage.cold.s3.heal_ticks good ticks in a row it may lead again. For a node shed for failed writes, the good ticks start with the first write that succeeds after the last failed one. A node that has handed off every leadership flushes nothing, so its good ticks start only once its last failed write leaves the window, about a minute after that write with the defaults.

A shedding node hands off at most four leaderships per tick, the groups with the most unflushed bytes first, and only to peers that may campaign. A peer that is draining, waiting at its recovery gate, or shedding for its own snapshot store, cluster egress or WAL disk is passed over. The gate asks each peer for its leadership-shed state and waits governance.leadership_balance.peer_timeout (500ms by default, must be non-zero) for the answer, the same timeout the leadership balancer uses. A peer that does not answer in time is treated as unable to campaign. A handoff is done once another node leads the group, and one that Raft abandons is handed off again on a later tick. A node with N leaderships therefore evacuates in about ceil(N / 4) ticks, for example 22 ticks or 110 seconds for 86 leaderships at the default interval. The cap is per gate. The cold-health gate (governance.cold_health, every 2s by default) drains the same way and independently, so a node that sheds for both moves up to four leaderships per 2 seconds plus four per 5 seconds. The leadership balancer (governance.leadership_balance) also keeps moving leaderships off a node that holds more than its fair share, up to max_per_tick per balancer tick. The cluster-egress gate (governance.cluster_probe) still hands every leadership off at once, because a node that cannot reach a quorum cannot replicate, and it plans again on every tick for the ones still pending. Its questions to the peers travel the path it judges impaired, so a lost answer says little about a peer. It passes over a peer that answered that it may not campaign. A peer that did not answer may take a leadership's first handoff, and the Raft handoff check decides. A leadership whose handoff did not move goes only to a peer that answered that it may campaign, so a peer that never answers costs each leadership at most one stalled handoff per shed. On a tick when a peer answered that it may not campaign, the gate passes over the peers that did not answer too, because the shed may be cluster-wide.

Raft snapshotting does not force a partial cold flush. Payload below the cold threshold stays in the bounded hot tail and is included in the next group snapshot, so a short snapshot interval does not turn small tails into small cold objects.

Helm and the EC2 scripts map their deployment inputs onto these same keys, so this table applies to every deployment method.

Complete examples

Disk-backed single node (streams survive a restart):

[server]
listen = "127.0.0.1:4437"

[raft]
node_id = 1
group_count = 16

[raft.wal]
path = "./data"

ursula --config ./ursula.toml

Three-node durable cluster on a single port per node (same file on each host, override the node ID per host). Deploy a Cluster shows the same cluster with the cluster plane on its own port, which keeps Raft off the client port:

[server]
listen = "0.0.0.0:4437"
cluster_id = "prod-us-east-1"

[runtime]
core_count = 16

[raft]
node_id = 1
group_count = 256
init_membership_per_group = true

[raft.wal]
path = "/var/lib/ursula"

[storage.cold]
backend = "s3"
root = "ursula-prod-20260518"

[storage.cold.s3]
bucket = "my-ursula-bucket"
region = "us-east-1"

[[raft.peers]]
node_id = 1
url = "http://10.0.0.1:4437"

[[raft.peers]]
node_id = 2
url = "http://10.0.0.2:4437"

[[raft.peers]]
node_id = 3
url = "http://10.0.0.3:4437"

ursula --config /etc/ursula/ursula.toml --node-id 1

See Deploy a Cluster for the full deployment walk-through.