Browser Telemetry

Collect replayable browser events in a JSON stream and query them by client event time.

Browser telemetry is a natural fit for an application/json stream: the browser appends ordinary JSON over HTTP, Ursula stores each event as one message in commit order, and consumers replay or tail events from an offset.

The runnable example is in examples/browser-telemetry.

Data model

Keep one timestamp: the time captured by the browser. It remains application data and may arrive out of order after offline buffering or retries.

{
  "captured_at": "2026-07-18T10:30:00.100Z",
  "type": "network_response",
  "status": 500
}

Ursula does not reorder the stream by captured_at and does not invent a second protocol timestamp. Commit order and offsets are the stream's only coordinates: an out-of-order captured_at never changes the committed order, the stored bytes or any offset. Event time is therefore not a stream read parameter (there is no ?captured_at= read); the experimental indexer below answers time-range queries with offset locators.

Browser collector

Point plain fetch at a same-origin route that proxies to the Ursula stream. The gateway owns browser authentication and CORS policy.

const events = [{
  captured_at: new Date().toISOString(),
  type: "navigation",
  duration_ms: performance.getEntriesByType("navigation")[0]?.duration,
}];

const response = await fetch("/telemetry/events", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(events),
  keepalive: true,
});

console.log(response.headers.get("Stream-Next-Offset"));

One array append stores one message per element. A successful response returns the stream's new tail in Stream-Next-Offset.

Replay and live tail

Read events from the start of the stream:

curl -i 'http://127.0.0.1:4437/telemetry/events?offset=-1'

The body is NDJSON, one event per line. A response returns at most 8 MiB and can end mid-line: buffer any incomplete trailing line and continue from the response's Stream-Next-Offset; Stream-Up-To-Date: true means the read reached the tail. Offsets are opaque: store and pass back the values Ursula returns, and compare them only with each other. For a live native EventSource, add live=sse.

Event-time queries (experimental)

Event time is an ordinary secondary index, not another Durable Streams coordinate. The optional ursula-indexer worker pool is experimental in this release: its API and storage format may change in a minor release. It:

reads each registered stream with ordinary offset reads;
extracts an event time from every message, skipping and counting messages that have none;
writes immutable sorted Parquet parts to S3 and advances a manifest with an ETag compare-and-swap;
returns locators — an opaque offset plus the message's len — that callers resolve back through Ursula.

The source stream remains canonical, and the index can be rebuilt from retained history. Equal timestamps are ordered by offset.

For Kubernetes, deploy one shared worker pool with its own S3 prefix and workload identity:

s3:
  bucket: my-telemetry-index
  region: us-east-1

indexer:
  enabled: true
  replicaCount: 2
  s3:
    prefix: production/telemetry-indexes
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/ursula-telemetry-index

When an application creates a browser stream, register it without changing the Helm release:

curl -X PUT http://ursula-indexer:4493/v1/indexes/browser-session-42 \
  -H 'Content-Type: application/json' \
  -d '{"stream_url":"http://ursula-gateway:4437/sessions/browser-session-42",
       "extract":{"time":["/captured_at"]}}'

extract names the event time with JSON pointers: time is a fallback list, end optionally names when an event ended, each selects several elements of one message (with * matching every member or element), and unit is auto, rfc3339, s, ms, us or ns. The older {"timestamp_field":"captured_at"} form still works. Registration accepts application/json and application/x-ndjson streams and starts at the oldest retained event, or at the tail with "start":"tail".

Query a time window:

GET http://ursula-indexer:4493/v1/indexes/browser-session-42/events?from=2026-07-18T10:00:00Z&until=2026-07-18T11:00:00Z&limit=100

{
  "source": {"stream_url": "http://ursula-gateway:4437/sessions/browser-session-42", "incarnation": "1784370000000"},
  "coverage": {"from": "00000000000000000000", "floor": "00000000000000000000",
               "through": "00000000000000052114", "durable": "00000000000000052114",
               "complete": true, "trimmed_bytes": 0},
  "skipped": {"missing": 0, "invalid": 0, "unparseable": 0, "oversize": 0},
  "entries": [
    {"t_ms": 1784370600100, "t_end_ms": 1784370600100, "offset": "00000000000000004211", "len": 97},
    {"t_ms": 1784370600100, "t_end_ms": 1784370600100, "offset": "00000000000000006630", "len": 88}
  ],
  "next": "8000019f74c698a400000000000019e6"
}

Fetch an event with its locator. A read returns at most 8 MiB, so keep reading from each response's Stream-Next-Offset until len bytes have arrived; never compute an offset yourself. Locators are valid for source.incarnation; if the stream's Stream-Incarnation (HEAD) differs, the stream was recreated and the locators are stale:

async function fetchEvent(streamUrl, { offset, len }) {
  const chunks = [];
  let received = 0;
  while (received < len) {
    const response = await fetch(`${streamUrl}?offset=${offset}&max_bytes=${len - received}`);
    if (!response.ok) throw new Error(`event is no longer readable: ${response.status}`);
    const chunk = new Uint8Array(await response.arrayBuffer());
    if (chunk.length === 0) throw new Error("event is no longer readable");
    chunks.push(chunk);
    received += chunk.length;
    offset = response.headers.get("Stream-Next-Offset");
  }
  const bytes = new Uint8Array(received);
  let at = 0;
  for (const chunk of chunks) {
    bytes.set(chunk, at);
    at += chunk.length;
  }
  return JSON.parse(new TextDecoder().decode(bytes));
}

Fetch the next page with the first response's next and coverage.through, so events indexed in the meantime do not appear halfway through one paginated answer:

GET /v1/indexes/browser-session-42/events?from=2026-07-18T10:00:00Z&until=2026-07-18T11:00:00Z&limit=100&after=8000019f74c698a400000000000019e6&through=00000000000000052114

match=overlap returns events whose [t_ms, t_end_ms] span intersects the window instead of events that start inside it.

GET /v1/indexes/{id}/status reports the same source, coverage and skipped fields. Bad data never stops indexing: messages without a usable time are counted under skipped. When retention trims events before they were indexed, the index follows the stream's retained position and reports "complete": false. When the stream is deleted and created again, the index restarts on the new stream and /status reports restarted_from_incarnation. Indexing stops only when two workers disagree about the same events; the existing index stays queryable, and POST /v1/indexes/{id}/status/resume clears the block after repair.

S3 stores immutable content-addressed Parquet parts and versioned manifests. Parts are split into UTC-day event-time partitions, and bounded size-tiered compaction merges a fixed number of parts within one day. Manifest-aware garbage collection keeps a configurable number of recent generations and an in-flight grace window. The local cache is bounded and disposable, so a replacement pod recovers from S3 without a persistent volume.

OpenTelemetry (OTLP-JSON)

The same indexer serves OpenTelemetry data exported as OTLP-JSON. Create one application/json stream per service, signal and day, and point the stock collector's otlphttp exporter at it:

exporters:
  otlphttp/ursula:
    traces_endpoint: http://ursula-gateway:4437/otel/traces-checkout-20261003
    encoding: json
    compression: none
processors:
  batch:
    send_batch_max_size: 2048

Ursula does not decompress request bodies, so keep compression: none, and keep batches well under the 32 MiB request limit. Each export request is stored as one message. Register the stream with an extractor that walks every span of a batch:

curl -X PUT http://ursula-indexer:4493/v1/indexes/traces-checkout-20261003 \
  -H 'Content-Type: application/json' \
  -d '{"stream_url":"http://ursula-gateway:4437/otel/traces-checkout-20261003",
       "extract":{"each":"/resourceSpans/*/scopeSpans/*/spans/*",
                  "time":["/startTimeUnixNano"],"end":["/endTimeUnixNano"],"unit":"ns"}}'

Each batch becomes one entry spanning its earliest span start to its latest span end; query it with match=overlap. For logs, use "each":"/resourceLogs/*/scopeLogs/*/logRecords/*" and "time":["/timeUnixNano","/observedTimeUnixNano"]: OTLP leaves an unset timeUnixNano at 0, which counts as missing, so the observed time is used instead.