Operations
Observability and operational interfaces for the self-hosted engine.
This page documents Community Edition operational behavior. CE supports Prometheus metrics as an optional emitter and keeps live clients, tunnels, and streams in process memory.
The Engine API exposes live inventory and signaling endpoints under /api. GET /api/clients and GET /api/tunnels return point-in-time state. /api/sse and /api/websocket stream an initial snapshot followed by lifecycle events.
Runtime Checks
Use container or service-manager status first:
docker compose ps
docker compose logs --tail=100 engineThe Engine OpenAPI document is a simple TLS check for the base hostname:
curl -fsS https://edge.example.com/api/openapi.json | jq -r '.info.title'Inventory uses the base engine host and a valid engine JWT:
curl -fsS \
-H "Authorization: Bearer $RSTREAM_AGENT_TOKEN" \
https://edge.example.com/api/tunnels | jqIf you validate against a local self-signed certificate, use --cacert and --resolve rather than disabling TLS verification:
curl -fsS \
--resolve edge.localhost.rstream.test:18443:127.0.0.1 \
--cacert tls/ca.crt \
-H "Authorization: Bearer $RSTREAM_AGENT_TOKEN" \
https://edge.localhost.rstream.test:18443/api/tunnels | jqPrometheus Metrics
When Prometheus metrics are enabled, the engine exposes a metrics endpoint. The default path is /metrics, and the path is normalized if it is configured without a leading slash.
Prometheus metrics require a listen address to be configured. When the emitter is disabled, no metrics server is started.
When the metrics listener binds to a non-loopback address, the engine requires metrics.prometheus.bearer_token. Scrape it with:
curl -fsS \
-H "Authorization: Bearer $RSTREAM_METRICS_TOKEN" \
http://127.0.0.1:9090/metricsKeep the metrics listener private when possible. If it must be reachable from another host or container, use a long random bearer token and restrict network access at the infrastructure layer.
Certificate Rotation
CE uses the static certificate provider. The provider reloads the certificate pair automatically on a later TLS handshake when cert_file or key_file changes.
Use atomic replacement from a certificate sidecar or deploy hook:
install -m 0644 fullchain.pem /etc/rstream/tls/cert.pem.new
install -m 0600 privkey.pem /etc/rstream/tls/key.pem.new
chown "$RSTREAM_RUNTIME_UID:$RSTREAM_RUNTIME_GID" /etc/rstream/tls/cert.pem.new /etc/rstream/tls/key.pem.new
mv /etc/rstream/tls/cert.pem.new /etc/rstream/tls/cert.pem
mv /etc/rstream/tls/key.pem.new /etc/rstream/tls/key.pemThen verify the served certificate without restarting the engine:
echo | openssl s_client \
-connect edge.example.com:443 \
-servername edge.example.com \
-verify_return_error 2>/dev/null \
| openssl x509 -noout -subject -fingerprint -sha256If the new files are invalid, the engine keeps serving the previously loaded certificate and tries again on later handshakes after the files change.
Upgrades
Pin image tags for production deployments. Upgrade with the same Compose lifecycle used for other infrastructure services:
docker compose pull engine
docker compose up -d engine
docker compose logs --tail=100 engineLive tunnels are in memory and are tied to agent control channels. Restarting the engine disconnects agents; agents should run with retry enabled so they reconnect and recreate their tunnels.
CE Integration Boundary
CE exposes Prometheus metrics and live Engine API state. It does not provide a persisted event store in the public runtime, so operational dashboards should scrape Prometheus and query or watch the live Engine API.
CLI Event Stream
The CLI can subscribe to the engine event stream using rstream events and can optionally forward each event as JSON. Add --webhook when the local receiver should see the signed webhook request shape.
See Signaling for transport behavior and examples.
Troubleshooting
| Symptom | Check |
|---|---|
| Engine exits on startup with a certificate error | Confirm certs.static.cert_file and certs.static.key_file exist, are readable by the container user, and contain a matching certificate/key pair. |
TLS works for edge.example.com but agents cannot connect | Agents must use the base engine host, for example edge.example.com:443, and DNS must resolve that name to the engine. |
| Published tunnel hostname does not resolve | Add or fix *.t.<engine.host> DNS. A wildcard under engine.host does not cover the .t. tunnel namespace. |
| Prometheus exporter refuses to start | A non-loopback metrics bind address requires metrics.prometheus.bearer_token. |
| Local self-signed validation fails | Trust the local CA with --cacert, SSL_CERT_FILE, or the OS trust store. Do not use the local CA pattern for production. |
| A tunnel option is rejected as unavailable | Confirm that the requested option is part of the CE scope documented on Self-Hosted. Hosted and private-runtime controls are rejected by the public CE engine. |