Build Adaptive Real-Time Video Streaming with WebRTC and rstream

Robots, drones, remote cameras, and field devices share the same media problem: the source must stay private, browser connectivity must survive NAT and network changes, and the encoder must yield before congestion turns live video into stale video. This guide builds that complete device-to-browser path and shows how to qualify its behavior under controlled bandwidth, latency, jitter, and packet loss.

The device runs one Go binary. GStreamer captures and encodes the source, Pion owns WebRTC, and the rstream Go SDK publishes the viewer and signaling endpoint through an outbound tunnel. Managed STUN/TURN establishes the media path. Trickle ICE and ICE restart handle reachability changes; TWCC/GCC, bounded pacing, NACK/RTX, and optional FlexFEC keep the stream inside the path's current capacity.

The repository is a production-oriented reference implementation. It includes H.264 and AV1 profiles, adaptive bitrate, shared or per-viewer media allocation, session diagnostics, tunnel recovery, product provisioning, static Linux builds, and a reproducible qualification harness. This producer carries the media implementation through the complete series; product control and multi-viewer distribution evolve around it.

Clone the reference implementation:

git clone https://github.com/rstreamlabs/rstream-examples.git
cd rstream-examples/webrtc-video/producer

One media core, three delivery paths

The Go code in webrtc-video/producer is the stable device-side boundary throughout the video series. Capture, encoding, congestion control, pacing, repair, recovery, and OpenMetrics stay in that process. The later guides change who provisions it and how an encoded stream reaches viewers.

1. Adaptive producer — this guide — Go producer ⇄ browser over WebRTC

One producer serves one browser. rstream publishes signaling and supplies managed STUN/TURN; ICE uses a direct media path when available and a TURN relay when required. This is the reference path for qualifying the device-side media loop.

2. Product control with Next.js — Next.js provisions → Go producer ⇄ browser

The same producer runs in provisioning mode. Next.js owns device identity, short-lived rstream material, viewer authorization, and live fleet state. Signaling and media continue between the browser and producer, preserving the one-to-one WebRTC path and its adaptive behavior.

3. MediaMTX fan-out — next in the series — Go producer ⇄ MediaMTX ⇄ browsers

The producer gains a distribution adapter rather than a separate media implementation. It sends one adaptive upstream to MediaMTX, which owns fan-out and one downstream per browser. The Next.js control plane selects the direct or fan-out route; direct WebRTC remains available for one-to-one sessions and transport diagnosis.

This boundary matters operationally. In direct mode, browser feedback drives the producer. In fan-out mode, MediaMTX feedback drives the shared device upstream and each browser has a separate downstream control loop. Producer OpenMetrics therefore describe the device uplink in both architectures; MediaMTX and browser telemetry will describe viewer delivery. Separate measurements on both legs show precisely where quality changes.

This guide focuses on the standalone shape and the complete media path: a one-to-one device-to-browser WebRTC session, publication through an rstream HTTP tunnel, managed STUN/TURN, Trickle ICE and ICE restart, packet recovery, adaptive bitrate, and static Linux delivery.

From media pipeline to product-controlled session

rstream supports two complementary paths for live video. Both use the same private network; the difference is where the application wants the media and session policy to live.

The GStreamer and FFmpeg guides connect existing media tools through rstream nc. Capture, encoding, muxing, buffering, recovery, and playback remain part of the media pipeline, while rstream supplies the private connection. This compact model fits operator workflows, trusted point-to-point streams, RTSP gateways, and applications whose media framework already owns adaptation.

A browser-facing product usually owns more of the session: signaling, short-lived TURN credentials, ICE recovery, congestion feedback, packet repair, lifecycle, and user-visible health. The Go SDK brings tunnel state and WebRTC feedback into the same process as the encoder policy. The application can then distinguish current video from repair traffic, expire obsolete work, and keep its latency budget consistent as the network changes.

The CLI path stays simple and composable. The SDK path gives product code direct control over the media loop. They can coexist in the same deployment.

Prepare the project and local context

Before running the sample, install the rstream CLI, create an account, create a project, and select that project locally. The installation and login details are covered in Installation, CLI Login, and CLI Workflow. Once the project exists, select it locally with this setup.

rstream login
rstream project use <project-endpoint>

The sample reads the active rstream context from the local machine. That context is used both to publish the tunnel and to obtain the TURN material required by the WebRTC path.

For local development, the machine also needs Go 1.26.6+, Node.js 20+, pkg-config, and a GStreamer installation that includes the elements required by the selected pipeline. The sample README includes the package-level setup for macOS and Ubuntu/Debian.

Direct device-to-browser path

The device-side process keeps the browser-facing surface on one origin. Its local HTTP server serves the viewer, signaling, TURN bootstrap, and status endpoints. GStreamer produces H.264 or AV1 access units, and Pion sends them over WebRTC. rstream publishes the HTTP server through one public URL.

The library choices follow the same logic. Pion is the reference WebRTC implementation in the Go ecosystem. It exposes the transport feedback, interceptor model, and media primitives needed for a device-side streamer. GStreamer covers capture, conversion, and encoding while keeping the source graph configurable as a pipeline string.

The Linux build uses gstreamer-full, static linking, cgo, and musl to package the application as a standalone binary. Development retains flexible GStreamer pipelines; deployment receives one self-contained application binary.

The page loads from the tunnel URL, opens signaling on the same origin, asks the device process for TURN credentials, and receives media from that process. The device runs one binary for capture, signaling, session control, and tunnel publication. Its optional metrics listener is deliberately separate from the published application listener.

Use the Go SDK

The device process reads the active local rstream context, opens a published HTTP tunnel with the Go SDK, and serves its local HTTP handler through that tunnel. The tunnel implements the standard Go listener interface, so the local and public paths use the same HTTP server.

client, err := newRstreamClient(opts, cfg.Tunnel.Transport)
if err != nil {
	return nil, err
}
control, err := client.Connect(ctx, nil)
if err != nil {
	return nil, fmt.Errorf("connect to rstream tunnel engine: %w", err)
}
auth := cfg.Tunnel.Auth
name := strings.TrimSpace(cfg.Tunnel.Name)
properties := rstream.TunnelProperties{
	Name:        rstream.StringPtr(name),
	Publish:     rstream.BoolPtr(true),
	Protocol:    rstream.ProtocolPtr(rstream.ProtocolHTTP),
	HTTPVersion: rstream.HTTPVersionPtr(rstream.HTTP1_1),
}
if auth.Token {
	properties.TokenAuth = rstream.BoolPtr(true)
}
if auth.Rstream {
	properties.RstreamAuth = rstream.BoolPtr(true)
}
rawTunnel, err := control.CreateTunnel(ctx, properties)
if err != nil {
	_ = control.Close()
	return nil, fmt.Errorf("create published HTTP tunnel: %w", err)
}

mode: auto controls the tunnel transport between the producer and the rstream engine. It prefers QUIC and falls back to TLS while opening the producer control channel, then keeps the selected transport for the lifetime of that client. Environment variables take precedence over the YAML profile. The public service remains an HTTP tunnel for the viewer, signaling WebSocket, and small API surface served by the Go process.

Once the tunnel is open, the public side still behaves like a regular listener. That keeps the HTTP serving code idiomatic and easy to reuse with the rest of the Go standard library.

func (m *Manager) Listener() net.Listener {
	return m.tunnel
}
 
server := &http.Server{Handler: handler}
serverErrors := serveHTTP(server, tunnelManager.Listener())

On the TURN side, the device asks rstream-go for credentials and passes them to Pion as ICE servers.

credentials, err := rsconfig.CreateTURNCredentialsFromEnv(ctx, p.options)
if err != nil {
	return nil, err
}
configuration := ICEConfig(credentials)

The same project context drives both pieces. The device uses it to publish the viewer and to obtain TURN credentials for the WebRTC path, which keeps the first version deployable as a single device-side process. When the local context already contains the routing data for the project, the SDK derives TURN credentials directly from that context. Otherwise it falls back to the Control plane API. In both cases, the device keeps control of the TURN bootstrap path and the browser still talks directly to the device process during session setup.

More background on the managed TURN side is covered in STUN and TURN. The tunnel publication model is covered in HTTP Tunnels.

Run the reference path

config.h264.yaml is the reference profile. It uses a test pattern source, H.264, the default recovery path, and a fixed encoder bitrate. The shortest way to run it is the following command.

make run

The manual flow builds the device binary first, then starts it with the reference profile.

make build
./webrtc-video-producer -config ./config.h264.yaml

With an active local rstream context, that is enough to bring the full stack online. The process starts locally and prints both the local URL and the public URL.

info  Local URL: http://127.0.0.1:8080
info  Public URL: https://xxxxxxxx.t.<cluster-domain>

Open the public URL and wait for the sample status to load, then select an ICE policy and click Start streaming. The page and signaling WebSocket use the tunnel origin. The browser obtains TURN credentials from the device process, negotiates the WebRTC session, and attaches the remote video track. A working session shows Peer: connected, ICE: connected or completed, and Playback: Playing.

For a local-only run, use the following command.

make run-local

That disables publication and serves the viewer on http://127.0.0.1:8080.

Reference profiles

The repository ships several profiles that cover the main capture, codec, and recovery combinations.

H.264 is the default because it is the most predictable path for low-latency live capture across browsers and devices. AV1 is an additional profile to benchmark on the target hardware.

Tunnel authentication

The sample configures tunnel authentication with two explicit flags. If both are false, the tunnel is public. If token is true, the tunnel requires a valid rstream tunnel token. If rstream is true, rstream identity auth is enforced at the edge. The two flags are product policy, not WebRTC policy, and the decision is enforced on the tunnel before the request reaches the device process.

tunnel:
  auth:
    token: false
    rstream: false

The standalone sample logs the published URL returned by rstream and leaves viewer token distribution to a trusted surface. Long-lived device credentials remain outside browser-visible URLs.

For quick validation in a controlled environment, keep both flags false. For operator-facing workflows, rstream auth is usually the cleanest standalone option. For product-facing access where one backend issues short-lived producer and viewer tokens, use the provisioning mode covered in the next guide.

The broader authentication model is covered in Authentication and HTTP tunnel authentication.

Add feedback and packet repair

The sample enables the first useful layer of transport hardening by default. That matters on remote devices where the uplink may move between Wi-Fi, 4G, and 5G, where the radio environment may change during a session, or where the available bandwidth can collapse without warning.

TWCC gives the sender a transport-wide congestion signal. NACK lets the browser report packet loss. RTX provides the retransmission path. The reference scheduler serves repair traffic promptly, protects current video from starvation, and expires a queued repair after 225 ms. That deadline keeps the repair useful to playback instead of converting packet loss into additional latency.

FlexFEC remains opt-in because it reserves capacity before a loss occurs. The loss-resilient reference emits one proactive repair packet per five media packets and includes that 20% overhead in the sender's wire-rate budget. A separate stress profile uses two repair packets per four media packets. Production deployments should measure the trade-off on their own path; links that recover cleanly through NACK and RTX can leave FlexFEC disabled.

Primary and RTX packets receive their transport-wide sequence number at actual pacer egress. The congestion controller therefore sees the order placed on the wire rather than the scheduler's internal priority decisions. This is the level of control exposed by the Go integration: the sender knows both the feedback semantics and the role and age of every queued packet.

The viewer page exposes the signals that matter while you test.

• negotiated codec
• active recovery path
• current auth mode
• current TWCC target
• current encoder target
• runtime log for tunnel, signaling, ICE, and playback events

These signals make the main transport and media transitions visible from the sample UI. Browser WebRTC diagnostics remain useful for lower-level investigation.

Keep the link resilient

The important design detail is that this sample treats signaling and media as two different planes. The signaling plane is the HTTP/WebSocket surface served by the device process and published through the rstream tunnel. The media plane is the WebRTC path negotiated by ICE. Each plane can recover according to its own protocol and lifecycle.

On the signaling side, the producer prefers QUIC as its tunnel transport to the rstream engine and retains TLS as a fallback.

tunnel:
  transport:
    mode: auto

The public browser entrypoint remains a standard HTTP tunnel. The browser loads the page and opens the signaling WebSocket normally. The transport setting applies only between the producer and the rstream engine. If that connection breaks, the sample's tunnel loop creates a new control channel and republishes the service; it does not rely on the media session to repair the signaling path.

On the media side, ICE gathers local, server-reflexive, and relay candidates, forms candidate pairs, runs connectivity checks, and selects a viable media path. Trickle ICE sends candidates as they appear, so signaling can progress while gathering continues.

The browser and device exchange those candidates over the published signaling WebSocket. Candidates that arrive before the remote description are buffered, candidates from obsolete sessions are discarded, and the current session flushes its queue once SDP negotiation is ready. The complete browser and producer implementations remain visible in app.ts and broadcaster.go.

After connection, ICE continues validating the selected pair. When that path fails while signaling remains available, the browser creates a new offer with ICE restart enabled. Fresh ICE credentials and candidates let both peers select another path while the product session stays active. The producer keeps transient disconnections inside a bounded recovery window and tears the session down only after recovery expires.

The result is a recovery model with separate responsibilities. Automatic tunnel transport selection uses QUIC when it is reachable and TLS otherwise; the tunnel loop republishes the signaling service if that connection is lost. Trickle ICE exchanges candidates incrementally. ICE restart renegotiates the media path when the selected candidate pair no longer works. TURN provides a relay when a direct path cannot be established.

ICE and TURN maintain reachability, while TWCC, GCC, NACK, RTX, and the encoder control loop react to changing path quality. Trickle ICE and adaptive bitrate solve different parts of the same runtime problem: selecting a viable media path and keeping the stream within that path's capacity.

Adaptive bitrate

Once congestion feedback is in place, the next step is to let the encoder react to it. The sample includes an adaptive bitrate backend named twcc-gcc. The transport estimate comes from the standard Pion TWCC and GCC path. The application layer then applies bounded bitrate updates to the active encoder in GStreamer.

The standard Pion WebRTC estimator remains the source of truth. A small application policy keeps the transport budget, encoder, pacer, and product quality floor consistent. Material decreases apply immediately, callback bursts coalesce to the newest estimate, and recovery increases remain gradual. The stream yields quickly when capacity collapses and avoids an optimistic burst as the path recovers.

The first bitrate increase after a measured-loss hold also requests one coalesced recovery key frame. The browser reaches a fresh decodable image sooner, while ordinary healthy-link ramp steps remain free of additional key-frame bursts.

The reference pacer also prevents the encoder and transport from forming two independent queues. It drains normal encoded-frame bursts with 1.5x headroom and bounds admission to 225 ms of sustained-rate backlog. When a new access unit exceeds that budget, the sender rejects the complete frame before RTP packetization and resumes from a new key frame once capacity returns. This preserves a current, decodable stream instead of accumulating stale video.

The current backend keeps the capture profile, frame size, and frame rate stable while it changes encoder bitrate. This produces one measurable feedback loop per encoder, which means either

• media.mode: per-viewer
• or webrtc.maxViewers: 1

The bitrate controller is enabled from the WebRTC section of the configuration.

webrtc:
  maxViewers: 1
  initialBitrateKbps: 5000
  adaptive:
    enabled: true
    backend: twcc-gcc
    twccGCC:
      minBitrateKbps: 2000
      maxBitrateKbps: 8000
      updateInterval: 2s
      changeThresholdPct: 10
      decreaseThresholdPct: 5
      maxIncreasePct: 15
      maxIncreaseStepKbps: 500
      maxIncreaseLossPct: 1
      increaseHoldAfterLoss: 5s

The 1080p30 H.264 qualification profile starts at 5 Mbps and lets the encoder move inside a 2–8 Mbps range. The 2 Mbps floor protects image quality at fixed resolution. Products that span a wider capacity range can place a measured source ladder above the same control loop, reducing frame size, frame rate, or capture profile when the floor is reached.

Observe the producer

The producer can expose its device-side media signals in OpenMetrics format. The exporter covers the source and encoder once, then aggregates the active WebRTC sessions without placing viewer or session identifiers in metric labels.

metrics:
  enabled: true
  listen: 127.0.0.1:9090

The listener is disabled by default and remains separate from the HTTP application published through rstream. A local vmagent or another collector can scrape http://127.0.0.1:9090/metrics and forward the series to the deployment's metrics store. Bind a private interface only when a remote collector is an intentional part of the device network. Add producer and deployment identity as fixed scrape-target labels; viewer and session ids stay out of the producer's metric dimensions.

The exporter separates the TWCC media estimate and encoder media target from the pacer's sustained wire budget and short-burst allowance. It also reports encoded-media and paced-RTP throughput, frame cadence, source freshness, measured loss and delay, queue depth and residence time, adaptive updates, key-frame recovery, NACK/RTX, and FlexFEC. These queries provide a useful first view.

# Encoder media output
rate(rstream_video_producer_encoded_bytes_total[1m]) * 8 / 1e6
 
# RTP written to the network, including RTX and FlexFEC
sum(rate(rstream_video_producer_pacer_sent_bytes_total[1m])) * 8 / 1e6
 
# Encoded frames per second
sum(rate(rstream_video_producer_encoded_frames_total[1m]))
 
# Capture staleness in seconds
time() - rstream_video_producer_last_encoded_frame_timestamp_seconds

The difference between encoded-media and paced-RTP throughput makes repair overhead visible. Frame rate and source freshness distinguish a network problem from a capture or encoder stall, while TWCC, queue, and loss series show whether the sender is yielding before delay accumulates.

Exercise the control loop under congestion

Validate the adaptive path on the interface that carries the device's media traffic. Browser throttling remains useful for page behavior; interface shaping exercises the actual uplink control loop.

On Linux, tc netem is the right baseline.

sudo tc qdisc add dev wlan0 root netem delay 80ms 20ms loss 3% rate 2mbit

To tighten the path further, use the following command.

sudo tc qdisc change dev wlan0 root netem delay 160ms 40ms loss 6% rate 1mbit

To remove the shaping, use the following command.

sudo tc qdisc del dev wlan0 root netem

The expected pattern is straightforward. TWCC target moves first as the transport estimate reacts to the new path. Encoder target follows within the configured update interval and within the configured rate-of-change bounds. If TWCC target is moving and the encoder target is flat, the application policy is the first thing to inspect. If both values move but the stream still behaves poorly, the encoder and source settings are the next place to inspect.

One control loop, several time scales

The reference combines mechanisms that act at different timescales. Each one protects a distinct part of the session.

1. ICE and TURN establish reachability. ICE selects a viable direct path and retains TURN for networks where NAT or policy requires a relay.
2. TWCC and GCC estimate capacity. Fresh receiver feedback drives the wire budget across Wi-Fi, Ethernet, and cellular links.
3. The encoder yields before latency grows. Urgent reductions take effect immediately; recovery increases remain bounded and progressive.
4. The pacer bounds local delay. Normal frame bursts are smoothed and over-budget access units are rejected before packetization.
5. NACK and RTX repair recent loss. Repair receives bounded priority while it can still improve playback.
6. FlexFEC protects selected links proactively. Part of the available capacity becomes repair traffic when measured loss and RTT justify it.
7. ICE restart repairs path failure. The application can select a new candidate pair while preserving the product session.

The in-process Go model coordinates tunnel state, WebRTC feedback, encoder policy, repair age, and user-visible diagnostics. A netcat pipeline continues to expose rstream's reliable and datagram transports to media frameworks that already provide the corresponding control loop.

AV1

The repository includes AV1 profiles for evaluating a more efficient codec without changing the transport architecture. H.264 remains the reference path.

AV1 live capture depends more heavily on the target machine and encoder path than H.264. The opt-in profile measures whether the selected encoder, source, and hardware can sustain the deployment's latency and smoothness envelope.

Use H.264 for the first validation. Treat AV1 as a profile to benchmark on the target device and browsers.

Package the code for Linux devices

Local development is straightforward.

make build
make test

The deployment path that matters for real devices is the Linux distribution build.

make dist-linux-amd64
make dist-linux-arm64
make dist

Those targets build a standalone Linux executable linked against a static gstreamer-full toolchain. The build compiles the GStreamer subset required by the sample, including the codec, parser, and appsink path used by the shipped profiles, and links the Go binary against that toolchain with musl.

The operational result is simple. Copy the binary and its config file to the target machine and run it there. That is the useful shape for Raspberry Pi deployments, embedded systems, and remote devices that do not have a full development stack installed.

The static GStreamer toolchain is defined in build-gstreamer-static-linux.sh. If the pipeline changes, the build script must change with it. Any new source element, encoder, parser, or plugin family introduced by the deployment has to be reflected in that script, otherwise the local development setup and the distribution build will drift apart.

Evolve the distribution layer

The standalone shape keeps validation and deployment compact. Build a Next.js WebRTC Video Platform with rstream uses the same producer in remote-provisioning mode and moves device inventory, credential issuance, viewer authorization, and live tunnel state into a product backend.

The MediaMTX guide will extend that platform with an optional fan-out tier for multiple viewers. The direct path remains useful for one-to-one sessions and for isolating producer behavior; the media-server path trades an additional distribution component for bounded device uplink usage. Both are designed around the producer developed and qualified here.

For a streaming profile tailored to a camera, codec, hardware target, network envelope, or product control plane, contact us.

Congestion and recovery qualification

The qualification separates sender behavior, rstream transit, and path mobility. A direct Docker path establishes the WebRTC reference. The relay profile forces both peers through managed TURN/UDP and applies the same impairment to the producer-to-TURN media flow. A separate mobility run changes the producer's source interface while keeping the signaling WebSocket and peer connection alive.

Each direct and relay configuration runs three times with NACK/RTX alone and three times with the release profile: TWCC/GCC, NACK, RTX, and one FlexFEC repair packet per five media packets. Before the first capacity step, the controller must be stable for a measured ten-second window. A run that misses that precondition is reported as an invalid experiment and excluded before any outcome is compared.

The 1080p30 H.264 profile starts unshaped. Linux traffic control settles at 32 Mbit/s, steps through 16, 12, 8, and 4 Mbit/s, then holds 4 Mbit/s with 120 ms one-way delay, 30 ms jitter, and 2% random loss. Recovery restores 32 Mbit/s after a measured zero-loss queue drain. Signaling and HTTP publication remain outside the shaped flow.

The sender must reduce its encoder target by at least 20% within 30 seconds of the capacity transition, recover to 80% of its stable pre-transition target within 35 seconds, and sustain that level for ten seconds. The graph keeps media rates and wire rates distinct: the encoder, TWCC estimate, and received stream describe media; the pacer includes the configured repair budget; the dashed line is the independent link input.

Transport adaptation is only useful when playback remains current and image quality remains controlled. The receiver must preserve 1920×1080 output, stay above 25 fps on healthy phases and 20 fps while impaired, and spend at most 2% and 10% of those phases frozen. The pinned x264 build reports frame-level quantization; the average QP ceiling under impairment is 42.

The final view explains how that continuity was obtained. RTT stays below 600 ms under the 120 ms one-way impairment. Receiver target buffering is capped at 250 ms and phase-average effective buffering at 300 ms. Sender packet residence is capped at 375 ms and new-media admission at 225 ms. Injected loss must produce valid NACK, RTX, and FlexFEC activity without malformed RTCP/TWCC feedback, pacer overflow, or unexplained kernel socket drops.

All three full-profile direct repetitions passed. They reacted in 1.005–1.006 seconds, recovered in 16.1–18.1 seconds, delivered 29.7–29.9 fps under impairment, and held frozen time between 0.59% and 2.42%. Average impaired H.264 QP stayed between 31.2 and 31.6 and maximum RTT between 195 and 199 ms.

All three full-profile relay repetitions also passed. They delivered 29.5–29.6 fps, held frozen time between 2.11% and 4.34%, kept average QP between 31.3 and 31.4, and measured 251–313 ms maximum RTT under the same impairment. The NACK/RTX-only relay baseline reached 12.8–17.0% frozen time; bounded FlexFEC reduced the median from 16.2% to 3.6%. Only runs that satisfied the stable pre-transition condition enter either comparison.

The qualification record contains the selected matrix, representative direct and relay time series, mobility evidence, every automated assertion, and the rejected-run register. Its manifests identify and link the tested source revision ca8a308. The adaptive-streaming runner reproduces the same profiles.

References

• rstream examples repository
• rstream Go SDK
• Pion WebRTC
• GStreamer
• tc-netem(8)