RTSP to WebRTC: Gateway, Codecs & Browser Checks | Callaba

RTSP-to-WebRTC is a gateway workflow, not a URL substitution. The gateway acts as an RTSP client toward a camera or media server, receives the associated RTP media, and creates a separately negotiated WebRTC session for the browser. Codec, audio, timing, security and connection state must be validated on both sides.

The gateway terminates two different session models

RTSP controls the source session; RTP normally carries its media

Probe the source's actual RTSP version. Many cameras implement RTSP 1.0, while RTSP 2.0 is a later protocol that is not backwards-compatible with 1.0 beyond version negotiation. Both control a media presentation, commonly with RTP and RTCP carrying media and reports, but their detailed session and transport requirements must not be mixed.

Validate the observed version, credentials, resource URL and session state. Record whether media uses negotiated UDP ports or RTP/RTCP interleaved with RTSP over TCP under that version and implementation. Then observe first RTP packet, sequence continuity, timestamps and track activity. A successful control response with no media points to a different boundary than an authentication rejection.

WebRTC adds signaling, ICE and protected real-time transport

The W3C WebRTC Recommendation defines browser APIs such as RTCPeerConnection, while the IETF WebRTC transport specification covers the real-time transport environment. An application still needs signaling to exchange descriptions and candidates.

Record ICE gathering, selected candidate pair, connection state, round-trip time, loss, jitter and received frames. Test direct and TURN-relayed cases from representative networks. “Connected” without inbound bytes or decoded frames is not a usable camera view.

Codec compatibility decides whether media can pass through

Inspect the exact RTSP video and audio payloads, including codec profile and parameters. RFC 7742 requires WebRTC endpoints to implement VP8 and H.264 Constrained Baseline, with H.264 packetization mode 1 support and negotiated profile-level-id. RFC 7874 defines Opus, PCMA and PCMU as the baseline WebRTC audio codecs. Actual use still depends on the offer, answer and endpoint implementation.

Choose separately for every track. Compatible encoded video may still need RTSP depayloading and WebRTC RTP repacketization without decoding pixels. A codec, profile, parameter or audio mismatch can require decode and encode. Transcoding adds compute, delay and a new failure boundary; repacketization adds protocol work but is not transcoding. Do not call the whole stream “passthrough” because one codec name appears on both sides.

Work one camera example from SDP to browser counters

Assume the RTSP DESCRIBE response identifies H.264 video and AAC audio. First parse the H.264 profile-level-id, packetization parameters and in-band parameter-set behaviour; compare them with the browser's WebRTC SDP. If the H.264 profile and parameters are accepted, the gateway may depayload and repacketize video into the negotiated WebRTC RTP form. If not, select a supported H.264 or VP8 transcode.

AAC is not one of RFC 7874's required WebRTC audio codecs, so the gateway cannot assume it will negotiate. It may transcode AAC to Opus, omit audio under an explicit policy, or use another mutually offered codec. Verification then follows the decision: RTSP packets arrive, the video repacketizer or encoder emits frames, the audio encoder emits Opus when selected, offer and answer agree on payload formats, and browser inbound and decoded-frame counters advance.

Timing and keyframes shape startup and recovery

A newly joined browser needs decodable media. Long keyframe intervals can delay the first picture or recovery after loss. The gateway may request or wait for an upstream keyframe, but camera behaviour varies. Measure time from session request to first decoded frame and observe what happens after a brief source interruption.

Preserve monotonic timestamps across the gateway and define how discontinuities are handled after an RTSP reconnect. Do not hide a frozen last frame behind a healthy WebRTC connection state; alert on frame advancement and media age.

Authorization exists on both sides of the gateway

Protect camera credentials and keep the source network private. Store secrets outside browser-visible URLs and redact them from logs. Record the security properties the actual RTSP 1.0 or 2.0 source and selected media transport provide; do not infer protected RTP from a protected control connection. On the viewer side, authorize the WebRTC session, constrain its lifetime and prevent one token from enumerating unrelated cameras.

TURN credentials, signaling endpoints and session identifiers also need rotation and rate limits. Decide how many peers may watch one source and whether the gateway shares an upstream RTSP session or opens one per viewer; that choice affects camera limits and isolation.

Capacity depends on camera sessions, transcoding and peer fan-out

Count upstream sessions, received bitrate, depayload and jitter buffers, decoder and encoder work, WebRTC peers, TURN relay traffic and per-peer state. A camera may limit concurrent RTSP clients, so a shared gateway ingest can be safer than allowing each viewer to open one.

Load-test the real codec and resolution. Add peers gradually, split direct and relayed connections, and record CPU, memory, outbound bitrate, encoder queue and first-frame time. Leave capacity for reconnect waves when a camera or gateway restarts.

Validate the gateway from camera to target browser

1. Probe the source. Save RTSP methods, selected transport, tracks, codec parameters, timestamps and keyframe cadence.
2. Choose per-track handling. Document pass-through, repacketization, transcode or omission and the reason.
3. Test WebRTC negotiation. Capture offer, answer, codec result, ICE candidates and selected path without exposing credentials.
4. Prove media. Observe upstream packets, gateway frames, outbound RTP, browser received bytes and decoded frames.
5. Exercise networks. Test direct and TURN-relayed viewers, packet loss, camera restart and gateway restart.
6. Check policy and scale. Deny an unauthorized viewer, expire a session and add peers up to the planned limit.