Building a Real-Time Video SFU in Node.js: Routing Multi-Stream WebRTC Media at Scale

{ “title”: “Building a Real-Time Video SFU in Node.js: Routing Multi-Stream WebRTC Media at Scale”, “summary”: “Learn how to build a scalable Selective Forwarding Unit (SFU) from scratch in Node.js using Mediasoup and WebSockets, complete with simulcast and bandwidth adaptation.”, “tags”: [“Node.js”, “WebRTC”, “Real-Time”, “Backend”, “Software Architecture”], “body”: “# Building a Real-Time Video SFU in Node.js: Routing Multi-Stream WebRTC Media at Scale\n\nWhen building real-time multi-party audio and video applications, developers inevitably hit the scaling wall of WebRTC. While peer-to-peer (P2P) mesh networks work wonderfully for 1-to-1 calls, trying to connect 5, 10, or 50 participants in a mesh quickly saturates CPU and bandwidth. Each peer must upload their video stream $N-1$ times and download $N-1$ streams. \n\nTo break through this limitation, modern real-time applications use a Selective Forwarding Unit (SFU) architecture. Instead of mixing streams together (like an MCU) or sending everything to everyone (Mesh), an SFU acts as a smart selective router. It ingests media streams from each client and selectively forwards them to other participants based on network conditions and subscriber requests.\n\nIn this technical guide, we will walk through building a production-grade WebRTC SFU gateway in Node.js using Mediasoup—one of the industry’s most powerful and performant WebRTC SFU engines—coupled with a WebSocket signaling layer.\n\n—\n\n## Understanding the SFU Architecture\n\nAn SFU does not decode and re-encode incoming media packets (unlike an MCU). This keeps CPU overhead remarkably low. Instead, it reads the Real-Time Transport Protocol (RTP) packets coming from a Producer and routes them directly to one or more Consumers.\n\n\n[Client A (Producer)] ---(RTP/WebRTC)---> [ Node.js SFU (Mediasoup) ]\n |\n +------------------------------+------------------------------+\n |\ |\n ---(RTP/WebRTC)--- ---(RTP/WebRTC)---\n v v\n [Client B (Consumer)] [Client C (Consumer)]\n\n\nOur system will consist of two primary parts:\n1. The Signaling Channel (WebSockets): Used by clients and the server to negotiate WebRTC parameters (SDP offers/answers, ICE candidates, track metadata).\n2. The Media Plane (Mediasoup / C++ Worker Nodes): Manages actual RTP/RTCP packet routing, simulcast layers, and bandwidth estimation.\n\n—\n\n## Project Setup and Dependencies\n\nFirst, initialize a Node.js project and install the required dependencies. We need mediasoup for media routing and ws for our signaling server.\n\nbash\nmkdir rtc-sfu-gateway\ncd rtc-sfu-gateway\nnpm init -y\nnpm install mediasoup ws express\nnpm install --save-dev nodemon\n\n\nMake sure your development environment has the necessary build tools installed (python3, make, g++), as mediasoup compiles native C++ modules upon installation to achieve maximum packet-forwarding performance.\n\n—\n\n## Initializing the Mediasoup Worker and Router\n\nMediasoup architecture is structured hierarchically:\n* Worker: A native C++ child process running on a specific CPU core that handles actual media processing.\n* Router: A logical entity inside a Worker that routes media from Producers to Consumers.\n* Transport: Encapsulates network connections (WebRTC or Plain RTP).\n* Producer: Represents an upstream media source (Webcam/Mic from a client).\n* Consumer: Represents a downstream media sink (Media stream sent to a client).\n\nLet’s write our server initialization script (server.js):\n\njavascript\nconst http = require('http');\nconst express = require('express');\nconst { WebSocketServer } = require('ws');\nconst mediasoup = require('mediasoup');\n\nconst app = express();\nconst server = http.createServer(app);\nconst wss = new WebSocketServer({ server });\n\nlet worker;\nlet router;\n\nasync function createWorker() {\n worker = await mediasoup.createWorker({\n rtcMinPort: 20000,\n rtcMaxPort: 40000,\n logLevel: 'warn',\n logTags: [\n 'info',\n 'ice',\n 'dtls',\n 'rtp',\n 'srtp',\n 'rtcp',\n ],\n });\n\n worker.on('died', () => {\n console.error('Mediasoup worker died, exiting in 2 seconds...');\n setTimeout(() => process.exit(1), 2000);\n });\n\n // Media Codecs supported by our SFU\n const mediaCodecs = [\n {\n kind: 'audio',\n mimeType: 'audio/opus',\n clockRate: 48000,\n channels: 2,\n },\n {\n kind: 'video',\n mimeType: 'video/VP8',\n clockRate: 90000,\n parameters: {\n 'x-google-start-bitrate': 1000,\n },\n },\n ];\n\n router = await worker.createRouter({ mediaCodecs });\n console.log(`Mediasoup worker created. Router ID: ${router.id}`);\n}\n\ncreateWorker().catch(console.error);\n\n\n—\n\n## Handling Signaling and Transports via WebSockets\n\nClients cannot stream media until they establish a WebRtcTransport. A transport represents the network path between the client browser and the Mediasoup worker.\n\nLet’s add client session management and WebRTC transport creation logic to our WebSocket server:\n\njavascript\n// Store peers and their associated transports/producers/consumers\nconst rooms = new Map(); // roomId -> { peers: Map(), router }\n\nwss.on('connection', (ws) => {\n let currentRoomId = null;\n let currentPeerId = null;\n\n ws.on('message', async (message) => {\n const data = JSON.parse(message);\n const { type, payload } = data;\n\n try {\n switch (type) {\n case 'JOIN_ROOM':\n currentRoomId = payload.roomId;\n currentPeerId = payload.peerId;\n\n if (!rooms.has(currentRoomId)) {\n const router = await worker.createRouter({\n mediaCodecs: router.observer.mediaCodecs || router.mediaCodecs\n });\n rooms.set(currentRoomId, { router, peers: new Map() });\n }\n\n const room = rooms.get(currentRoomId);\n room.peers.set(currentPeerId, { ws, transports: new Map(), producers: new Map(), consumers: new Map() });\n\n ws.send(JSON.stringify({\n type: 'JOINED_ROOM',\n payload: { routerRtpCapabilities: room.router.rtpCapabilities }\n }));\n break;\n\n case 'CREATE_WEBRTC_TRANSPORT':\n const activeRoom = rooms.get(currentRoomId);\n const transport = await createWebRtcTransport(activeRoom.router);\n \n const peer = activeRoom.peers.get(currentPeerId);\n peer.transports.set(transport.id, transport);\n\n ws.send(JSON.stringify({\n type: 'WEBRTC_TRANSPORT_CREATED',\n payload: {\n id: transport.id,\n iceParameters: transport.iceParameters,\n iceCandidates: transport.iceCandidates,\n dtlsParameters: transport.dtlsParameters,\n }\n }));\n break;\n\n case 'CONNECT_WEBRTC_TRANSPORT':\n {\n const { transportId, dtlsParameters } = payload;\n const peer = rooms.get(currentRoomId).peers.get(currentPeerId);\n const transport = peer.transports.get(transportId);\n await transport.connect({ dtlsParameters });\n ws.send(JSON.stringify({ type: 'TRANSPORT_CONNECTED' }));\n }\n break;\n\n case 'PRODUCE':\n {\n const { transportId, kind, rtpParameters } = payload;\n const peer = rooms.get(currentRoomId).peers.get(currentPeerId);\n const transport = peer.transports.get(transportId);\n \n const producer = await transport.produce({ kind, rtpParameters });\n peer.producers.set(producer.id, producer);\n\n producer.on('transportclose', () => {\n producer.close();\n peer.producers.delete(producer.id);\n });\n\n // Broadcast new producer to other peers in the room\n broadcastToRoom(currentRoomId, currentPeerId, {\n type: 'NEW_PRODUCER',\n payload: { producerId: producer.id, peerId: currentPeerId, kind }\n });\n\n ws.send(JSON.stringify({ type: 'PRODUCED', payload: { id: producer.id } }));\n }\n break;\n\n case 'CONSUME':\n {\n const { transportId, producerId, rtpCapabilities } = payload;\n const activeRoom = rooms.get(currentRoomId);\n const peer = activeRoom.peers.get(currentPeerId);\n const transport = peer.transports.get(transportId);\n\n if (activeRoom.router.canConsume({ producerId, rtpCapabilities })) {\n const consumer = await transport.consume({\n producerId,\n rtpCapabilities,\n paused: true,\n });\n\n peer.consumers.set(consumer.id, consumer);\n\n consumer.on('transportclose', () => {\n consumer.close();\n peer.consumers.delete(consumer.id);\n });\n\n consumer.on('producerclose', () => {\n consumer.close();\n peer.consumers.delete(consumer.id);\n ws.send(JSON.stringify({ type: 'CONSUMER_CLOSED', payload: { consumerId: consumer.id } }));\n });\n\n ws.send(JSON.stringify({\n type: 'CONSUMED',\n payload: {\n id: consumer.id,\n producerId,\n kind: consumer.kind,\n rtpParameters: consumer.rtpParameters,\n }\n }));\n\n // Resume consumer after client acknowledges\n await consumer.resume();\n }\n }\n break;\n }\n } catch (error) {\n console.error(`Error handling message type ${type}:`, error);\n ws.send(JSON.stringify({ type: 'ERROR', payload: { message: error.message } }));\n }\n });\n});\n\nasync function createWebRtcTransport(router) {\n const transport = await router.createWebRtcTransport({\n listenIps: [\n { ip: '0.0.0.0', announcedIp: '127.0.0.1' } // Replace with your public IP in production\n ],\n enableUdp: true,\n enableTcp: true,\n preferUdp: true,\n });\n\n return transport;\n}\n\nfunction broadcastToRoom(roomId, senderId, message) {\n const room = rooms.get(roomId);\n if (!room) return;\n \n for (const [peerId, peer] of room.peers.entries()) {\n if (peerId !== senderId) {\n peer.ws.send(JSON.stringify(message));\n }\n }\n}\n\nserver.listen(3000, () => {\n console.log('SFU Gateway running on port 3000');\n});\n\n\n—\n\n## Implementing Simulcast for Bandwidth Adaptation\n\nIn real-world networks, participants have wildly varying upload and download speeds. If an SFU blindly forwards a 1080p stream to a mobile user on a congested 4G connection, their stream will freeze.\n\nTo prevent this, we configure our Producers to send Simulcast streams. Simulcast allows an encoding client to send three simultaneous versions of the same video stream:\n1. High Quality (HD): Original resolution and bitrate.\n2. Medium Quality (Quarter resolution): Scaled down version for standard layouts.\n3. Low Quality (Thumbnail): Minimal bandwidth variant.\n\nWhen a consumer requests a stream, the SFU decides which spatial layer to send based on the consumer’s available bandwidth. \n\n### Enabling Simulcast on the Client Producer\n\nWhen creating a WebRTC video producer on the client side using mediasoup-client, you specify encodings:\n\njavascript\nconst encodings = [\n { maxBitrate: 100000, scaleResolutionDownBy: 4, scalabilityMode: 'L1T3' }, // Low\n { maxBitrate: 500000, scaleResolutionDownBy: 2, scalabilityMode: 'L1T3' }, // Medium\n { maxBitrate: 1500000, scaleResolutionDownBy: 1, scalabilityMode: 'L1T3' } // High\n];\n\nconst videoProducer = await transport.produce({\n track: videoTrack,\n encodings,\n codecOptions: {\n videoGoogleStartBitrate: 1000\n }\n});\n\n\n### Dynamically Switching Spatial Layers on the Consumer\n\nMediasoup allows you to change the preferred spatial layer of a consumer on the fly. For instance, if a user expands a video feed to full-screen, the client can request the high-quality layer (spatialLayer: 2). If they minimize it or experience packet loss, the SFU drops them down to layer 0 or 1.\n\nAdd a handler in your Node.js signaling router for layer switching:\n\njavascript\ncase 'SET_PREFERRED_LAYERS':\n {\n const { consumerId, spatialLayer, temporalLayer } = payload;\n const peer = rooms.get(currentRoomId).peers.get(currentPeerId);\n const consumer = peer.consumers.get(consumerId);\n \n if (consumer) {\n await consumer.setPreferredLayers({ spatialLayer, temporalLayer });\n ws.send(JSON.stringify({ type: 'LAYERS_SET', payload: { consumerId, spatialLayer } }));\n }\n }\n break;\n\n\n—\n\n## Managing Network Adaptation and Congestion Control\n\nWebRTC handles congestion control via Transport-CC (Congestion Control) and REMB (Receiver Estimated Maximum Bitrate) feedback packets. Mediasoup continuously analyzes incoming and outgoing RTP RTCP Receiver Reports (RR) to calculate round-trip time (RTT) and packet loss.\n\nWhen packet loss spikes on a consumer transport:\n1. Mediasoup automatically notifies the sender via RTCP to reduce its encoder bitrate.\n2. Alternatively, the SFU can selectively drop non-critical temporal layers or switch consumers to lower simulcast spatial tracks.\n\nYou can monitor these statistics programmatically inside your Node.js application for logging and auto-scaling decisions:\n\njavascript\n// Periodically check transport stats for analytics or adaptive logic\nsetInterval(async () => {\n for (const [roomId, room] of rooms.entries()) {\n for (const [peerId, peer] of room.peers.entries()) {\n for (const [transportId, transport] of peer.transports.entries()) {\n try {\n const stats = await transport.getStats();\n // stats contains bytesReceived, bytesSent, rtt, packetLoss, etc.\n } catch (err) {\n // Transport might be closing/closed\n }\n }\n }\n }\n}, 10000);\n\n\n—\n\n## Production Scaling Considerations\n\nRunning an SFU in production requires careful planning beyond a single Node.js process:\n\n> CPU Bound Media Routing: While Node.js handles the signaling layer asynchronously using event loops, Mediasoup workers run as native C++ worker processes pinned to specific CPU cores. Ensure your server provisioning maps workers 1:1 with available CPU cores.\n\n* Horizontal Scaling Across Multiple VMs: To scale beyond a single machine, you cannot simply put a standard round-robin load balancer in front of WebRTC ports. You need an architecture where clients connect to specific worker nodes, using Redis or a message broker to coordinate room metadata across instances.\n* UDP Port Ranges: Ensure your cloud provider’s security groups / firewalls allow UDP traffic over your configured rtcMinPort to rtcMaxPort range (e.g., 20000-40000).\n* TURN/STUN Servers: Real-world users sit behind symmetric NATs and corporate firewalls. Deploy a high-performance TURN server (like Coturn) alongside your SFU and provide iceServers configuration when creating transports.\n\n—\n\n## Conclusion\n\nBy migrating from a P2P mesh architecture to a custom Node.js and Mediasoup SFU gateway, you unlock the ability to scale real-time audio and video applications to dozens—and with proper clustering, hundreds—of concurrent participants per room.\n\nThrough this guide, you’ve implemented:\n* A modular WebRTC signaling infrastructure via WebSockets.\n* Media transport negotiation and routing pipelines with Mediasoup.\n* Simulcast encoding layers for high-definition and bandwidth-constrained participants.\n\nWith this foundation, you can extend your SFU gateway to include features like active speaker detection, server-side recording, and adaptive layout mixing.\n”}