Building a Custom Load Balancer in Node.js: Round-Robin Routing, Health Checks, and Failover

Modern distributed systems rely heavily on load balancers to distribute incoming traffic across a pool of backend servers. While tools like Nginx, HAProxy, and AWS ALB are industry standards, understanding how a load balancer works under the hood is an invaluable skill for any backend engineer.

1. Architecture Overview

A Layer-7 load balancer operates at the application layer of the OSI model. It intercepts incoming HTTP/HTTPS requests from clients, evaluates routing policies, and forwards them to one of several backend application servers.

Our custom load balancer will consist of three core components:

1. The Reverse Proxy Core: Uses Node’s native http.createServer and http.request to stream request and response data between clients and backends.
2. The Routing Engine: Implements a thread-safe Round-Robin algorithm to cycle requests through healthy backend instances.
3. The Health Check Daemon: Periodically pings backend servers in the background, dynamically updating their availability status to prevent routing traffic to dead nodes.

—|

2. Setting Up the Project and Configuration

Let’s initialize a fresh Node.js project. We won’t need any external dependencies since we are crafting this using core libraries.

mkdir node-load-balancer
cd node-load-balancer
npm init -y

Create a file named balancer.js. Let’s define our configuration structure, including our backend server pools and health-check parameters.

const http = require('http');
const { URL } = require('url');

// Configuration
const PORT = 8080;
const HEALTH_CHECK_INTERVAL = 10000; // 10 seconds
const REQUEST_TIMEOUT = 5000; // 5 seconds

// Define our backend server pool
const servers = [
  { url: 'http://localhost:3001', healthy: true, activeConnections: 0 },
  { url: 'http://localhost:3002', healthy: true, activeConnections: 0 },
  { url: 'http://localhost:3003', healthy: true, activeConnections: 0 }
];

—|

3. Implementing Round-Robin Routing

Round-robin is the simplest yet remarkably effective load balancing strategy. It cycles sequentially through the list of available servers. To make it dynamic and resilient, our routing function must skip any servers marked as unhealthy.

let currentIndex = 0;

function getNextHealthyServer() {
  const initialIndex = currentIndex;
  
  while (true) {
    const server = servers[currentIndex];
    // Increment index for the next request (wrap around using modulo)
    currentIndex = (currentIndex + 1) % servers.length;

    if (server.healthy) {
      return server;
    }

    // If we've checked every server and none are healthy, return null
    if (currentIndex === initialIndex) {
      return null;
    }
  }
}

Why check for healthy nodes?

If a backend crashes, a naive round-robin implementation would continue sending 33% of traffic to the dead server, resulting in connection timeouts for users. By skipping unhealthy instances, we maintain system availability.

—|

4. Building the Reverse Proxy Engine

Node.js is uniquely suited for building proxies because of its non-blocking I/O and stream-based architecture. Instead of buffering entire request bodies in memory (which causes high memory footprints and latency), we pipe the incoming client request directly to the backend server, and pipe the backend response back to the client.

function handleProxy(req, res) {
  const targetServer = getNextHealthyServer();

  if (!targetServer) {
    res.writeHead(503, { 'Content-Type': 'text/plain' });
    res.end('Service Unavailable: No healthy backends available.');
    return;
  }

  const targetUrl = new URL(targetServer.url);

  // Options for outgoing proxy request
  const options = {
    hostname: targetUrl.hostname,
    port: targetUrl.port,
    path: req.url,
    method: req.method,
    headers: {
      ...req.headers,
      'X-Forwarded-For': req.socket.remoteAddress,
      'X-Proxy-By': 'NodeJS-Custom-LB'
    }
  };

  targetServer.activeConnections++;

  const proxyReq = http.request(options, (proxyRes) => {
    // Forward status code and headers from backend to client
    res.writeHead(proxyRes.statusCode, proxyRes.headers);
    
    // Pipe response stream
    proxyRes.pipe(res, { end: true });
    
    proxyRes.on('end', () => {
      targetServer.activeConnections--;
    });
  });

  proxyReq.on('error', (err) => {
    console.error(`[Proxy Error] Failed to reach backend ${targetServer.url}: ${err.message}`);
    targetServer.activeConnections--;
    
    // Trigger immediate failover state for this server
    targetServer.healthy = false;

    // Retry request with another server
    handleProxy(req, res);
  });

  // Timeout handling
  proxyReq.setTimeout(REQUEST_TIMEOUT, () => {
    console.error(`[Timeout] Backend ${targetServer.url} timed out.`);
    proxyReq.destroy();
    targetServer.healthy = false;
  });

  // Pipe client request stream into proxy request
  req.pipe(proxyReq, { end: true });
}

// Start the load balancer server
const server = http.createServer(handleProxy);
server.listen(PORT, () => {
  console.log(`Load balancer running on port ${PORT}`);
});

—|

5. Active Health Checks and Automatic Failover

Passive health checks (handling errors when requests fail) are helpful, but they mean the first user hitting a dead server experiences an error. Active health checks run continuously in the background, proactively pinging an endpoint (e.g., /health) on each backend server.

Let’s implement a background daemon that periodically evaluates our server pool.

function checkServerHealth(server) {
  const targetUrl = new URL(`${server.url}/health`);

  const req = http.get({
    hostname: targetUrl.hostname,
    port: targetUrl.port,
    path: targetUrl.pathname,
    timeout: 3000
  }, (res) => {
    if (res.statusCode === 200) {
      if (!server.healthy) {
        console.log(`[Recovery] Backend recovered: ${server.url}`);
        server.healthy = true;
      }
    } else {
      if (server.healthy) {
        console.warn(`[Unhealthy] Backend returned status ${res.statusCode}: ${server.url}`);
        server.healthy = false;
      }
    }
    res.resume(); // Consume response data to free up memory
  });

  req.on('error', (err) => {
    if (server.healthy) {
      console.warn(`[Unhealthy] Backend failed health check: ${server.url} - ${err.message}`);
      server.healthy = false;
    }
  });

  req.on('timeout', () => {
    if (server.healthy) {
      console.warn(`[Unhealthy] Backend health check timed out: ${server.url}`);
      server.healthy = false;
    }
    req.destroy();
  });
}

// Run health checks periodically
setInterval(() => {
  console.log('Running active health checks...');
  servers.forEach(checkServerHealth);
}, HEALTH_CHECK_INTERVAL);

—|

6. Testing the Load Balancer

To verify our implementation works, we can spin up a few trivial HTTP servers to act as our backend pool.

Create a file named backend.js:

const http = require('http');
const port = process.argv[2] || 3001;

const server = http.createServer((req, res) => {
  if (req.url === '/health') {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('OK');
    return;
  }
  
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({
    message: 'Hello from backend instance',
    port: port,
    timestamp: new Date()
  }));
});

server.listen(port, () => {
  console.log(`Backend server listening on port ${port}`);
});

Running the Test

1. Open three terminal windows and start three backends:

   node backend.js 3001
   node backend.js 3002
   node backend.js 3003
   

2. In another terminal, start your load balancer:

   node balancer.js
   

3. Send requests to http://localhost:8080 using curl or Apache Bench:

   curl http://localhost:8080
   

   You will see responses cycling seamlessly through ports 3001, 3002, and 3003.
4. Kill one of the backend servers (e.g., stop port 3002). Notice how the load balancer instantly removes it from rotation via active health checks and passive failover, routing 100% of subsequent traffic exclusively to the remaining healthy nodes without dropping client connections.

—|

7. Production Considerations & Next Steps

While our custom load balancer handles routing, failover, and health checks cleanly, taking a proxy to production requires handling several edge cases:

• Sticky Sessions (Session Persistence): If your application stores session state locally in memory rather than a distributed cache like Redis, you can hash the client’s IP address (req.socket.remoteAddress) to pin them to a specific backend server.
• Least-Connections Algorithm: Instead of rigid round-robin, route traffic to the server with the lowest activeConnections count to optimize for heterogeneous backend performance.
• HTTPS / TLS Termination: Add SSL certificate handling using Node’s https module or terminate TLS at a preceding network layer.
• Cluster Mode: Utilize Node’s cluster module or PM2 to scale the load balancer across multiple CPU cores to handle massive network throughput.

Conclusion

By leveraging Node.js’s native event loop, stream piping capabilities, and core networking libraries, building a resilient Layer-7 load balancer takes less than 150 lines of code. Understanding these foundational blocks empowers you to debug complex networking bottlenecks and custom-tailor routing algorithms for your microservices architecture.