Building a Real-Time Collaborative Code Execution Sandbox in Node.js with Docker and WebSockets

Allowing users to write and execute code directly in their browsers is a powerful feature for modern educational platforms, coding interview tools, and online IDEs. However, running arbitrary, untrusted code on a server is an immense security risk. A malicious user could easily spawn fork bombs, attempt to read sensitive environment variables, or compromise the host system.

In this architectural guide, we will build a robust, production-grade code execution sandbox. We will combine Node.js, WebSockets (via ws), and Docker to safely execute code in isolated containers, stream stdout and stderr back to the client in real time, and enforce strict memory and CPU limits with timeout controls.

System Architecture Overview

Our sandbox architecture relies on three primary components working in tandem:

1. The WebSocket Server (Node.js): Acts as the central coordinator. It accepts client connections, authenticates requests, spawns ephemeral Docker containers, and streams process output back over the socket.
2. The Docker Engine: Provides OS-level virtualization using Linux cgroups and namespaces. Each code execution request spins up a transient, isolated container with read-only filesystems and dropped privileges.
3. The Client (Browser): Connects to the WebSocket server, sends source code and configuration (language/stdin), and renders output streams in real time.

+--------+      WebSocket      +-------------------+      Docker API      +-------------------+
| Client | <-----------------> | Node.js WebSocket | <------------------> | Ephemeral Docker  |
|        |   (stdout/stdin)    |      Server       |    (Container)     |     Container     |
+--------+                     +-------------------+                      +-------------------+

Step 1: Preparing the Docker Execution Image

Before writing Node.js code, we need a lightweight Docker image containing the runtimes we want to support (e.g., Python, Node.js). For this guide, we will focus on executing Python scripts.

Create a Dockerfile.python in your project root:

FROM python:3.11-slim

# Create a non-privileged user to run the code
RUN useradd -ms /bin/bash sandboxuser
USER sandboxuser
WORKDIR /home/sandboxuser

# Disable Python output buffering for real-time streaming
ENV PYTHONUNBUFFERED=1

Build the image locally:

docker build -t code-sandbox-python:latest -f Dockerfile.python .

Step 2: Setting up the Node.js WebSocket Server

Initialize a new Node.js project and install the required dependencies: ws for WebSockets and dockerode for interacting with the Docker Daemon API.

npm init -y
npm install ws dockerode

Create server.js and set up the WebSocket server structure:

const WebSocket = require('ws');
const Docker = require('dockerode');
const { Duplex } = require('stream');

const docker = new Docker();
const wss = new WebSocket.Server({ port: 8080 });

console.log('WebSocket sandbox server running on ws://localhost:8080');

wss.on('connection', (ws) => {
  console.log('Client connected.');

  ws.on('message', async (message) => {
    try {
      const data = JSON.parse(message);
      const { code, language } = data;

      if (language !== 'python') {
        ws.send(JSON.stringify({ type: 'error', data: 'Unsupported language.' }));
        return;
      }

      await executePythonCode(code, ws);
    } catch (err) {
      ws.send(JSON.stringify({ type: 'error', data: err.message }));
    }
  });

  ws.on('close', () => {
    console.log('Client disconnected.');
  });
});

Step 3: Implementing Secure Container Execution

This is the core of our sandbox. When a request arrives, we must:

1. Create a container from our base image.
2. Attach to its stdout/stderr streams so we can pipe data back to the WebSocket.
3. Enforce strict resource limits (Memory, CPU shares, network isolation).
4. Handle execution timeouts to prevent infinite loops.
5. Ensure cleanup (container.remove()) occurs regardless of success, failure, or timeout.

Add the executePythonCode function to server.js:

async function executePythonCode(code, ws) {
  let container;
  const TIMEOUT_MS = 5000; // 5 seconds max execution time

  try {
    // 1. Create the container with strict security settings
    container = await docker.createContainer({
      Image: 'code-sandbox-python:latest',
      Cmd: ['python3', '-c', code],
      Tty: false,
      // Security hardening
      NetworkDisabled: true, // Disable network access
      HostConfig: {
        Memory: 128 * 1024 * 1024, // 128 MB RAM limit
        MemorySwap: 128 * 1024 * 1024, // Disable swap
        CpuQuota: 50000, // 50% of a single CPU core (Quota/Period = 50000/100000)
        CpuPeriod: 100000,
        AutoRemove: false,
      },
    });

    // 2. Attach to stdout and stderr streams
    const stream = await container.attach({
      stream: true,
      stdout: true,
      stderr: true,
    });

    // Docker multiplexes stdout and stderr into a single stream with an 8-byte header.
    // We use docker.modem.demuxStream to split them cleanly.
    const stdoutStream = new Duplex({
      read() {},
      write(chunk, encoding, callback) {
        ws.send(JSON.stringify({ type: 'stdout', data: chunk.toString() }));
        callback();
      }
    });

    const stderrStream = new Duplex({
      read() {},
      write(chunk, encoding, callback) {
        ws.send(JSON.stringify({ type: 'stderr', data: chunk.toString() }));
        callback();
      }
    });

    docker.modem.demuxStream(stream, stdoutStream, stderrStream);

    // 3. Start the container
    await container.start();
    ws.send(JSON.stringify({ type: 'status', data: 'Execution started...' }));

    // 4. Handle timeouts and container completion using a race condition
    const waitPromise = container.wait();
    
    let timeoutHandle;
    const timeoutPromise = new Promise((_, reject) => {
      timeoutHandle = setTimeout(async () => {
        try {
          await container.stop({ t: 0 });
          reject(new Error('Execution timed out (exceeded 5s limit).'));
        } catch (e) {
          // Container might have already stopped
        }
      }, TIMEOUT_MS);
    });

    await Promise.race([waitPromise, timeoutPromise]);
    clearTimeout(timeoutHandle);

    ws.send(JSON.stringify({ type: 'status', data: 'Execution finished.' }));

  } catch (err) {
    ws.send(JSON.stringify({ type: 'error', data: err.message }));
  } finally {
    // 5. Cleanup container resources
    if (container) {
      try {
        await container.remove({ force: true });
      } catch (cleanupErr) {
        console.error('Failed to clean up container:', cleanupErr.message);
      }
    }
  }
}

Step 4: Building the Client Interface

To test our WebSocket sandbox, create a simple index.html file that connects to the server, sends Python code, and displays the real-time output streams.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Code Sandbox Client</title>
    <style>
        body { font-family: monospace; background: #1e1e1e; color: #d4d4d4; padding: 20px; }
        textarea { width: 100%; height: 200px; background: #252526; color: #d4d4d4; border: 1px solid #333; padding: 10px; }
        pre { background: #252526; padding: 15px; border: 1px solid #333; height: 150px; overflow-y: auto; }
        button { padding: 10px 20px; background: #0e639c; color: white; border: none; cursor: pointer; margin-top: 10px; }
        button:hover { background: #1177bb; }
        .stderr { color: #f44747; }
        .status { color: #4ec9b0; }
    </style>
</head>
<body>
    <h2>Python Sandbox Terminal</h2>
    <textarea id="code">print("Hello from the sandbox!")

# Test infinite loop timeout protection
# while True:
#     pass
</textarea>
    <br>
    <button onclick="runCode()">Run Code</button>

    <h3>Output:</h3>
    <pre id="output"></pre>

    <script>
        const ws = new WebSocket('ws://localhost:8080');
        const outputEl = document.getElementById('output');

        ws.onmessage = (event) => {
            const msg = JSON.parse(event.data);
            const span = document.createElement('span');
            
            if (msg.type === 'stderr') span.className = 'stderr';
            if (msg.type === 'status') span.className = 'status';

            span.textContent = msg.data;
            outputEl.appendChild(span);
            outputEl.scrollTop = outputEl.scrollHeight;
        };

        function runCode() {
            outputEl.innerHTML = '';
            const code = document.getElementById('code').value;
            ws.send(JSON.stringify({ language: 'python', code }));
        }
    </script>
</body>
</html>

Step 5: Hardening and Production Best Practices

Running code execution services in production requires layers of defense-in-depth beyond basic containerization. Consider implementing the following strategies before launching to users:

Important Security Advisory: Never expose the Docker daemon socket (/var/run/docker.sock) directly to untrusted web applications without authentication, rate limiting, and resource isolation. A compromised Node.js server with access to the Docker socket can easily escape containment by mounting host volumes.

• Resource Quotas & Scaling: Use container orchestration tools like Kubernetes with ResourceQuota limits, or deploy worker pools (via Redis queues like BullMQ) to queue execution requests if concurrent container creation overwhelms the host CPU.
• Seccomp and AppArmor Profiles: Restrict system calls inside the container by applying custom Docker security profiles to block dangerous syscalls (ptrace, kexec, etc.).
• Ephemeral Storage: Ensure containers have no persistent storage attached. Any data written to the container filesystem should be destroyed immediately upon container removal.
• Rate Limiting: Implement token-bucket rate limiting per IP address or user account on the WebSocket endpoint to prevent Denial-of-Service (DoS) attacks via rapid container spawning.

Conclusion

By pairing Node.js and WebSockets for real-time bidirectional communication with Docker for OS-level isolation, you can build a secure, performant code execution sandbox. Using Docker’s native Memory, CpuQuota, and container streams allows you to capture live output while strictly enforcing timeouts and guardrails against malicious scripts.