Local-First Architecture: Building Bulletproof Offline-First Sync with CRDTs and IndexedDB

For decades, the web has operated on a cloud-first model: your client is a thin pane of glass, and every keystroke, click, and state change relies on a round-trip to a remote server. When the network drops, your app shows a dreary dinosaur or an endless loading spinner.

Local-first software flips this script. The canonical source of truth lives on your device, stored in local persistence layers like IndexedDB. The app works instantly, completely offline, and synchronizes in the background when connectivity returns.

Yet, building this utopian developer experience from scratch can quickly drive you mad. How do you handle concurrent edits from two different offline devices? How do you merge conflicting data without writing complex, bug-prone three-way merge algorithms?

In this guide, we will walk through building a resilient, local-first sync engine from scratch using Conflict-Free Replicated Data Types (CRDTs) and IndexedDB, keeping our sanity intact along the way.

The Core Architectural Pillars

A production-grade local-first engine relies on three foundational layers:

1. The Storage Layer: High-performance, structured local persistence (IndexedDB) capable of holding application state and CRDT operation logs.
2. The Consensus Layer: Conflict-Free Replicated Data Types (CRDTs) that guarantee mathematical convergence. No matter what order updates arrive in, every peer eventually reaches the exact same state.
3. The Transport Layer: A synchronization worker that gossips state or operation logs with a backend relay or peer-to-peer network whenever connectivity allows.

+-------------------------------------------------------------+
|                        Client App                           |
+------------------------------+------------------------------+
                               |
         +---------------------+---------------------+
         | (State Reads/Writes)                      | (Changes)
         v                                           v
+------------------+                       +------------------+
|    IndexedDB     |                       |   CRDT Engine    | 
|  (Local Storage) |                       | (State & Merges) |
+------------------+                       +------------------+
                                                     |
                                          (Sync)     |
                                                     v
                                           +-----------------+
                                           | WebSocket/Peer  | 
                                           |     Relay       |
                                           +-----------------+

Step 1: Setting up IndexedDB for CRDT Operations

Native IndexedDB APIs are notoriously verbose, callback-heavy, and error-prone. To maintain our sanity, we will use a lightweight wrapper or native promises. Crucially, our schema must store two distinct things: our current materialized state (for fast UI reads) and our operation log / state vector (for CRDT synchronization).

Let’s initialize our database using standard modern JavaScript:

// db.js
const DB_NAME = 'LocalFirstEngine';
const DB_VERSION = 1;

export function openDatabase() {
  return new Promise((resolve, reject) => {
    const request = indexedDB.open(DB_NAME, DB_VERSION);

    request.onerror = () => reject(request.error);
    request.onsuccess = () => resolve(request.result);

    request.onupgradeneeded = (event) => {
      const db = event.target.result;

      // Store for materialized application state
      if (!db.objectStoreNames.contains('documents')) {
        db.createObjectStore('documents', { keyPath: 'id' });
      }

      // Store for raw CRDT updates/patches to be synced
      if (!db.objectStoreNames.contains('sync_log')) {
        const logStore = db.createObjectStore('sync_log', { 
          keyPath: 'sequenceId',
          autoIncrement: true 
        });
        logStore.createIndex('by-synced', 'synced', { unique: false });
      }
    };
  });
}

Step 2: Implementing Conflict-Free State with CRDTs

Instead of attempting to resolve conflicts via server-wins or client-wins timestamp overwrites (which inevitably result in lost data), we use a State-based CRDT (CvRDT) or Operation-based CRDT (CmRDT).

For a general-purpose document or text store, a Last-Write-Wins Register (LWW-Register) or an array-based sequence CRDT is standard. Let’s implement a simplified LWW-Map CRDT where every field update carries a Lamport timestamp or hybrid logical clock (HLC) and a node ID.

// crdt.js
export class LWWMap {
  constructor(nodeId, initialState = {}) {
    this.nodeId = nodeId;
    // Values take the shape: { value, timestamp, nodeId }
    this.store = new Map(Object.entries(initialState));
  }

  set(key, value) {
    const timestamp = Date.now();
    this.store.set(key, {
      value,
      timestamp,
      nodeId: this.nodeId
    });
    return { key, value, timestamp, nodeId: this.nodeId };
  }

  get(key) {
    const entry = this.store.get(key);
    return entry ? entry.value : undefined;
  }

  // Merge incoming remote state or operation
  merge(remoteEntries) {
    for (const [key, remoteEntry] of Object.entries(remoteEntries)) {
      const localEntry = this.store.get(key);

      if (!localEntry) {
        this.store.set(key, remoteEntry);
      } else if (remoteEntry.timestamp > localEntry.timestamp) {
        this.store.set(key, remoteEntry);
      } else if (remoteEntry.timestamp === localEntry.timestamp) {
        // Tie-breaker: lexicographical comparison of Node IDs
        if (remoteEntry.nodeId > localEntry.nodeId) {
          this.store.set(key, remoteEntry);
        }
      }
    }
  }

  toJSON() {
    const obj = {};
    for (const [k, v] of this.store.entries()) {
      obj[k] = v;
    }
    return obj;
  }
}

Step 3: Bridging CRDTs and IndexedDB

Now, we combine our storage layer with our CRDT logic. Whenever a user makes an edit, we update the CRDT in memory, persist the materialized state to IndexedDB, and append the operation to our sync_log store.

// engine.js
import { openDatabase } from './db.js';
import { LWWMap } from './crdt.js';

export class SyncEngine {
  constructor(nodeId) {
    this.nodeId = nodeId;
    this.crdt = new LWWMap(nodeId);
    this.db = null;
  }

  async init() {
    this.db = await openDatabase();
    await this.loadFromStorage();
  }

  async loadFromStorage() {
    const tx = this.db.transaction('documents', 'readonly');
    const store = tx.objectStore('documents');
    const request = store.get('current_state');

    request.onsuccess = () => {
      if (request.result && request.result.data) {
        this.crdt = new LWWMap(this.nodeId, request.result.data);
      }
    };
  }

  async mutate(key, value) {
    // 1. Apply to CRDT
    const op = this.crdt.set(key, value);

    // 2. Persist state and log transactionally
    const tx = this.db.transaction(['documents', 'sync_log'], 'readwrite');
    
    tx.objectStore('documents').put({
      id: 'current_state',
      data: this.crdt.toJSON()
    });

    tx.objectStore('sync_log').add({
      ...op,
      synced: false,
      createdAt: Date.now()
    });

    return new Promise((resolve, reject) => {
      tx.oncomplete = () => resolve(op);
      tx.onerror = () => reject(tx.error);
    });
  }
}

Step 4: Handling Network Partitions & Syncing

Network connectivity is fickle. A robust local-first engine must handle sudden offline transitions, reconnections, and backpressure gracefully without locking up the UI thread.

We build a background sync loop using WebSockets (or WebRTC) that monitors network status, reads unsynced operations from our sync_log, pushes them to the server, and pulls remote updates.

// sync-worker.js
export class NetworkSyncManager {
  constructor(engine, serverUrl) {
    this.engine = engine;
    this.serverUrl = serverUrl;
    this.socket = null;
    this.isSyncing = false;
  }

  connect() {
    this.socket = new WebSocket(this.serverUrl);

    this.socket.onopen = () => {
      console.log('[Sync] Connected to relay. Flushing local log...');
      this.flushOutbox();
    };

    this.socket.onmessage = async (event) => {
      const remoteOps = JSON.parse(event.data);
      await this.handleRemoteOps(remoteOps);
    };

    this.socket.onclose = () => {
      console.log('[Sync] Connection lost. Retrying in 5s...');
      setTimeout(() => this.connect(), 5000);
    };
  }

  async flushOutbox() {
    if (this.isSyncing) return;
    this.isSyncing = true;

    try {
      const tx = this.engine.db.transaction('sync_log', 'readonly');
      const index = tx.objectStore('sync_log').index('by-synced');
      const request = index.getAll(IDBKeyRange.only(false));

      request.onsuccess = async () => {
        const unsyncedOps = request.result;
        if (unsyncedOps.length === 0) {
          this.isSyncing = false;
          return;
        }

        // Push batch to server
        this.socket.send(JSON.stringify(unsyncedOps));
        // Mark local entries as synced (omitted for brevity in transaction handling)
      };
    } catch (err) {
      console.error('[Sync] Failed to flush outbox:', err);
    } finally {
      this.isSyncing = false;
    }
  }

  async handleRemoteOps(remoteOps) {
    // Merge incoming remote state into our CRDT
    const remoteMap = {};
    remoteOps.forEach(op => {
      remoteMap[op.key] = { value: op.value, timestamp: op.timestamp, nodeId: op.nodeId };
    });

    this.engine.crdt.merge(remoteMap);

    // Save converged state
    const tx = this.engine.db.transaction('documents', 'readwrite');
    tx.objectStore('documents').put({
      id: 'current_state',
      data: this.engine.crdt.toJSON()
    });

    // Trigger UI update event
    window.dispatchEvent(new CustomEvent('crdt-state-updated', {
      detail: this.engine.crdt.toJSON()
    }));
  }
}

Step 5: Optimizing IndexedDB and Avoiding Storage Limits

Browsers enforce strict storage limits on IndexedDB (often a percentage of total available disk space, typically capped around origin quotas). If your app logs every single keystroke indefinitely, users will eventually hit quota errors and experience data corruption.

The Compaction Strategy

To prevent infinite log growth without losing historical sync guarantees, we implement Log Compaction (or snapshotting):

1. Periodically (e.g., upon successful sync confirmation with the server), the client establishes a high-water mark sequence ID.
2. All operations older than the watermark whose states are fully materialized in the current document snapshot are safe to prune.
3. We write a new compacted snapshot and delete old log entries in a single atomic IndexedDB transaction.

async function compactSyncLog(db, upToSequenceId) {
  const tx = db.transaction('sync_log', 'readwrite');
  const store = tx.objectStore('sync_log');

  // Delete records up to the confirmed synced watermark
  const range = IDBKeyRange.upperBound(upToSequenceId);
  const request = store.delete(range);

  return new Promise((resolve, reject) => {
    tx.oncomplete = () => {
      console.log(`[Storage] Successfully compacted sync log up to ID ${upToSequenceId}`);
      resolve();
    };
    tx.onerror = () => reject(tx.error);
  });
}

Conclusion

Building local-first applications forces you to shift from thinking about request-response cycles to thinking about distributed systems, state convergence, and eventual consistency. By pairing IndexedDB for durable persistence with CRDTs for deterministic conflict resolution, you remove the burden of custom merge logic from your application layer.

Your users get an app that opens instantly, works everywhere regardless of Wi-Fi stability, and never loses their data. It takes careful upfront architecture, but once your sync engine is running smoothly, your sanity—and your users’ satisfaction—will be fully preserved.