Published on

Building Real-Time Collaboration Backends — CRDTs, OT, and the Sync Problem

Authors

Introduction

When Alice and Bob edit the same document simultaneously, their changes can collide. Traditional locking serializes edits (slow). Operational Transformation (OT) is complex to implement. CRDTs (Conflict-free Replicated Data Types) let each user edit locally, merge changes without coordination, and stay consistent.

Yjs is a battle-tested CRDT library. Build a backend around it, and you have a real-time collaboration platform that handles concurrent edits, offline work, and complex merging.

The Sync Problem (Multiple Users Editing Simultaneously)

Without conflict resolution:

  • Alice edits position 5 in a document
  • Bob edits position 10
  • Both send changes to the server
  • Who wins? Lastest-write-wins causes data loss

With locks:

  • Alice locks the document
  • Bob waits
  • Alice releases the lock
  • Bob edits
  • Slow and blocks real-time collaboration

Traditional databases can't handle concurrent edits from multiple sources without explicit coordination.

CRDTs solve it with locally-deterministic merging. Each edit has a unique identifier and a vector clock. Merge any two edits in any order and you get the same result.

Operational Transformation (OT) Concepts

OT adjusts operations based on concurrent edits. When Alice and Bob both insert at position 0:

  • Alice's insert becomes position 0
  • Bob's insert transforms to position 1 (because Alice inserted before it)

Implementing OT correctly is hard. Google Docs and Microsoft Office use OT, but the algorithms are complex and error-prone.

CRDTs for Conflict-Free Merging

A CRDT is a data structure where any two replicas merge to the same state, regardless of order. No coordination needed.

Yjs is a CRDT that models documents as Y.Map, Y.Array, and Y.Text. Each element has a unique ID and vector clock.

When Alice inserts "hello" at position 0 and Bob inserts "world" at position 0, both insertions get unique IDs. When merged, they appear in insertion-order, deterministically.

Key property: Commutativity. If two edits can reorder without changing outcome, they commute. CRDTs ensure all edits commute.

Yjs Library (CRDT Implementation)

Yjs models documents:

import * as Y from 'yjs';

const ydoc = new Y.Doc();
const ytext = ydoc.getText('shared-text');
const ymap = ydoc.getMap('config');
const yarray = ydoc.getArray('tasks');

// Edit locally
ytext.insert(0, 'Hello');
ymap.set('theme', 'dark');
yarray.push([{ id: 1, title: 'Task 1' }]);

// Get state as update
const state = Y.encodeStateAsUpdate(ydoc);

Send state to other clients. They apply it:

const remotState = ... // Received from network
Y.applyUpdate(ydoc, remoteState);

// Local state now includes remote changes
console.log(ytext.toString()); // "Hello" + remote edits

Yjs handles all conflict resolution. No special logic needed.

Building a Yjs Backend With y-websocket or y-redis

The simplest: y-websocket server in Node.js.

import http from 'http';
import WebSocket from 'ws';
import * as Y from 'yjs';
import { setupWSConnection, setPersistence } from 'y-websocket/bin/utils';

const server = http.createServer();
const wss = new WebSocket.Server({ server });

const docs = new Map<string, Y.Doc>();

function getDoc(docName: string): Y.Doc {
  if (!docs.has(docName)) {
    const ydoc = new Y.Doc();
    docs.set(docName, ydoc);
    setPersistence(ydoc, docName); // Persist to file
  }
  return docs.get(docName)!;
}

wss.on('connection', (ws, request) => {
  const docName = new URL(request.url || '', 'http://localhost').pathname;
  const ydoc = getDoc(docName);

  // y-websocket protocol
  setupWSConnection(ws, request, { docName, doc: ydoc });
});

server.listen(3000);

Client connects:

import { WebsocketProvider } from 'y-websocket';
import * as Y from 'yjs';

const ydoc = new Y.Doc();
const provider = new WebsocketProvider(
  'ws://localhost:3000',
  'my-doc',
  ydoc
);

const ytext = ydoc.getText('text');

// Bind to editor
ytext.observe((event) => {
  console.log('Text changed:', ytext.toString());
});

ytext.insert(0, 'Hello');

For scaling, use y-redis:

import redis from 'redis';
import * as Y from 'yjs';
import { RedisPersistence } from 'y-redis';

const client = redis.createClient();
const persistence = new RedisPersistence({
  client,
  prefix: 'yjs'
});

const ydoc = new Y.Doc();
persistence.bindState(ydoc);

// All writes go to Redis

Persistence of CRDT State

A persistent CRDT survives server restarts. Store the update log in a database:

// Save updates as they arrive
ws.on('message', async (update) => {
  await db.collection('updates').insertOne({
    docId: 'my-doc',
    update,
    timestamp: new Date(),
    clientId: ws.clientId
  });

  // Broadcast to all clients
  for (const client of wss.clients) {
    client.send(update);
  }
});

// On startup, replay updates
const updates = await db
  .collection('updates')
  .find({ docId: 'my-doc' })
  .sort({ timestamp: 1 })
  .toArray();

const ydoc = new Y.Doc();
for (const { update } of updates) {
  Y.applyUpdate(ydoc, update);
}

Replaying updates reconstructs state. Add snapshots to speed up loading large documents:

// Snapshot every 1000 updates
if (updateCount % 1000 === 0) {
  const state = Y.encodeStateAsUpdate(ydoc);
  await db.collection('snapshots').insertOne({
    docId: 'my-doc',
    state,
    updateNumber: updateCount
  });
}

// On startup, load latest snapshot + delta updates
const snapshot = await db
  .collection('snapshots')
  .findOne(
    { docId: 'my-doc' },
    { sort: { updateNumber: -1 } }
  );

if (snapshot) {
  Y.applyUpdate(ydoc, snapshot.state);
  const deltaUpdates = await db
    .collection('updates')
    .find({
      docId: 'my-doc',
      updateNumber: { $gt: snapshot.updateNumber }
    })
    .toArray();
  for (const { update } of deltaUpdates) {
    Y.applyUpdate(ydoc, update);
  }
}

Awareness (Cursor Positions, Presence)

Who's editing right now? Where are their cursors? Yjs Awareness tracks this without CRDT logic (awareness is ephemeral).

const awareness = ydoc.awareness;

// Set local state
awareness.setLocalState({
  user: {
    name: 'Alice',
    color: '#ff0000'
  },
  cursor: {
    anchor: { line: 0, ch: 5 },
    head: { line: 0, ch: 10 }
  }
});

// Listen for remote awareness changes
awareness.on('change', (changes) => {
  changes.forEach((clientId) => {
    const state = awareness.getStates().get(clientId);
    console.log('Client', clientId, ':', state);
  });
});

Broadcast awareness through WebSocket:

ws.on('message', (update, isBinary) => {
  if (isBinary && update[0] === 1) {
    // Awareness update
    Y.applyAwarenessUpdate(ydoc.awareness, Buffer.from(update.slice(1)));
  } else {
    // Document update
    Y.applyUpdate(ydoc, Buffer.from(update));
  }
});

Awareness updates are small and frequent. Send them separately from document updates for low latency.

Implementing Undo/Redo With CRDTs

CRDTs make undo/redo tricky: deleting and reinserting at the same ID causes conflicts.

Yjs solves this with UndoManager:

import { UndoManager } from 'yjs';

const undoManager = new UndoManager([ytext, ymap, yarray]);

// User edits
ytext.insert(0, 'Hello');

// User hits Ctrl+Z
undoManager.undo();
// "Hello" is removed

// User hits Ctrl+Y
undoManager.redo();
// "Hello" is restored

UndoManager tracks edit groups and separates them from document state. Undo reverts changes without affecting concurrent edits.

Integrating Yjs With Tiptap, CodeMirror, ProseMirror

Tiptap (built on ProseMirror):

import { useEditor, EditorContent } from '@tiptap/react';
import StarterKit from '@tiptap/starter-kit';
import Collaboration from '@tiptap/extension-collaboration';
import CollaborationCursor from '@tiptap/extension-collaboration-cursor';
import { WebsocketProvider } from 'y-websocket';
import * as Y from 'yjs';

const ydoc = new Y.Doc();
const provider = new WebsocketProvider(
  'ws://localhost:3000',
  'my-doc',
  ydoc
);

const yXmlFragment = ydoc.getXmlFragment('shared-doc');

export function Editor() {
  const editor = useEditor({
    extensions: [
      StarterKit,
      Collaboration.configure({
        document: ydoc
      }),
      CollaborationCursor.configure({
        provider,
        user: { name: 'Alice', color: '#ff0000' }
      })
    ],
    content: '<p>Start typing...</p>'
  });

  return <EditorContent editor={editor} />;
}

CodeMirror:

import { EditorView, basicSetup } from 'codemirror';
import { yCollab } from 'y-codemirror.next';
import { WebsocketProvider } from 'y-websocket';
import * as Y from 'yjs';

const ydoc = new Y.Doc();
const ytext = ydoc.getText('code');
const provider = new WebsocketProvider(
  'ws://localhost:3000',
  'my-doc',
  ydoc
);

const editor = new EditorView({
  doc: ytext.toString(),
  extensions: [basicSetup, yCollab(ytext, provider.awareness)],
  parent: document.body
});

Both handle cursor synchronization and real-time updates automatically.

Liveblocks as Managed Alternative

Liveblocks is a managed real-time collaboration platform. No ops, automatic scaling:

import { createClient } from '@liveblocks/client';
import { useStorage, useOthers } from '@liveblocks/react';

const client = createClient({
  publicApiKey: 'pk_...'
});

export function Room() {
  const storage = useStorage();
  const others = useOthers();

  return (
    <div>
      <h1>Collaborators: {others.length + 1}</h1>
      {others.map((user) => (
        <div key={user.connectionId}>
          {user.presence.cursor}
        </div>
      ))}
    </div>
  );
}

Liveblocks manages persistence, scaling, and conflict resolution. Pay per-user-month. Best for non-technical teams or when ops burden exceeds cost.

Scaling CRDT Backends

Single server handles ~1000 concurrent editors. For more:

  1. Shard by document: Document A runs on server 1, document B on server 2. Clients reconnect if their server dies (failover).

  2. Replicate state: All servers hold all documents. Updates broadcast via Redis pub/sub. High memory cost but instant failover.

const redisPub = redis.createClient();
const redisSub = redis.createClient();

wss.on('connection', (ws) => {
  ws.on('message', async (update) => {
    // Apply locally
    Y.applyUpdate(ydoc, update);

    // Broadcast to other servers
    await redisPub.publish('ydoc-updates', Buffer.from(update));
  });
});

await redisSub.subscribe('ydoc-updates', (update) => {
  Y.applyUpdate(ydoc, Buffer.from(update));
  // Broadcast to local clients
  for (const client of wss.clients) {
    client.send(update);
  }
});
  1. Use managed service: Liveblocks, Figma's multiplayer, or PartyKit handle scaling.

Checklist

  • Choose CRDT library (Yjs is default)
  • Design document structure (Y.Text, Y.Map, Y.Array)
  • Implement persistence layer (database + snapshots)
  • Set up WebSocket server for sync
  • Integrate with editor (Tiptap, CodeMirror, ProseMirror)
  • Implement awareness for cursor/presence
  • Add UndoManager for undo/redo
  • Test concurrent edits from multiple clients
  • Monitor connection churn and update throughput
  • Plan failover strategy

Conclusion

CRDTs eliminate the coordination overhead of real-time collaboration. Yjs is production-ready and integrates with every major editor. Build a backend around it, add persistence, and you have a platform supporting unlimited concurrent users with instant sync.

For teams building collaborative products, CRDTs are the future. Start with Yjs + WebSocket. Migrate to managed services when your ops burden exceeds the cost.

Real-time collaboration is now within reach of small teams.