Every real-time collaborative editor has the same hard problem underneath it. Two people type into the same paragraph at the same moment, possibly while one of them is offline on a train, and when everything reconnects both screens have to end up showing exactly the same text without anyone’s edit vanishing. The family of data structures built to solve that is called a CRDT (Conflict-free Replicated Data Type): a structure designed so that copies of a document (called replicas) can accept edits independently and always merge to the same result, in any order, with no central referee deciding who wins.
One of the most widely used CRDT libraries in the JavaScript world is Yjs. It sits under a lot of collaborative editors, and the format it uses to ship edits between replicas (its wire format, the exact bytes that go over the network, which Yjs calls document updates) has effectively become a standard of its own. The catch is that it’s a JavaScript library. If your backend is written in Go, joining a Yjs session has historically meant running a Node.js sidecar next to your service, or embedding a JavaScript engine, or calling into a Rust port through a foreign function bridge.
This week the Rabbit Hole Generator gave me ygo, a pure-Go port of Yjs that makes a strong, specific, testable claim: the bytes it produces are byte-for-byte identical to what Yjs produces. That’s the kind of claim I like for this series, because it’s not a vibe. Either the bytes match or they don’t.
What is ygo used for in collaborative application backends?
ygo is a native Go implementation of the Yjs CRDT, including its shared types (maps, arrays, rich text, and XML for editors like ProseMirror), undo/redo, snapshots, and the sync protocol that Yjs clients speak over WebSockets (a long-lived, two-way connection between browser and server). Its purpose is to let Go services, CLI tools, bots, and mobile apps participate directly in an existing Yjs deployment without a Node.js runtime anywhere in the path; for mobile, Go’s gomobile tool packages Go code as an iOS or Android library. It also ships yserve, a single static binary meant as a drop-in replacement for Hocuspocus, a popular Node.js server for Yjs, with SQLite persistence built in. It’s MIT-licensed, at v1.25.0 as I write this, and listed in the Ports section of the Yjs README.
The Architecture: compatibility as the product
Most CRDT libraries compete on algorithms: how fast they merge, how small their history gets, how gracefully they handle a pathological editing trace. ygo deliberately doesn’t. Its README is blunt about the non-goals: no other CRDT designs, no C interface, no attempt to be a JavaScript runtime. The one thing it promises is that a Go replica is indistinguishable on the wire from a JavaScript one. A browser running stock Yjs can sync with a ygo server and never know.
That’s a harder promise than it sounds, because “compatible” in CRDT land usually means “eventually converges to the same visible state.” ygo is promising something much stricter: the encoded update bytes themselves match. To get there it has to reproduce not just the merge algorithm (Yjs uses one called YATA, which decides the order of concurrent inserts at the same position; the Yjs internals doc walks through it) but every encoding detail: how integers are packed into variable-length bytes (varints, 7 bits of number per byte, the same idea Protocol Buffers uses), how deleted content is compacted, how runs of single-character edits get squashed into one block, and how each replica’s client ID is written. Yjs actually has two encodings of the same data, the original “V1” format and a more compact “V2”; ygo claims both. A mismatch in any of those details still converges fine in a demo and then quietly breaks a byte-level cache, a deduplication step, or a signature somewhere downstream.
The diagram below is what “byte-for-byte” means in practice, and what it buys you. The top half is what I tested in the lab; the bottom half is the deployment the claim exists to enable.
The way the project keeps that promise honest is a fixture suite, not a design document. The README describes 273 cross-language scenarios generated from a pinned [email protected]: JavaScript encodes, Go decodes and compares, and in the other direction Go encodes and JavaScript decodes. Those fixtures are regenerated on every CI run, and a byte-level diff fails the build. The project also documents its known exceptions instead of hiding them: ygo sorts the keys of multi-key objects where Yjs keeps insertion order, and it writes whole-number floats differently. Byte-for-byte is the rule, with a short, published list of places where it isn’t.
The Reality Check: where this breaks
That’s the pitch: a Go replica you can drop into a Yjs world without anyone noticing. Every week, the Rabbit Hole Generator also writes a critique of its pick: where the tool is likely to struggle under load, which assumptions its design quietly depends on, and what security exposure it brings with it. This section goes through that critique point by point, because knowing where a tool breaks is what tells you where to trust it.
The Rabbit Hole Generator’s critique of ygo raised six distinct concerns. Each numbered heading below is one of them, in three groups:
- Points 1 to 3 are about load. They describe busy documents, chatty presence traffic, and thousands of rooms. I didn’t test these: they need many concurrent clients hammering a running server for a sustained period, and a 30-minute lab on one laptop can’t reproduce that honestly. ygo ships a load generator,
yload, which is the right tool if you want to; I’ve described what each point means and where the project has already put guardrails. - Points 4 and 5 are assumptions the design leans on. Point 4 (pinning to one Yjs version) I ran into directly, because the version had already moved since the Rabbit Hole Generator wrote this week up. Point 5 is about ecosystem fit rather than behavior, so there was nothing to measure.
- Point 6 is the security exposure. I tested the most specific part of it, malformed and hostile input, in lab step 4.
1. Hot documents with many concurrent writers
When dozens of people hammer the same region of the same document, the server has to repeatedly decode incoming updates, integrate them against each other, garbage-collect deleted content, and re-encode state for newcomers, all inside a latency budget that users feel as typing lag. A correct implementation can still fall over here. Correctness and throughput under contention are separate properties, and the fixture suite only proves the first one.
2. Presence traffic is a different animal from document traffic
Yjs has a side channel called awareness: cursor positions, selections, “Scott is typing” indicators. It’s ephemeral and bursty, it never gets persisted, and in a busy room it can easily outnumber real edits. It still competes for the same CPU, memory, and WebSocket write slots as durable document changes. ygo does ship caps for this (a per-room limit on awareness clients, bounded broadcast write timeouts, periodic sweeping of stale presence), which tells me the authors have hit it, but caps are a mitigation you have to tune, not a solution you get for free.
3. SQLite as the bottleneck at multi-tenant scale
yserve’s single-binary simplicity comes from storing every document’s update log and its periodic versions in SQLite, a database that lives in a single file inside the server process. That’s a great default for one team’s wiki. With thousands of active rooms, uneven write volume, and long-lived connections, a single embedded database becomes the coordination point everything queues behind. The project’s answer is a pluggable store interface and a NATS-based backplane (NATS is a lightweight message bus) for clustering multiple server instances, which means “single binary” is the starting point, not the shape you’d run at scale.
4. Pinning to one Yjs version assumes the target holds still (ran into it)
Byte-for-byte compatibility is defined against a specific release. When the Rabbit Hole Generator wrote up this week in June, it named [email protected] as the target; the README today says 13.6.33. That’s a small, healthy bump, and it’s also the point: the target moves, and fixtures only capture the behavior someone thought to write a scenario for. The README already tracks the wire change coming in Yjs 14 (wider client IDs) and notes that full support “waits for the v14 format to stabilize.” Anyone depending on exact bytes is depending on that tracking staying current.
So what, and how to check it yourself: if you only need replicas to converge (same visible document), version drift is mostly harmless. If anything you build depends on the bytes (caching, deduplication, signing updates), pin both versions explicitly and run a cross-language comparison like lab step 3 in your own CI whenever either side is upgraded.
5. Language purity versus ecosystem maturity
The pitch is “no Node in your stack.” But teams running Hocuspocus in production often lean on its extension ecosystem: auth hooks, webhooks, logging and metrics integrations, years of operational experience. ygo replicates the protocol and a solid set of lifecycle hooks, but protocol compatibility isn’t plugin compatibility. If your deployment’s real value lives in those extensions, removing Node removes them too.
6. Introduced Vulnerability: untrusted bytes, real work (tested)
A Yjs server, by design, accepts binary updates and awareness messages from clients and does real work on them: decode, allocate, merge, persist. Much of that can happen before anything has established whether the sender should be trusted, and a carefully shaped payload can try to make that work expensive, a denial-of-service pattern. The classic version is a length-prefix amplification attack: a few bytes that claim “a trillion entries follow,” tricking a naive decoder into allocating memory for all of them up front. ygo’s README says every wire-supplied count is bounded against the actual input length and that the decoders are fuzzed (bombarded with automatically mutated random inputs to find crashes) nightly in CI. I tested the amplification case directly in lab step 4.
It’s worth being clear about scope, though. Bounding counts doesn’t stop a valid but enormous update, and it doesn’t stop a client that is allowed in from deleting content. The project’s own Matrix integration docs say it plainly: Yjs updates don’t authenticate deletes, so write access to the channel is write access to the document.
The 30-Minute Lab: do the bytes actually match?
The Rabbit Hole Generator’s challenge was a small Go program that edits a document, copies its state to a second replica, exercises undo and redo, and prints SHA-256 hashes of the encoded bytes. I ran it as written, then extended it into the test the README’s claim actually invites: do the same edits in real Yjs and compare the bytes. That covers Reality Check 4 directly. A final step probes Reality Check 6. Everything ran in throwaway Docker containers (golang:1.26 for Go and node:22-slim for JavaScript) per this series’ usual rule of not installing toolchains on my own machine.
1. Running the generated lab, and the first two surprises
docker run --rm -v "$PWD":/lab -w /lab golang:1.26 sh -c \
'go mod init ygo-lab && go get github.com/Deln0r/ygo@latest && go run .'
The first surprise was a compile error, not a runtime one:
./main.go:51:21: cannot use len("hello CRDT") (constant 10 of type int) as uint64 value in argument to text.Insert
Text.Insert takes its position as a uint64 in the current release, and Go won’t silently convert an int from len() into one. Wrapping it as uint64(len("hello CRDT")) fixes it. Small, but it’s the same pattern this series keeps running into: a generated exercise is a snapshot of an API that has kept moving (ygo shipped four minor releases, 1.22 through 1.25, in the first two weeks of October alone).
With that fixed, the program ran, and the second surprise was in the output:
DST_THEME= dark
DST_LANG= go
DST_TEXT= hello CRDT
SRC_THEME_AFTER_EDIT= light
SRC_TEXT_AFTER_EDIT= hello CRDT + undo
SRC_THEME_AFTER_UNDO= <nil>
SRC_TEXT_AFTER_UNDO=
SRC_THEME_AFTER_REDO= light
SRC_TEXT_AFTER_REDO= hello CRDT + undo
The DST_ lines are the second replica, the SRC_ lines the original. The replication part is exactly right: a fresh replica that applied the encoded bytes has dark, go, and hello CRDT. But the Rabbit Hole Generator’s challenge predicted that one undo would bring the theme back to dark. Instead, one undo wiped out everything: the theme is gone entirely (<nil>) and the text is empty.
That’s not a bug; it’s how Yjs’s undo manager is designed, and ygo matches it. Undo groups edits that happen close together in time (500 milliseconds by default, the captureTimeout setting) into a single step, the same way your editor doesn’t make you press Ctrl+Z once per keystroke. The program’s two transactions ran microseconds apart, so they became one undo step, and undoing it reverted both. The fix is one line, um.StopCapturing(), which tells the undo manager “start a new step here.” It’s a good reminder that “undo” in collaborative software is a policy decision, not just a stack.
The third issue was in the hashes. The challenge’s optional step asked me to check that the hash is stable across reruns. It isn’t:
run 1: FINAL_SHA256= fcb11fdfd9e42e84...
run 2: FINAL_SHA256= 4028d9a588197460...
Same operations, different bytes every run. The reason is the client ID: every replica gets a random identifier at creation (Yjs documents it on Y.Doc), and every edit it makes is stamped with that ID so concurrent edits from different replicas can be told apart and ordered. Different random ID, different bytes. That isn’t a determinism bug either; it’s the CRDT doing its job. But it does mean “compare the hashes” is meaningless until you pin the ID.
2. Making it deterministic
ygo exposes the client ID as an option, with a comment saying to fix it only for deterministic tests (in production, two replicas sharing an ID would corrupt each other’s history). Yjs lets you set doc.clientID the same way. So the corrected Go side became:
src := ygo.NewDocWithOptions(ygo.Options{ClientID: 424242})
settings := ygo.NewMap(src, "settings")
text := ygo.NewText(src, "body")
um := ygo.NewUndoManager(src, settings, text)
txn := src.WriteTxn()
settings.Set(txn, "theme", "dark")
settings.Set(txn, "lang", "go")
text.Insert(txn, 0, "hello CRDT")
txn.Commit()
// write EncodeStateAsUpdate(src) to go-step1.bin
um.StopCapturing() // new undo step from here
txn2 := src.WriteTxn()
text.Insert(txn2, 10, " + undo")
settings.Set(txn2, "theme", "light")
txn2.Commit()
// write EncodeStateAsUpdate(src) to go-step2.bin
EncodeStateAsUpdate serializes the replica’s entire state into one update, in the V1 format. Two runs, identical output, and undo now behaves the way the challenge expected:
STEP1 len=70 sha256=542ef08c3cbcadec
STEP2 len=96 sha256=7950d34c2dc30055
AFTER_UNDO theme=dark text="hello CRDT"
AFTER_REDO theme=light text="hello CRDT + undo"
3. The actual test: same edits in real Yjs
Then the part that tests the claim. A Node.js script using [email protected] (the exact version ygo’s fixtures pin) performs the same two transactions with the same client ID, encodes the state after each, and compares its bytes to the files the Go program wrote. It compares with Node’s Buffer.compare, which returns 0 only when two byte buffers are identical, rather than trusting shortened hashes. It also loads Go’s bytes into a fresh JavaScript replica to check the reverse direction:
import * as Y from 'yjs'
import fs from 'fs'
import crypto from 'crypto'
const h = b => crypto.createHash('sha256').update(b).digest('hex').slice(0,16)
const doc = new Y.Doc(); doc.clientID = 424242
const settings = doc.getMap('settings'), text = doc.getText('body')
const um = new Y.UndoManager([settings, text])
doc.transact(() => { settings.set('theme','dark'); settings.set('lang','go'); text.insert(0,'hello CRDT') })
const s1 = Y.encodeStateAsUpdate(doc)
um.stopCapturing()
doc.transact(() => { text.insert(10,' + undo'); settings.set('theme','light') })
const s2 = Y.encodeStateAsUpdate(doc)
const g1 = fs.readFileSync('go-step1.bin'), g2 = fs.readFileSync('go-step2.bin')
console.log(`yjs ${JSON.parse(fs.readFileSync('node_modules/yjs/package.json')).version}`)
console.log(`STEP1 js len=${s1.length} sha256=${h(s1)} identical_to_go=${Buffer.compare(Buffer.from(s1), g1)===0}`)
console.log(`STEP2 js len=${s2.length} sha256=${h(s2)} identical_to_go=${Buffer.compare(Buffer.from(s2), g2)===0}`)
// Go -> JS: a fresh yjs doc reads the bytes ygo wrote.
const peer = new Y.Doc(); Y.applyUpdate(peer, g2)
console.log(`JS_READS_GO theme=${peer.getMap('settings').get('theme')} lang=${peer.getMap('settings').get('lang')} text=${JSON.stringify(peer.getText('body').toString())}`)
My first run was a version of that script without the UndoManager line:
yjs 13.6.33
STEP1 js len=70 sha256=542ef08c3cbcadec identical_to_go=true
STEP2 js len=90 sha256=115f6e5fda68e3a7 identical_to_go=false
Step one matched exactly. Step two didn’t: 90 bytes from JavaScript, 96 from Go. It would have been easy to stop there and write “byte-for-byte compatibility breaks on the second edit.” It would also have been wrong.
The difference was my test, not ygo. My Go program had an undo manager attached; that JavaScript script didn’t. When the second transaction overwrote theme from dark to light, the old dark value became deleted content. Normally that content is garbage-collected (thrown away, leaving just a small marker that something was deleted; the gc option on Y.Doc controls it). But an undo manager needs that old value to be able to restore it, so both Yjs and ygo keep it around while an undo manager is watching. Go kept it; JavaScript, with nothing watching, threw it away. Six bytes of difference, explained entirely by retained undo history.
With the undo manager (and the same stopCapturing() boundary) on the JavaScript side, as in the script above:
yjs 13.6.33
STEP1 js len=70 sha256=542ef08c3cbcadec identical_to_go=true
STEP2 js len=96 sha256=7950d34c2dc30055 identical_to_go=true
JS_READS_GO theme=light lang=go text="hello CRDT + undo"
Identical, both steps, compared on the raw bytes. And the reverse direction works: a stock Yjs replica that loaded ygo’s bytes sees light, go, and the full text.
What’s worth taking from this beyond “it passed”: the encoded bytes are a function of the whole replica state, not just the visible content. Client ID and retained history both change them, even when the document looks identical. If you’re ever comparing CRDT output across implementations (or caching, deduplicating, or signing it), control every one of those inputs first, or you’ll either chase a bug that isn’t there or, worse, miss one that is. It’s the same instinct as re-computing the hash yourself in the Tejolote lab: a mismatch is information, but only once you’ve made sure you’re comparing like with like.
4. Feeding it garbage on purpose
Last, Reality Check 6. The question: if someone sends ygo bytes that are broken or deliberately hostile, does it reject them cheaply, or can a few bytes make it do a lot of work? I fed two functions a set of payloads and measured time and memory for each. ValidateUpdate is a cheap decode-only check; ApplyUpdate is the real thing, merging into a fresh document. The payloads: a valid update, a truncated one, random text, and two hand-built amplification payloads, one whose first varint claims 2^50 clients in 10 bytes, and one claiming 2^40 structs (Yjs’s internal unit of content) for a single client.
valid update 96 bytes 22µs alloc= 9.9KB
validate: <nil>
apply: <nil>
truncated (first 40 bytes) 40 bytes 30µs alloc= 3.6KB
validate: ygo: invalid update: lib0: truncated input
apply: ApplyUpdate decode: lib0: truncated input
random garbage 31 bytes 5µs alloc= 2.3KB
validate: ygo: invalid update: lib0: truncated input
apply: ApplyUpdate decode: lib0: truncated input
claims 2^50 clients 10 bytes 4µs alloc= 2.3KB
validate: ygo: invalid update: lib0: truncated input
apply: ApplyUpdate decode: lib0: truncated input
claims 2^40 structs 9 bytes 4µs alloc= 2.6KB
validate: ygo: invalid update: lib0: truncated input
apply: ApplyUpdate decode: lib0: truncated input
(lib0 in the errors is the low-level encoding library Yjs is built on, which ygo ports too.) The column to look at is alloc, the total memory Go allocated during each probe. A decoder vulnerable to amplification would show that number exploding for the last two rows, or the container getting killed. Instead, both hostile payloads were rejected in microseconds with about the same allocation as random noise, because the decoder noticed that 10 bytes can’t possibly hold 2^50 entries and stopped before allocating. The error message is unexciting, truncated input, and that’s the good outcome: boring rejection, no crash, no work done.
So what, and what this doesn’t cover. The decoder itself is in good shape against malformed input. But that’s the narrowest slice of Reality Check 6, and a server needs the other slices too. Reading yserve’s options and docs:
- Message size is capped by default.
yserve -read-limitlimits each WebSocket message (default 32,768 bytes), which bounds the “valid but enormous update” case. Raising it for big documents widens that window again, so raise it deliberately. - Some caps are off by default.
-max-conns-per-docand-max-awareness-clientsdefault to 4,096, but-max-docsand-max-connsdefault to unlimited. On anything internet-facing, set them. - Authentication isn’t a flag. The stock
yservebinary has no auth option; authentication plugs in through theOnAuthenticatehook when you embed ygo’s server package in your own Go program, with aReadOnlyoption for viewers. The project’s own tech-debt notes say per-document permissions decided from the auth token aren’t there yet. So “who may write to this document” is your code’s job, and since Yjs doesn’t authenticate deletes, it’s the control that matters most.
To verify those in your own setup, rather than taking my reading of the docs on faith: run yserve with your intended flags, push an oversized message and confirm it’s refused, and use yload to see how latency behaves at your expected room and connection counts (Reality Checks 1 to 3). I didn’t run those in this lab; they’re the next 30 minutes, not this one.
Where this fits in a real environment
ygo is a natural fit when you already have Yjs clients and the piece you want to add is Go: a backend service that needs to read or edit collaborative documents directly, a bot that annotates a shared doc, a CLI that exports one, or a mobile app that wants native sync without a JavaScript bridge. For those cases, lab step 3 is a fairly convincing argument. The bytes really are the same, in both directions, once you control the inputs.
It stops being an easy call at the operational edges the Reality Check names. If you’d be swapping out a Hocuspocus deployment that leans on its extension ecosystem, point 5 is the real cost, not the protocol. If you’re planning thousands of rooms, point 3 means you’re designing a store and a clustering setup, not just running one binary. If the server faces untrusted clients, lab step 4’s caps and the auth hook are your work to do, not defaults. And because exact-bytes compatibility is defined against a moving release (point 4), anything that depends on bytes rather than converged state, such as caching or signing updates, should pin versions on both sides and test the pair the way I did above.
Actionable Takeaways
- Pin the client ID before comparing CRDT bytes. Random replica IDs make every run’s encoding different even for identical edits; fix them for tests and never in production.
- Match the whole replica setup, not just the edits. An attached undo manager keeps deleted content alive and changes the bytes, so a comparison without the same setup on both sides proves nothing either way.
- Treat undo grouping as a policy you choose. Edits inside the capture window collapse into one step; call
StopCapturing()where your users would expect a boundary. - Put auth and caps in front of a Yjs server yourself. A hardened decoder stops malformed bytes, not authorized clients deleting content or unbounded rooms and connections.
Compatibility claims are only as strong as the comparison you run against them, and the most useful thing this lab did was fail once for a reason that turned out to be mine.
