Discord reported reducing its Cassandra fleet from 177 nodes to 72 after migrating to ScyllaDB. That is a substantial infrastructure change. But the more useful lesson for system designers is understanding the workload behind the database: popular channels concentrate reads, and concurrent requests can repeatedly ask for overlapping data.

Two problems emerge from that pattern: uneven demand and duplicated work.

1. Why a large database cluster can still have a hotspot

Distributed databases partition data so that storage and requests can be spread across a cluster. In a messaging system, however, traffic is rarely distributed evenly.

A small channel might see occasional reads. A popular channel can attract a steady stream of requests for recent messages, potentially concentrating demand on a much smaller part of the storage system.

This creates a distinction that matters in production: aggregate cluster capacity is not the same as capacity available to a hot partition.

Adding nodes can increase overall capacity, but it does not automatically eliminate a hotspot. The outcome depends on the partitioning scheme, access patterns, replication, and whether the additional resources can actually serve the concentrated workload.

This is why database sizing cannot be reduced to counting messages or estimating average requests per second. The distribution of traffic matters just as much.

2. Reducing database work before it reaches storage

Discord also described a request-coalescing layer written in Rust.

The underlying idea is simple, but powerful: when multiple concurrent requests need overlapping data, the system may be able to combine some of their database work instead of issuing an independent read for every request.

Consider a simplified example. Several clients request recent messages from the same busy channel at nearly the same time. If those requests overlap, independently fetching the same underlying data can create avoidable load.

A coalescing layer can coordinate compatible requests and share the result where appropriate. The application still needs to satisfy each request correctly, but the database may not need to perform the same work repeatedly.

This changes the optimisation target. Instead of asking only how to make database reads faster, engineers can also ask how many separate reads the system really needs to perform.

That distinction is useful well beyond chat applications. Any service with concurrent requests for overlapping data may benefit from examining duplicated work before adding storage capacity.

3. Why request coalescing is not a universal fix

Coalescing works best when requests overlap sufficiently to justify the coordination. If requests ask for unrelated data, there may be little work to combine.

It also introduces engineering concerns of its own: coordinating requests, managing timeouts and failures, handling cancellation, and ensuring that sharing work does not introduce unacceptable latency or correctness problems.

Nor does it automatically resolve every hot partition. If demand remains concentrated on a particular key or partition, engineers may still need to revisit data distribution, access patterns, or other parts of the architecture.

The important point is that these are different problems. A storage engine can improve how efficiently individual operations execute, while an application-level layer can reduce the number of operations that reach storage. Depending on the workload, both may matter.

4. What the Cassandra-to-ScyllaDB migration teaches us

A database migration should be evaluated in the context of the workload it serves—not as a universal ranking of database products.

Discord's reported reduction from 177 nodes to 72 is an infrastructure result worth examining, but node counts alone do not establish the exact dollar savings or prove that any single optimisation caused the change.

For engineers making similar decisions, the useful questions are more specific:

• Where does traffic concentrate, and which partitions become bottlenecks?

• How much database work is duplicated by concurrent requests?

• Which improvements come from the storage engine, and which come from application-level changes?

• How will the migration preserve correctness while moving existing data?

• Are the improvements visible in tail latency, resource utilisation, operational complexity, and total cost?

These questions lead to a better architecture decision than choosing a database based on headline benchmarks alone.

The full architecture breakdown

The details matter: how Discord's message storage evolved, why hot partitions are difficult, how request coalescing fits into the design, and what the migration tells us about scaling distributed systems.

I've covered the broader architecture and migration trade-offs in the full BuildOpsy analysis.

Read the full breakdown: How Discord Stores Trillions of Messages

Sources

• Discord Engineering — How Discord Stores Trillions of Messages

• ScyllaDB — How Discord Migrated Trillions of Messages from Cassandra to ScyllaDB

• InfoQ — Discord's Cassandra-to-ScyllaDB migration