Wrap up

We had an in-depth discussion about consistent hashing, including why it is needed and how it works. The benefits of consistent hashing include:

• Minimized keys are redistributed when servers are added or removed.

• It is easy to scale horizontally because data are more evenly distributed.

• Mitigate hotspot key problem. Excessive access to a specific shard could cause server overload. Imagine data for Katy Perry, Justin Bieber, and Lady Gaga all end up on the same shard.

Consistent hashing helps to mitigate the problem by distributing the data more evenly. Consistent hashing is widely used in real-world systems, including some notable ones:

• Partitioning component of Amazon’s Dynamo database

• Data partitioning across the cluster in Apache Cassandra

• Discord chat application

• Akamai content delivery network

• Maglev network load balancer