System Design - Sharding

A technique to break up a big database into many smaller parts. Horizontal scaling means adding more machines, which is cheaper and more feasible. Vertical scaling means improves servers.

Sharding methods

Horizontal partitioning

Putting different rows into different DBs. i.e., based on alphabetically names. Unbalanced partitions.

Vertical partitioning

Divide tables based on features. I.e, for user or location. Also inbalanced partitions. Best if most of the time, only partial data in a row is required. But still might result int horizontal partition, which becomes very complicate.

Directory based partitioning:

Each row has a DB number, a map between row and DB number.

Sharding Criteria

Hash based: Using hashcode on entity value List Partioning: based on list, i.e., regions, countries, etc Round-Robin partitioning Composite partitioning

Challenges

ACID Joint inefficiency

Written on May 20, 2021